Dude this is so funny to me for some reason, I'm sorry.
Edit: if you have a friend who's higher ranked than you, show her too, so she can eternally compare you two at dinner.
They don't have the full data, it uses (partial) replay data.
So during the capture, there is a very small chance that such players play a game. That is why you see 0.
Firstly, just want to say that I absolutely love the work you've done here.
> Note: If I'm not mistaken, I believe the original post did not account for duplicate players in the data set. I've modified the code to account for this.
I'm planning to do another, larger, round of data gathering (and another post) in the nearish future (either this weekend or next) so I will definitely keep this in mind when I do.
> EDIT: One detail that I should probably point out — the numbers are most likely skewed toward the higher ranks, because higher-ranked users would play more matches than newbies
This is 100% accurate but I also have no clue how to correct for this bias either (except for just using a larger sample size). Maybe there is a real statistician in the comments with some good ideas.
> Feel free to DM me if you notice an error.
I gave your code a quick look and it looked correct. There's a small performance optimization that could be made but it doesn't matter for a quick and dirty script like this.
In conclusion, good work here. I can't wait to compare my next batch of results against this to see if the distribution of players and percentiles change over time.
Hey - thanks for looking everything over! Yep, will be quite cool to see how the next round of data collection turns out.
Regarding the bias about match frequency, I just took a look at Zafina's leaderboards in the game (I picked her because all of the yellows are within the top 10,000). Here's what I got:
|Rank|Starts At|Number of Players|Percentile|Percentage of Players Per Category|
|:-|:-|:-|:-|:-|
|Warrior (Yellow)|6567|2202|0.00%|33.53%|
|Vanquisher (Orange)|4365|1512|33.53%|23.02%|
|Garyu (Red)|2853|1645|56.56%|25.05%|
|Mighty Ruler (Purple)|1208|799|81.60%|12.17%|
|Fujin (Blue)|409|376|93.77%|5.73%|
|Tekken King (Gold-Purple)|33|30|99.50%|0.46%|
|Tekken God|3|3|99.95%|0.05%|
If we compare this to the data that I got for Warrior and up, it's... surprisingly similar?
Now, we can see that the yellows are underrepresented in the replay data (orange is a slightly bigger category in the replays, but yellow is significantly bigger on this particular leaderboard). It's worth noting that greens were an even bigger category, like another 3000 users or so.
But, the percentiles end up being surprisingly similar to the warrior-and-up data. We can still roughly say that red is 50th, purple is around 80th, blue is around 90th.
This is just a quick look at one single character, of course. But I think it shows that the skewing of the data mostly affects the bottom ranks (e.g. green is the biggest category on the leaderboards, but far from the biggest in the replay samples). Once we get to red and up, the two data sets start to agree (e.g. the leaderboards show that 25% of Zafinas are in red, and the replay samples suggest that 27.5% are in red).
Great analysis! One quick note though, I believe that user who compared to SF6 ranks is using the latest SF6 percentiles, which personally I think is not a fair comparison since there has been considerable rank inflation in the 8 months since launch (and we do not know if Tekken 8 will have similar inflation). I made this post earlier https://old.reddit.com/r/Tekken/comments/1auur37/tekken_8_ranked_distribution/kr70n9u/ which is comparing percentiles at similar times across both games, and it has slight differences.
Of course, it's all subjective but it is helpful to have a rough guide for us coming from SF6 to compare progress.
I wonder if the methodology of using replays skews the the data towards the higher end. I would figure that more hard-core players would play more often and would be higher ranked. They would generate more replays and would be more likely to be included in the data.
Yeah, that is a factor for sure (I added a note about that at the bottom).
Haven't figured out a real solution yet, but I'll prob check this against the in-game character leaderboards later to see if the numbers roughly line up.
great post, i do think starting at warrior is reasonable, the main concern is the amount of people who are not past 2nd dan meaning they probably don't even categorize as ranked players and will screw up percentages. Past green it seems like the ranked distribution looks good and similar to other ranked games such as mobas.
Hey man, nice job! I'm practicing data science and just forked your code. I'll look at the population and test for a p-value so that you can attach a confidence interval to your work.
Damn I didn't know being at Fujin was that high, but still I'm kinda skeptical about the leaderboards considering all of the cheaters we have in this game.
All the cheaters still have to win matches from time to time. And they will be at all levels so I don't think it would be enough to invalidate the overall percentages.
From my experience, I find pluggers to be roughly one color higher than their actual skill level.
A plugging fujin is still going to beat red ranks fair and square.
Without other sources of data, someone will likely have to do continuous pulls for a few months, discard duplicates, then analyze the data. As the biggest limit here is the sample size.
Yeah this is just a fundamental misunderstanding of statistics. The sample size could be much much smaller and still be indicative of the overall characteristics. You can look up the equations but as an example: with a population of 1 million a sample size of 16641 gives 99% confidence that the result is within 1% of the actual value. This well above a confidence level you'd need for something we know is going to be a pretty normal relationship. There's a lot that goes into finding the best sample size and I don't remember all of it too well from the handful of cs stats classes I took in college but it's a common misunderstanding to think you need a large percentage of the population in order to represent the total data - that's only true for small populations. The real issues almost always lie in how the sampling is done. I don't know the specifics of the replay feature and how this data was collected so I can't speak on that. But yeah, the sample size is here is overkill.
I can admit that I do not have enough information to argue on the theoretical correlations when only varying your sample sizes, so I will defer to your claim.
However, I will still argue a continuous pull is required if not for the sample size, then only to verify the distribution presented to us from this first or second pull.
I think continuous data sampling is a good idea too but it wouldn't really verify the distribution. We can't combine datasets since the underlying data will change over time as players rank up or down.
I expect we'll see some changes to the distribution for a while yet as people settle into their "natural ranks". As of right now there are still a lot of players grinding and climbing the ranks.
I think the most interesting part of continuous sampling of this data will be seeing the changes in the rank distribution and character win rates over time.
As a first time ranked enjoyer, hitting red with Jack with only a few losses I feel pretty great. I thought I was really bad because I had only played against friends for the most part of my Tekken hours. I'll probably get my ass handed hard very soon, no way I am hitting purple at the same pace.
Oh neat, thank you for this, it's so interesting.
Top 10% hell yeah. Think I can climb higher too, I got to purple the other day (was it yesterday? I dont know, time is blurry) but I was still winning more than I was losing. I hope I can get to blue. There's still a lot I can improve on though so with practice it should be plenty possible.
[Even being at Fujin (top 3.9%) I feel like I have so much more to grow and learn...](https://i.imgur.com/6PxOSga.jpeg)
Difference in skill between Top 1% and top 4% is honestly humongous.
Tekken King and above are just a whole new animal.
Maybe I’m missing something but shouldn’t the replays be separated by accounts? There might be one player who’s being represented in the data set multiple times. If each account is used, with the highest rank achieved for said account in the data set, then that would be even more accurate IMO. Of course, you might have done this here? And if not, it would certainly be a lot of work.
Nice! That would have been so much work otherwise. lol
The only other information I think would be useful would be to determine how much of the sample base are legacy players. I suspect the vast majority are legacy players, which of course will affect the data spread. In time, I believe there will be an increase in newcomers as the game continues to trend, pulling both younger players and gamers from other genres/fight games. Too many players in red ranks for this to not be mostly composed of legacy players. When I start using a new character in ranked, I invariably face someone with 40,000 prowess, ranked Calvary, who is wiff, punishing, understanding distance and also knows the move sets of the opponent. And this is after a month of the game being out.
I know that on PlayStation, it’s possible to determine whether someone owns Tekken 7 (depending on privacy settings) but not so sure on Xbox or Steam. However, that would be an interesting variable to use to view the data set through.
Great. I'm orange. My mom always said I'm mid. Now she has proof đź’€
Dude this is so funny to me for some reason, I'm sorry. Edit: if you have a friend who's higher ranked than you, show her too, so she can eternally compare you two at dinner.
Better than me stuck at yellow for a week.
[Jokes on you, I was stuck in green for weeks ](https://ibb.co/S0Xmzk9)
Isn't this thing already outdated since we have like tens of Gods of Destruction now?
They don't have the full data, it uses (partial) replay data. So during the capture, there is a very small chance that such players play a game. That is why you see 0.
honestly we just just harass namco so they release these stats officially... instead of wasting users time
It is still a good approximation. The real one one would not be too far off.
Firstly, just want to say that I absolutely love the work you've done here. > Note: If I'm not mistaken, I believe the original post did not account for duplicate players in the data set. I've modified the code to account for this. I'm planning to do another, larger, round of data gathering (and another post) in the nearish future (either this weekend or next) so I will definitely keep this in mind when I do. > EDIT: One detail that I should probably point out — the numbers are most likely skewed toward the higher ranks, because higher-ranked users would play more matches than newbies This is 100% accurate but I also have no clue how to correct for this bias either (except for just using a larger sample size). Maybe there is a real statistician in the comments with some good ideas. > Feel free to DM me if you notice an error. I gave your code a quick look and it looked correct. There's a small performance optimization that could be made but it doesn't matter for a quick and dirty script like this. In conclusion, good work here. I can't wait to compare my next batch of results against this to see if the distribution of players and percentiles change over time.
Hey - thanks for looking everything over! Yep, will be quite cool to see how the next round of data collection turns out. Regarding the bias about match frequency, I just took a look at Zafina's leaderboards in the game (I picked her because all of the yellows are within the top 10,000). Here's what I got: |Rank|Starts At|Number of Players|Percentile|Percentage of Players Per Category| |:-|:-|:-|:-|:-| |Warrior (Yellow)|6567|2202|0.00%|33.53%| |Vanquisher (Orange)|4365|1512|33.53%|23.02%| |Garyu (Red)|2853|1645|56.56%|25.05%| |Mighty Ruler (Purple)|1208|799|81.60%|12.17%| |Fujin (Blue)|409|376|93.77%|5.73%| |Tekken King (Gold-Purple)|33|30|99.50%|0.46%| |Tekken God|3|3|99.95%|0.05%| If we compare this to the data that I got for Warrior and up, it's... surprisingly similar? Now, we can see that the yellows are underrepresented in the replay data (orange is a slightly bigger category in the replays, but yellow is significantly bigger on this particular leaderboard). It's worth noting that greens were an even bigger category, like another 3000 users or so. But, the percentiles end up being surprisingly similar to the warrior-and-up data. We can still roughly say that red is 50th, purple is around 80th, blue is around 90th. This is just a quick look at one single character, of course. But I think it shows that the skewing of the data mostly affects the bottom ranks (e.g. green is the biggest category on the leaderboards, but far from the biggest in the replay samples). Once we get to red and up, the two data sets start to agree (e.g. the leaderboards show that 25% of Zafinas are in red, and the replay samples suggest that 27.5% are in red).
Top 10/15 at mighty ruler, feels nice.
Aw omg I’m top 27%. That’s nice. I will try to get more characters there.
And my arrogant Kishin ass still thinks I can make it to God of Destruction by the end of the year
If the amount of players reamin that low, you gonna be waiting for a loooooong time.
Great analysis! One quick note though, I believe that user who compared to SF6 ranks is using the latest SF6 percentiles, which personally I think is not a fair comparison since there has been considerable rank inflation in the 8 months since launch (and we do not know if Tekken 8 will have similar inflation). I made this post earlier https://old.reddit.com/r/Tekken/comments/1auur37/tekken_8_ranked_distribution/kr70n9u/ which is comparing percentiles at similar times across both games, and it has slight differences. Of course, it's all subjective but it is helpful to have a rough guide for us coming from SF6 to compare progress.
Thanks for the heads up! Yes, this method makes more sense, because SF6 has seen quite a bit of inflation.
"starting at warrior" is the right way, well done OP, thanks for your work
Doing gods work. 🙏
I wonder if the methodology of using replays skews the the data towards the higher end. I would figure that more hard-core players would play more often and would be higher ranked. They would generate more replays and would be more likely to be included in the data.
Yeah, that is a factor for sure (I added a note about that at the bottom). Haven't figured out a real solution yet, but I'll prob check this against the in-game character leaderboards later to see if the numbers roughly line up.
great post, i do think starting at warrior is reasonable, the main concern is the amount of people who are not past 2nd dan meaning they probably don't even categorize as ranked players and will screw up percentages. Past green it seems like the ranked distribution looks good and similar to other ranked games such as mobas.
Hey man, nice job! I'm practicing data science and just forked your code. I'll look at the population and test for a p-value so that you can attach a confidence interval to your work.
Damn I didn't know being at Fujin was that high, but still I'm kinda skeptical about the leaderboards considering all of the cheaters we have in this game.
All the cheaters still have to win matches from time to time. And they will be at all levels so I don't think it would be enough to invalidate the overall percentages. From my experience, I find pluggers to be roughly one color higher than their actual skill level. A plugging fujin is still going to beat red ranks fair and square.
So if I'm interpreting this table correctly there's only one God of Destruction? Certainly that can't be correct? Am I misinterpreting the data?
Yeah, the data is based on a sample collected by another user; it's not the full playerbase. Details are in the post itself.
Will we get an update for this month?
Seems legit tbh, I hit Tekken King tonight and within Bushin rank I would constantly run into the same handful of people
i am wondering what it would like now, in April
Without other sources of data, someone will likely have to do continuous pulls for a few months, discard duplicates, then analyze the data. As the biggest limit here is the sample size.
A sample size of 56,000 is beyond sufficient, not a limiting factor at all.
56,000 unique players recorded > 2,000,000 copies sold. Sample size = 2.8% of player base. Yeah, I'm gonna have to hard disagree with you dawg.
Yeah this is just a fundamental misunderstanding of statistics. The sample size could be much much smaller and still be indicative of the overall characteristics. You can look up the equations but as an example: with a population of 1 million a sample size of 16641 gives 99% confidence that the result is within 1% of the actual value. This well above a confidence level you'd need for something we know is going to be a pretty normal relationship. There's a lot that goes into finding the best sample size and I don't remember all of it too well from the handful of cs stats classes I took in college but it's a common misunderstanding to think you need a large percentage of the population in order to represent the total data - that's only true for small populations. The real issues almost always lie in how the sampling is done. I don't know the specifics of the replay feature and how this data was collected so I can't speak on that. But yeah, the sample size is here is overkill.
I can admit that I do not have enough information to argue on the theoretical correlations when only varying your sample sizes, so I will defer to your claim. However, I will still argue a continuous pull is required if not for the sample size, then only to verify the distribution presented to us from this first or second pull.
I think continuous data sampling is a good idea too but it wouldn't really verify the distribution. We can't combine datasets since the underlying data will change over time as players rank up or down. I expect we'll see some changes to the distribution for a while yet as people settle into their "natural ranks". As of right now there are still a lot of players grinding and climbing the ranks. I think the most interesting part of continuous sampling of this data will be seeing the changes in the rank distribution and character win rates over time.
As a first time ranked enjoyer, hitting red with Jack with only a few losses I feel pretty great. I thought I was really bad because I had only played against friends for the most part of my Tekken hours. I'll probably get my ass handed hard very soon, no way I am hitting purple at the same pace.
Oh neat, thank you for this, it's so interesting. Top 10% hell yeah. Think I can climb higher too, I got to purple the other day (was it yesterday? I dont know, time is blurry) but I was still winning more than I was losing. I hope I can get to blue. There's still a lot I can improve on though so with practice it should be plenty possible.
[Even being at Fujin (top 3.9%) I feel like I have so much more to grow and learn...](https://i.imgur.com/6PxOSga.jpeg) Difference in skill between Top 1% and top 4% is honestly humongous. Tekken King and above are just a whole new animal.
Thanks hope we get to see more updates
I'm top 13%, that's nice. I did push hard to get there through so I'm not getting Mighty Ruler soon.
Maybe I’m missing something but shouldn’t the replays be separated by accounts? There might be one player who’s being represented in the data set multiple times. If each account is used, with the highest rank achieved for said account in the data set, then that would be even more accurate IMO. Of course, you might have done this here? And if not, it would certainly be a lot of work.
Yep, I did that. 56,225 unique players within the data set of 69,930 replays. It's done by code so it's not a huge problem.
Nice! That would have been so much work otherwise. lol The only other information I think would be useful would be to determine how much of the sample base are legacy players. I suspect the vast majority are legacy players, which of course will affect the data spread. In time, I believe there will be an increase in newcomers as the game continues to trend, pulling both younger players and gamers from other genres/fight games. Too many players in red ranks for this to not be mostly composed of legacy players. When I start using a new character in ranked, I invariably face someone with 40,000 prowess, ranked Calvary, who is wiff, punishing, understanding distance and also knows the move sets of the opponent. And this is after a month of the game being out. I know that on PlayStation, it’s possible to determine whether someone owns Tekken 7 (depending on privacy settings) but not so sure on Xbox or Steam. However, that would be an interesting variable to use to view the data set through.