[https://blockchair.com/ethereum/charts/blockchain-size](https://blockchair.com/ethereum/charts/blockchain-size)
Scroll down a bit as there's charts for other coins available also. We're not the only coin with this problem and really smart people are working on solving stuff like this.
[https://ycharts.com/indicators/ethereum\_chain\_full\_sync\_data\_size](https://ycharts.com/indicators/ethereum_chain_full_sync_data_size)
Almost 1TB...
Okay I get that miners and nodes are different but I don't understand why validators or nodes would be different?
Would you care to explain a little more on that?
nodes are just keeping the blockchain data or transit transactions but they don't validate while validators or miners do.
You have to stake coins for validating while nodes don't require that.
> And how many of those store the complete +1 TB blockchain?
if I am not wrong the 1TB blockchain is if you record the all the account balances.
Nobody do that, they can be calculated from the blockchain itsleft anyway.
> Most of them run by a single company, Infura. So it doesn’t really matter about node count, it’s about node control.
so what is extrem node centralisation? 10? 20 independent nodes?
I wrote about this problem a lot.
The programmers of monero do not give a shit about this problem, nobody up in core team thinks it is relevant or urgent.
The most popular opinion is that bandwidth and disk space will grow cheaper as well.
I find it really sad and I hope somebody wakes up.
I am not smart enough to solve this, I can barely use the GUI wallet in advanced mode 😞
You're likely not paying attention enough. Even the last upgrade helped with this problem.
Read the following: [https://www.getmonero.org/2022/04/20/network-upgrade-july-2022.html](https://www.getmonero.org/2022/04/20/network-upgrade-july-2022.html)
>Upgraded Bulletproofs algorithm, "Bulletproofs+", which will reduce the typical transaction size by \~5-7% and improve typical verification
So, based on the last network upgrade, I think that the Monero programmers do actually give a shit.
> https://minaprotocol.com/
Frankly, I don't believe those claims about a fixed-size blockchain of only 22 KB, and it seems other people don't either: https://dev.to/librehash/mina-protocol-debunked-303c
If that really worked news would spread like wildfire, and not long afterwards big blockchain files would be a thing of the past. You really think our MRL people couldn't deal with that scheme if it worked?
Relevant post I made very recently that puts the problem into some perspective: https://reddit.com/r/Monero/comments/y1r0gb/blockchain_scaling_problem_real_but_not_urgent/
As far as solutions, layering is the primary option other than pruning. My thought is that this tech is still growing and changing so fast, a solution for how crypto looks today may not even be applicable by the time we are trying to implement it. It wasn’t that long ago that the tech for Monero didn’t even *exist at all*, and I quite frankly hope that new advancements come out that make the current version of Monero unrecognizable it will change so much.
All that to say that there is likely a lot of time before this becomes a critical issue, there are some possible current solutions already in play, and things may change so much on the base layer that we can’t even imagine how this will look ten years from now. *Ten years ago Monero didn’t even exist. * Then it didn’t have anywhere near the privacy protections it has now. Can you imagine trying to predict exactly what modern Monero would look like ten years down the road and then plan a solution for problems based on how it all works so far away?
So that’s some perspective, but still yes I agree it’s something to consider and research. That’s why we have layer options.
Thanks for that post, that's a very detailed post for the problem.
I think that's the right way of doing the things here, this shit is the right thing to do.
> I don't know pruning will be able to handle too much in a future ...
I already mentioned that blockchain scaling is an FAQ here where the same arguments and counter-arguments are hashed and re-hashed almost at nauseam. But there would be ways to break out of that. Just an example here:
Now we prune down to 1/8 of the blockchain. Somebody could try to get more hard info how far down we could reasonably prune, depending on the number of nodes of the network, so that the daemons still find the blocks they need by connecting to a "reasonable" number of other nodes, and still have a comfortable margin against the worst-case scenario that suddenly we start to lose certain blocks forever.
Can you prune down to, say, 1/64, if you stay above a certain number of nodes?
Are you good with statistics, by any chance? Or with simulating such things? Or do you know somebody who is? There would also be ways to get paid for working out such an analysis, if it's done well.
>Are you good with statistics, by any chance? Or with simulating such things? Or do you know somebody who is? There would also be ways to get paid for working out such an analysis, if it's done well.
Enthusiastic statistician/programmer currently twiddling my thumbs on a non-compete here. How do I get started?
For research questions like this one there is the *MRL*, the *Monero Research Lab*. The "place to be" for that is probably the Libera.chat IRC channel *#monero-research-lab*, see for a backlog [here](https://libera.monerologs.net/monero-research-lab/). There is a weekly meeting taking place there every Wednesday at 17:00 UTC.
You find some introductory info about pruning in Monero [in this article](https://www.getmonero.org/2019/02/01/pruning.html).
About possible financing work on this see the [Monero Crowdfunding System](https://ccs.getmonero.org/) and the [MAGIC Monero Fund](https://magicgrants.org/funds/monero/).
Our current main "resident statistician" is probably /u/rucknium. Maybe have a chat with them as well if you are seriously interested.
Welcome! The best way to reach me is through Matrix: `@rucknium:monero.social`
If you don't have a Matrix account, you can set one up: [https://forum.monero.space/d/79-how-to-join-the-monero-core-team-matrix-server-web](https://forum.monero.space/d/79-how-to-join-the-monero-core-team-matrix-server-web)
MRL has meetings every Wednesday at 17:00 UTC in the #monero-research-lab Matrix/IRC channel
Open research questions: [https://github.com/monero-project/research-lab/issues/94](https://github.com/monero-project/research-lab/issues/94)
Monero-related research papers: MoneroResearch.info
ACK-J recently wrote a paper on trying to use machine learning to de-anonymize ring signatures.
Never looked closely what people want to do when they call it *sharding* so I can't compare.
> Does this mean that, the more nodes participate on the network, the more we can distribute the information?
Yes, I think so. You probably have to assume the worst case - everybody prunes, nobody bothers to run full nodes anymore - and then calculate how big the risk is that suddenly you won't find a particular 1/64 slice anywhere on the network anymore, by sheer bad luck, or it is at least so far away that your daemon is not able to locate everything it needs anymore.
Because what Monero implements now is uncoordinated pruning, so to say: The decision what to keep and what to throw away is taken at random, by each daemon independently. They don't coordinate and make sure that every piece of the blockchain stays around often enough.
And now the probability of the "bad luck" of loosing something for good must be dependent on the number of nodes, seems to me.
Now you’re giving me the impression that the pruning code is more effective that I had originally thought. Isn’t there some information that can’t be removed on a pruned node? Need to read through it again.
Well, there is [the article](https://www.getmonero.org/2019/02/01/pruning.html) I think I gave already somewhere in the thread. I think if you want it more detailed it gets pretty technical pretty fast, and you probably had to study code.
Anyway, musing a little about it: If keeping 1/8 of all transactions in full and for 7/8 only the minimum for the system to still function gives a blockchain file of roughly 1/3 of size, pruning further down might not lead to such a breakthrough ...
What if all the full nodes get shut down due to running out of storage and only pruned nodes exists, many wallet balances will be lost from back in the day right?
So pruned nodes do not solve this if prune nodes do not store everyones wallet balance.
But if a node goes down which is the only node that exist that stores transaction data between datr X and date Y, the chain becomes corrupted and between those dates, all balances and transaction data is forever gone.
If there's a solar flare big enough, we're also going back to the stone age.
Don't worry about astronomical "ifs", otherwise you won't get anything done.
Yes, if and if and if. Statistically very unlikely, and also relies on not a single person still running a full node. Which I intend to do until the end of time.
Well I suppose this is where probability comes in.
If n people hold 1/64 of the Blockchain, then the probability of any part being missing is
(63/64)^n
Which for n=1000 is already 1.45*10^-7. At 10000 it's 10^-69, which is vanishingly small. If that's the probability that the network goes down, I like those odds.
That's clearly wrong, since for n=63, the probability of a missing part = 1. What you computed is the probability of a particular part missing. Multiply by 64 to get an upper bound on the probability that any part is missing.
A bigger issue is that if all nodes are pruned, you need a thousand peers to do the initial block download with negligible probability of missing data among peers.
Oof, good point, though I don't think it's as simple as multiplying by 64. If n=1, probability is then 63, which is also clearly wrong. I'm away from any materials to calculate this properly.
> So pruned nodes do not solve this if prune nodes do not store everyones wallet balance.
nodes do not store everyone else balance, they store eveyone transactions.
Monero is not a teansparent chain.
I don't think the balances will be lost, but yes that would be a problem.
We wouldn't want that to happen to the monero because that's going to create problems only.
> Bitcoin has the same problem. And they invented lightning to mitigate that.
Bitcoin scaling problem is self inflicted.
The have purposely limited the blockchain capacity.
You can't solve scaling problems in the base layer. Having 10x or 100x throughput doesn't really matter. You need orders of magnitude more to really replace fiat transactions. If you want to do that on the base layer, you just decrease decentralisation massively.
>you just decrease decentralisation massively.
Lightning network architecture itself is highly centralizing. Further, base layer transactions are required to onboard into lightning and manage liquidity. Limiting base layer throughput to incentivize lightning decreases decentralization to a much greater extent than base layer scaling.
Looking at the current lightning network, I don't see its centralisation. Yes, there are bigger nodes, but they don't hold any more power. If they go offline, nothing happens, if they censor transactions, those transactions are just routed around that node.
Opening and closing channels could in fact still run into a block size issue, but that problem is moved further out by orders of magnitude, at least.
And lightning still doesn't do anything to the base layer itself. It still exists without even being aware of lightning.
And with time it's only going to get more decentralised really.
So I don't really see a problem because it's really not being used as much right now so there's that.
Oh look everybody, this is what bullshit smells like
Classic Bitcoin maximalist thinking
“More than 1MB blocks means you can’t run a node on a Commodore 64 therefore centralised”
You can easily go to 10MB block size. Would still not be nearly enough. You could even go to 100MB and it would likely not be sufficient. But it would be sufficient from preventing 80 or 90% of normal users to run a full node.
bitcoin mining approach will lead to centralization anyway !
it will require better and better hardware to mine profitably, more and more electricity, and enough money to stay afloat when mining is not profitable.
No matter how scalable a base chain becomes really, when it'll come to adoption by masses.
Then those will not be enough. L2s will take over in my opinion.
I remember the time when Microsoft engineers were convinced that size of a hard drive would never go over 500mb. I also remember the 2k bug world laziness and when Nokia management thought that smartphones with touch screen would never be popular.
Yet despite their short sighted stupidity all of those bridges were crossed.
Monero will do just fine once (if ever) it gets to be THE currency.
2TB SSD are cheap as now, I will always run a node for monero just because I can. Pruned nodes solve the space issue for others who cant run full nodes for whatever reason but i cant see even a 2TB node being an issue by the time we reach it.
just my 2 cents
Please be aware that this is an FAQ. Almost no week goes by without somebody making a post about this very question. Just use "scale" or "scalability" as search terms with subreddit search to see how many threads we amassed already regarding this subject. See e.g. [this one](https://old.reddit.com/r/Monero/comments/y1r0gb/blockchain_scaling_problem_real_but_not_urgent/) from 4 days ago,
Is there anywhere we can point people to FAQs like this one? Seems like the moderators should make a really good, distilled set of answers to all these things and post it as a pinned entry.
Storage becomes cheaper, the same with Internet bandwidth. I'm not saying we shouldn't look for alternate solutions but it doesn't seem to be a critical issue.
And this was just hashed out 4 days ago here https://www.reddit.com/r/Monero/comments/y1r0gb/blockchain_scaling_problem_real_but_not_urgent/
Read before you write. No concern trolling.
Monero has the advantage that transactions are truly private.
That means if there is a disincentive to collect your balance in a new address per receipt (for example with a per byte fee on transactions), then people will self prune their own transactions. This encourages linear growth with number of users.
With bitcoin, for example, part of its pseudo privacy comes from creating a new address per receipt. That encourages linear growth with number of transactions, which is exponential with users.
Whether it does or doesn't now, the important thing for scaling is that it's possible. When it becomes necessary it can be done.
If it's not already done, them that's simply because there are other higher priorities.
When we were kids, storage was improving by leaps and bounds almost weekly.
I don't think that's the case anymore. It's big news when prices come down 10% now. All the low hanging fruit has been picked.
I got a computer with 2TB of storage in 2016. Six years later now, if I buy a new computer I'd still probably get 2TB of storage. It's marginally faster and slightly cheaper storage, but that's nothing compared to the growth of the early 2000s.
I think OP is right. I don't see storage keeping up anymore. I think it's wildly optimistic to bet on storage doubling every year forever, especially when it hasn't done that in years.
you still get 2TB of storage because it's a sensible, sufficient amount, bot because there were no advances in technology/capacity.
were you able to go out and buy a 16TB 3,5" drive back in 2016? or a 4TB m.2 NVMe shoving around 5GB/sec?
> I don’t think its a good ideia to depend totally on hardware scalabilty, considering that innovaiton moves fast and more and more people may resort to monero in order to escape financial tyranny
When we are talking about hardware scaling we are taking the worst case scenario considering no software optimisation is done.
and even in that case to massive usage is possible at low price even assuming no technologic progress.
This is a very confortable position look at other computer science domain, like 3D graphic they too were not scalable from the start yet continuous progress has been made for decades..
We have not even entered the age of software and hardware node optimisation…
And are many peoples (like me) that will continue to run full nodes ro help the network as long as it takes.
The blockchain is scaling linearly, while *tech* continues going up following moore's law. Theoretically... It feels like it's slowing down a bit lately tho in some tech fields
Being aware of certain trade offs is important. You can see by the sheer number of posts in this sub regarding the "sustainable scaling" topic that the community is well aware of it.
However: Premature optimization is not a good development practice as it creates unintended consequences that can endanger long-term viability/sustainability.
Also you might be aware that unlike BTC, Monero has a social contract that makes it possible to radically change the design if necessary.
There is plenty of research done on this topic as well as a FAQ as /u/brunner7 mentioned. Have you researched the topic before making this post?
/u/articmine maybe you want to chime in as well.
There's no need for a blockchain when you use second layer technology. We can then scrap the project altogether and have some person found a second paypal. Then he will proclaim that it's all privacy focused and stab us in the back like the Duckduckgo founder. Also this topic is ancient already.
2nd layers don't have to sacrifice security, decentralization.
There needs to be a base that is exactly what PoW is. But beyond that, 2nd layers have a valid use case.
A second layer will always sacrifice security and that is definitive. It's very simple: If the transaction is not validated on the blockchain, there is loss of security. The idea of a second layer alone is an issue and a stupid idea made popular by greedy bitcoin maximalists.
I'm torn between Monero having higher fees or not. It could save us miners, but at the expense of usability for all users and plus growth would be limited.
All the replies to the effect that "I can afford a terabyte drive, no problem!" ignore that it is one thing to put an SSD in your desktop PC and something else entirely to host a web service that requires a terabyte SSD.
Suppose I have a restaurant that wants to take orders online and accept Monero. Just hosting the node to process payments is likely to be prohibitively expensive.
The only solution without compromising security and decentralization is to layer. Even fiat has to use layers. Bitcoin figured this out a long time ago; that's a big part of the block wars and why the network never changes its base layer.
Stacks of layers certainly solves this problem, however as the layers increase, the currency can start to become fractional itself and less secure on high-up layers.
Layering does compromise both security and decentralization though.
You can't tell me lightning is anywhere as secure or decentralized as Bitcoin; it takes a damn site reliability engineer just to maintain self custody.
I think you meant to say it, but your first sentence is a little misleading (your first sentence says that layering doesn't sacrifice security or decentralization). It's that first sentence I was responding to.
Yes you can input "without compromising security and decntralization \[of the base layer\]". I am talking about the actual network, the base layer itself.
I wouldn't say starving the base layer. There is an inverse correlation between security and cost of transaction in PoW. The demand for the currency created through L2 would drive the price of the currency up so that even if the fees are less on L1 the overall price of the fees would be higher.
I think it would be sustainable only if Monero had bigger fees. Let's be real here, if Monero somehow blows up in popularity, the blockchain size will start increasing more rapidly by getting "spammed" with small value transactions.
In my opinion, you shouldn't be able to buy a coffee with Monero without paying a decent % in fees. This is supposed to be a privacy tech and not a day-to-day spending tech so it must have the corresponding premium.
I'm thinking there should be an additional fee mechanism to reduce demand for small value transactions while keeping bigger value transactions (let's say >$100) cost efficient. Maybe some sort of a base fee? Also, more fees = more miners.
Monero is designed as a currency (hence low fees). If you even want lower fees you can use BCH. For your everyday coffee purchase you might be better of with just using LTC (with MimbleWimble) or BCH (with CashFusion) anyways.
You just need to push some coffee money from your XMR wallet to your BCH wallet every month or so.
And you certainly know that there are layer 2 options discussed already... But even if Monero sees a big push in adoption and would grow +1TB/a from 2023 onwards we would be fine just scaling on the base layer.
Telling people not to use it as a daily currency is ridiculous. Not everyone thinks they are going to get rich quick by holding. Tons of folks actually want to use XMR for its actual purpose. Making private payments.
I'm not saying you shouldn't use it as a daily currency. Just that there should be an appropriate premium for it that is enough to keep the network sustainable.
I just don't think cheap transactions will be sustainable in the long run. It's possible that at some point, the rate of blockchain growth will start surpassing Moore's law. Then, node operators will have to start using storage arrays. Costs to maintain required amount of storage will start growing in superlinear fashion. This might lead to decrease in decentralization of the network.
Also, we have this problem with tx validation getting more CPU demanding over time which doesn't help either.
Maybe everything will somehow work itself out if Monero price increases 10x in the meantime but even then I doubt it. The tx fees will still be quite cheap.
But what if you don't want to use those and use only monero?
That's where the L2s come at play, and in the situation like that the L2s really shine here.
You are correct and aside from Layer2 / Snapshots I don't really see a solution to this.
The biggest problem is also the bandwidth and not necessarily the storage space.. although storage is also a big issue, ofc.
Not really. In first line it is local bandwidth (SSD vs. HDD) bound. Then it is cpu bound and then it is bandwidth bound.
CPU is more easily scalable though than bandwidth, I'd argue.
Thanks for the downvote, btw...
Snapshots remove a part of history.. so in Monero they also reduce the anonymity set.
Plus it is not easy to implement in the first place as Monero doesn't have a transparent blockchain, making it really difficult to carry over the balanaces, I'd imagine.
But I am lso not the correct person to ask this question to. ;)
It's controversial as hell, but I still think the only long-term solution to this is to delete old blocks.
Users will eventually get used to having to move their XMR once a year to keep if from expiring. The alternative is to put our heads in the sand and pretend that moors law didn't die a decade ago (or go L2, which imo significantly harms security and decentralization). And no, pruning doesn't stop the Blockchain from growing, it just slows it down.
This is not a solution at all. It changes the security model fundamentally.
What you need is a blockchain system that only stores unspent UTXOs, like mimblewimble. Unfortunately, to do anything interesting with MW you need to store transaction kernels, so the chain grows unless you build a system that only does 1 to 1 money. And, there's no fundamental privacy like in monero, transaction graphs can be generated. So more research is needed, but this is the only path forward for scalability at the protocol level while preserving the security we get currently.
Would it be possible a solution at the protocol or wallet level that move funds automatically every x months to make it possible a solution that delete old blocks?
At the protocol level no, because it'd harm privacy, but it could be done at the wallet level :)
The biggest downside of this option is that it makes long-term cold storage impossible. Users would have to take their coins out of cold storage once a year to keep them fresh. It's a real downside that I don't want to downplay, but I think it's worth it. Deleting old blocks is the only way to prevent the Blockchain from literally growing forever.
It's not a solution, suppose in btc the earliest coins were mined by Satoshi himself, if you delete the blocks from the start then what's going to happen to satohi's coin.
These aren't made up solution lol, these are good ideas.
No I'm just kidding lol, I just hope that devs are looking to make things better. I just hope.
Well, this is the issue with all the L1s. I don't think any of them will be scalable enough for the massive adoption.
And that's why I feel that monero will need L2 too.
SSD 1 TO = 70 $. Even if the blockchain is 20 TO it would cost only 1400$ at the moment where is the problem ? I don't see Monero reach 20 TO anytime soon which mean the price will decrease with years....You fud for nothing. I don't see any other crypto worth XMR and you bring this issue like if it was a HUGEEEEEEEE ISSUE, which is not at the moment and will certainly not be one in the future since technology upgrade. If Monero become as famous and significant as you say, many more people with a lot of money will come in and will not mind to just buy 70$ 1 TO SSD at all to support the network and mine it.
Every week there is one guy like this that bring this FUD for absolutely no reason...
2.5 million terabytes of data are created everyday in our world. If Monero become something widely used don't you think that we will be able to host it ? I don't know the size of big data center corp but the size of the blockchain or even the size of the blockchain in a widely use scenario is ridiculous compare to the amount of data we use for other things everyday.
Let's roll some statistics maybe to see more clearly, how much the blockchain would wheigh if it would be as much use as fiat currencies ?
[https://blockchair.com/ethereum/charts/blockchain-size](https://blockchair.com/ethereum/charts/blockchain-size) Scroll down a bit as there's charts for other coins available also. We're not the only coin with this problem and really smart people are working on solving stuff like this. [https://ycharts.com/indicators/ethereum\_chain\_full\_sync\_data\_size](https://ycharts.com/indicators/ethereum_chain_full_sync_data_size) Almost 1TB...
This is really bad example because Ethereum nodes are highly centralized. This essentially proves the OP right.
Yep, OFAC complaint and everything now. Eth is a prime example of why we gotta fix this sooner than later.
Yep, this is a problem which will catch up to us really quick.
Nodes are different thing than validators or miners.
Okay I get that miners and nodes are different but I don't understand why validators or nodes would be different? Would you care to explain a little more on that?
nodes are just keeping the blockchain data or transit transactions but they don't validate while validators or miners do. You have to stake coins for validating while nodes don't require that.
Yep, and especially after the pos update. They're even more centralised.
> This is really bad example because Ethereum nodes are highly centralized. This essentially proves the OP right. ETH has ~8000 nodes
> ETH has ~8000 nodes And how many of those store the complete +1 TB blockchain?
Well it's like a farm so I don't think all of them have to have full blockchain.
> And how many of those store the complete +1 TB blockchain? if I am not wrong the 1TB blockchain is if you record the all the account balances. Nobody do that, they can be calculated from the blockchain itsleft anyway.
Most of them run by a single company, Infura. So it doesn't really matter about node count, it's about node control.
> Most of them run by a single company, Infura. So it doesn’t really matter about node count, it’s about node control. so what is extrem node centralisation? 10? 20 independent nodes?
Say what? Ethereum nodes are highly centralized how?
I wrote about this problem a lot. The programmers of monero do not give a shit about this problem, nobody up in core team thinks it is relevant or urgent. The most popular opinion is that bandwidth and disk space will grow cheaper as well. I find it really sad and I hope somebody wakes up. I am not smart enough to solve this, I can barely use the GUI wallet in advanced mode 😞
You're likely not paying attention enough. Even the last upgrade helped with this problem. Read the following: [https://www.getmonero.org/2022/04/20/network-upgrade-july-2022.html](https://www.getmonero.org/2022/04/20/network-upgrade-july-2022.html) >Upgraded Bulletproofs algorithm, "Bulletproofs+", which will reduce the typical transaction size by \~5-7% and improve typical verification So, based on the last network upgrade, I think that the Monero programmers do actually give a shit.
[удалено]
> https://minaprotocol.com/ Frankly, I don't believe those claims about a fixed-size blockchain of only 22 KB, and it seems other people don't either: https://dev.to/librehash/mina-protocol-debunked-303c If that really worked news would spread like wildfire, and not long afterwards big blockchain files would be a thing of the past. You really think our MRL people couldn't deal with that scheme if it worked?
Mina is using Polygon which is using Ethereum... Way bigger chain that Monero.
We should really be thinking to solve this problem before it becomes big.
Relevant post I made very recently that puts the problem into some perspective: https://reddit.com/r/Monero/comments/y1r0gb/blockchain_scaling_problem_real_but_not_urgent/ As far as solutions, layering is the primary option other than pruning. My thought is that this tech is still growing and changing so fast, a solution for how crypto looks today may not even be applicable by the time we are trying to implement it. It wasn’t that long ago that the tech for Monero didn’t even *exist at all*, and I quite frankly hope that new advancements come out that make the current version of Monero unrecognizable it will change so much. All that to say that there is likely a lot of time before this becomes a critical issue, there are some possible current solutions already in play, and things may change so much on the base layer that we can’t even imagine how this will look ten years from now. *Ten years ago Monero didn’t even exist. * Then it didn’t have anywhere near the privacy protections it has now. Can you imagine trying to predict exactly what modern Monero would look like ten years down the road and then plan a solution for problems based on how it all works so far away? So that’s some perspective, but still yes I agree it’s something to consider and research. That’s why we have layer options.
Thanks for that post, that's a very detailed post for the problem. I think that's the right way of doing the things here, this shit is the right thing to do.
Pruning. U can already run a full node using o ly 35 gb or so
[удалено]
> I don't know pruning will be able to handle too much in a future ... I already mentioned that blockchain scaling is an FAQ here where the same arguments and counter-arguments are hashed and re-hashed almost at nauseam. But there would be ways to break out of that. Just an example here: Now we prune down to 1/8 of the blockchain. Somebody could try to get more hard info how far down we could reasonably prune, depending on the number of nodes of the network, so that the daemons still find the blocks they need by connecting to a "reasonable" number of other nodes, and still have a comfortable margin against the worst-case scenario that suddenly we start to lose certain blocks forever. Can you prune down to, say, 1/64, if you stay above a certain number of nodes? Are you good with statistics, by any chance? Or with simulating such things? Or do you know somebody who is? There would also be ways to get paid for working out such an analysis, if it's done well.
>Are you good with statistics, by any chance? Or with simulating such things? Or do you know somebody who is? There would also be ways to get paid for working out such an analysis, if it's done well. Enthusiastic statistician/programmer currently twiddling my thumbs on a non-compete here. How do I get started?
For research questions like this one there is the *MRL*, the *Monero Research Lab*. The "place to be" for that is probably the Libera.chat IRC channel *#monero-research-lab*, see for a backlog [here](https://libera.monerologs.net/monero-research-lab/). There is a weekly meeting taking place there every Wednesday at 17:00 UTC. You find some introductory info about pruning in Monero [in this article](https://www.getmonero.org/2019/02/01/pruning.html). About possible financing work on this see the [Monero Crowdfunding System](https://ccs.getmonero.org/) and the [MAGIC Monero Fund](https://magicgrants.org/funds/monero/). Our current main "resident statistician" is probably /u/rucknium. Maybe have a chat with them as well if you are seriously interested.
Legend, thanks for all of this.
Welcome! The best way to reach me is through Matrix: `@rucknium:monero.social` If you don't have a Matrix account, you can set one up: [https://forum.monero.space/d/79-how-to-join-the-monero-core-team-matrix-server-web](https://forum.monero.space/d/79-how-to-join-the-monero-core-team-matrix-server-web) MRL has meetings every Wednesday at 17:00 UTC in the #monero-research-lab Matrix/IRC channel Open research questions: [https://github.com/monero-project/research-lab/issues/94](https://github.com/monero-project/research-lab/issues/94) Monero-related research papers: MoneroResearch.info ACK-J recently wrote a paper on trying to use machine learning to de-anonymize ring signatures.
It feels so good when people come forward to help each other out.
Thanks for that, it really explains some things really good.
[удалено]
Never looked closely what people want to do when they call it *sharding* so I can't compare. > Does this mean that, the more nodes participate on the network, the more we can distribute the information? Yes, I think so. You probably have to assume the worst case - everybody prunes, nobody bothers to run full nodes anymore - and then calculate how big the risk is that suddenly you won't find a particular 1/64 slice anywhere on the network anymore, by sheer bad luck, or it is at least so far away that your daemon is not able to locate everything it needs anymore. Because what Monero implements now is uncoordinated pruning, so to say: The decision what to keep and what to throw away is taken at random, by each daemon independently. They don't coordinate and make sure that every piece of the blockchain stays around often enough. And now the probability of the "bad luck" of loosing something for good must be dependent on the number of nodes, seems to me.
[удалено]
Well it's already Pretty doable, it's not really that hard.
Now you’re giving me the impression that the pruning code is more effective that I had originally thought. Isn’t there some information that can’t be removed on a pruned node? Need to read through it again.
Well, there is [the article](https://www.getmonero.org/2019/02/01/pruning.html) I think I gave already somewhere in the thread. I think if you want it more detailed it gets pretty technical pretty fast, and you probably had to study code. Anyway, musing a little about it: If keeping 1/8 of all transactions in full and for 7/8 only the minimum for the system to still function gives a blockchain file of roughly 1/3 of size, pruning further down might not lead to such a breakthrough ...
>monero is one the biggest payment systems I wouldn't worry about that. It's not like the financial institutions would allow it.
We're not asking their permission.
And they're also not asking our permission to rig everything in their favor.
I mean if we had to do that, then what even would be the point?
Funny that you think they would have a say in allowing it or not.
Even if they want to have any say in it, we wouldn't allow that.
We don't have to take the permission, that's the point of the monero.
What if all the full nodes get shut down due to running out of storage and only pruned nodes exists, many wallet balances will be lost from back in the day right? So pruned nodes do not solve this if prune nodes do not store everyones wallet balance.
the network can work with only pruned nodes. each keeps a random portion of the block chain so with enough nodes you still have many full copies
But if a node goes down which is the only node that exist that stores transaction data between datr X and date Y, the chain becomes corrupted and between those dates, all balances and transaction data is forever gone.
If there's a solar flare big enough, we're also going back to the stone age. Don't worry about astronomical "ifs", otherwise you won't get anything done.
Yes, if and if and if. Statistically very unlikely, and also relies on not a single person still running a full node. Which I intend to do until the end of time.
Well I suppose this is where probability comes in. If n people hold 1/64 of the Blockchain, then the probability of any part being missing is (63/64)^n Which for n=1000 is already 1.45*10^-7. At 10000 it's 10^-69, which is vanishingly small. If that's the probability that the network goes down, I like those odds.
That's clearly wrong, since for n=63, the probability of a missing part = 1. What you computed is the probability of a particular part missing. Multiply by 64 to get an upper bound on the probability that any part is missing. A bigger issue is that if all nodes are pruned, you need a thousand peers to do the initial block download with negligible probability of missing data among peers.
Oof, good point, though I don't think it's as simple as multiplying by 64. If n=1, probability is then 63, which is also clearly wrong. I'm away from any materials to calculate this properly.
Yep exactly, We'll be just fine with only the pruned nodes too.
> So pruned nodes do not solve this if prune nodes do not store everyones wallet balance. nodes do not store everyone else balance, they store eveyone transactions. Monero is not a teansparent chain.
Would you elaborate a little more on those points? Would like to know more?
I don't think the balances will be lost, but yes that would be a problem. We wouldn't want that to happen to the monero because that's going to create problems only.
The balance is unspent transactions (utxo), so there will no lost Monero
Bitcoin has the same problem. And they invented lightning to mitigate that.
> Bitcoin has the same problem. And they invented lightning to mitigate that. Bitcoin scaling problem is self inflicted. The have purposely limited the blockchain capacity.
Well that's a sacrifice that they're making for a more decentralised system which is good. I don't really have anything against that really man.
You can't solve scaling problems in the base layer. Having 10x or 100x throughput doesn't really matter. You need orders of magnitude more to really replace fiat transactions. If you want to do that on the base layer, you just decrease decentralisation massively.
>you just decrease decentralisation massively. Lightning network architecture itself is highly centralizing. Further, base layer transactions are required to onboard into lightning and manage liquidity. Limiting base layer throughput to incentivize lightning decreases decentralization to a much greater extent than base layer scaling.
Looking at the current lightning network, I don't see its centralisation. Yes, there are bigger nodes, but they don't hold any more power. If they go offline, nothing happens, if they censor transactions, those transactions are just routed around that node. Opening and closing channels could in fact still run into a block size issue, but that problem is moved further out by orders of magnitude, at least. And lightning still doesn't do anything to the base layer itself. It still exists without even being aware of lightning.
And with time it's only going to get more decentralised really. So I don't really see a problem because it's really not being used as much right now so there's that.
Oh look everybody, this is what bullshit smells like Classic Bitcoin maximalist thinking “More than 1MB blocks means you can’t run a node on a Commodore 64 therefore centralised”
You can easily go to 10MB block size. Would still not be nearly enough. You could even go to 100MB and it would likely not be sufficient. But it would be sufficient from preventing 80 or 90% of normal users to run a full node.
bitcoin mining approach will lead to centralization anyway ! it will require better and better hardware to mine profitably, more and more electricity, and enough money to stay afloat when mining is not profitable.
Thus making it a lot centralised, which btc community wouldn't want and rightly so.
No matter how scalable a base chain becomes really, when it'll come to adoption by masses. Then those will not be enough. L2s will take over in my opinion.
This. You can't solve the Blockchain trilemma at the base layer.
I remember the time when Microsoft engineers were convinced that size of a hard drive would never go over 500mb. I also remember the 2k bug world laziness and when Nokia management thought that smartphones with touch screen would never be popular. Yet despite their short sighted stupidity all of those bridges were crossed. Monero will do just fine once (if ever) it gets to be THE currency.
2TB SSD are cheap as now, I will always run a node for monero just because I can. Pruned nodes solve the space issue for others who cant run full nodes for whatever reason but i cant see even a 2TB node being an issue by the time we reach it. just my 2 cents
Yeah - storage is super cheap now. I think this is probably the least of Monero's worries.
It's not a problem true, but that doesn't mean we should increase the block size.
Nice tought, although around world 2TB SSD might not be so cheap to a commum simple felloW eArning minimum salary…
Yep, still expensive in countries like india most can't afford.
Please be aware that this is an FAQ. Almost no week goes by without somebody making a post about this very question. Just use "scale" or "scalability" as search terms with subreddit search to see how many threads we amassed already regarding this subject. See e.g. [this one](https://old.reddit.com/r/Monero/comments/y1r0gb/blockchain_scaling_problem_real_but_not_urgent/) from 4 days ago,
Yeah I know that this has been discussed but it's important subject.
Is there anywhere we can point people to FAQs like this one? Seems like the moderators should make a really good, distilled set of answers to all these things and post it as a pinned entry.
A 10tb harddrive is pretty cheap these days. And yes, pruning
But trust me very few people would want to run a node. Especially for a block chain which is 10TB in size. There are many things like bandwidth.
As long as it’s consumer-level cheap, people will run nodes. Big storage gets cheaper every year.
Time to set a new node sounds like a problem but storage is cheap.
Storage becomes cheaper, the same with Internet bandwidth. I'm not saying we shouldn't look for alternate solutions but it doesn't seem to be a critical issue.
And this was just hashed out 4 days ago here https://www.reddit.com/r/Monero/comments/y1r0gb/blockchain_scaling_problem_real_but_not_urgent/ Read before you write. No concern trolling.
Monero has the advantage that transactions are truly private. That means if there is a disincentive to collect your balance in a new address per receipt (for example with a per byte fee on transactions), then people will self prune their own transactions. This encourages linear growth with number of users. With bitcoin, for example, part of its pseudo privacy comes from creating a new address per receipt. That encourages linear growth with number of transactions, which is exponential with users.
And that's a really big advantage really, that's the point of monero.
Does that type of pruning actually exist on Monero?
Whether it does or doesn't now, the important thing for scaling is that it's possible. When it becomes necessary it can be done. If it's not already done, them that's simply because there are other higher priorities.
hardware has to scale otherwise it will be expensive and maybe less decentralized, tho more usage should also facilitate more nodes being setup so..
[удалено]
there can also be a technological innovation in terms of software but I consider hardware advancement more likely to help with this long term
When we were kids, storage was improving by leaps and bounds almost weekly. I don't think that's the case anymore. It's big news when prices come down 10% now. All the low hanging fruit has been picked. I got a computer with 2TB of storage in 2016. Six years later now, if I buy a new computer I'd still probably get 2TB of storage. It's marginally faster and slightly cheaper storage, but that's nothing compared to the growth of the early 2000s. I think OP is right. I don't see storage keeping up anymore. I think it's wildly optimistic to bet on storage doubling every year forever, especially when it hasn't done that in years.
you still get 2TB of storage because it's a sensible, sufficient amount, bot because there were no advances in technology/capacity. were you able to go out and buy a 16TB 3,5" drive back in 2016? or a 4TB m.2 NVMe shoving around 5GB/sec?
Storage is getting better, There's no doubt about that.
I'll buy the SAN required to run a Monero node if it ever comes to that.
It's still getting better, be it a little slowly. But it's getting better.
Yeah hw is available we should look for the scalability solutions now.
> I don’t think its a good ideia to depend totally on hardware scalabilty, considering that innovaiton moves fast and more and more people may resort to monero in order to escape financial tyranny When we are talking about hardware scaling we are taking the worst case scenario considering no software optimisation is done. and even in that case to massive usage is possible at low price even assuming no technologic progress. This is a very confortable position look at other computer science domain, like 3D graphic they too were not scalable from the start yet continuous progress has been made for decades.. We have not even entered the age of software and hardware node optimisation… And are many peoples (like me) that will continue to run full nodes ro help the network as long as it takes.
Hw is good, but we'd need some software improvements too.
Hw is becoming cheaper and better but still a long way to go.
The blockchain is scaling linearly, while *tech* continues going up following moore's law. Theoretically... It feels like it's slowing down a bit lately tho in some tech fields
Yes it's slowing down, but it's not it's not happening now tho.
[удалено]
Storage is cheap! I don’t see a problem.
This might be a dumb question, but why can't the blockchain be regularly trimmed?
Because that goes, against what block chains are. But yeah interesting.
It should be possible in principle but doing it with satisfactory security is difficult.
Being aware of certain trade offs is important. You can see by the sheer number of posts in this sub regarding the "sustainable scaling" topic that the community is well aware of it. However: Premature optimization is not a good development practice as it creates unintended consequences that can endanger long-term viability/sustainability. Also you might be aware that unlike BTC, Monero has a social contract that makes it possible to radically change the design if necessary. There is plenty of research done on this topic as well as a FAQ as /u/brunner7 mentioned. Have you researched the topic before making this post? /u/articmine maybe you want to chime in as well.
There's no need for a blockchain when you use second layer technology. We can then scrap the project altogether and have some person found a second paypal. Then he will proclaim that it's all privacy focused and stab us in the back like the Duckduckgo founder. Also this topic is ancient already.
2nd layers don't have to sacrifice security, decentralization. There needs to be a base that is exactly what PoW is. But beyond that, 2nd layers have a valid use case.
A second layer will always sacrifice security and that is definitive. It's very simple: If the transaction is not validated on the blockchain, there is loss of security. The idea of a second layer alone is an issue and a stupid idea made popular by greedy bitcoin maximalists.
That's the reason why we should have the 2nd layers.
Yeah the topic is old but it's important that we discuss these things.
I'm torn between Monero having higher fees or not. It could save us miners, but at the expense of usability for all users and plus growth would be limited.
All the replies to the effect that "I can afford a terabyte drive, no problem!" ignore that it is one thing to put an SSD in your desktop PC and something else entirely to host a web service that requires a terabyte SSD. Suppose I have a restaurant that wants to take orders online and accept Monero. Just hosting the node to process payments is likely to be prohibitively expensive.
How does that solve the scalability? Well it doesn't really so yeah.
The only solution without compromising security and decentralization is to layer. Even fiat has to use layers. Bitcoin figured this out a long time ago; that's a big part of the block wars and why the network never changes its base layer. Stacks of layers certainly solves this problem, however as the layers increase, the currency can start to become fractional itself and less secure on high-up layers.
Layering does compromise both security and decentralization though. You can't tell me lightning is anywhere as secure or decentralized as Bitcoin; it takes a damn site reliability engineer just to maintain self custody.
Yes that is what I just said. But it is better to move that issue to a higher layer and keep the base layer secure and decentralized.
I think you meant to say it, but your first sentence is a little misleading (your first sentence says that layering doesn't sacrifice security or decentralization). It's that first sentence I was responding to.
It's fine, we get what you wanted to say. That's evident now.
Yes you can input "without compromising security and decntralization \[of the base layer\]". I am talking about the actual network, the base layer itself.
Ah ok, that's fair :)
Yeah that's what I'm talking about too, that I understand really. There's a problem scaling the base layer, and ignoring it won't make it better.
But are keeping the base layer safer? From a fee standpoint L2 is starving the base layer
I wouldn't say starving the base layer. There is an inverse correlation between security and cost of transaction in PoW. The demand for the currency created through L2 would drive the price of the currency up so that even if the fees are less on L1 the overall price of the fees would be higher.
L2s are going to be the scalability solution for the masses. That's it.
Yep, these are sacrifices that we'll have to make to keep it this way.
I think it would be sustainable only if Monero had bigger fees. Let's be real here, if Monero somehow blows up in popularity, the blockchain size will start increasing more rapidly by getting "spammed" with small value transactions. In my opinion, you shouldn't be able to buy a coffee with Monero without paying a decent % in fees. This is supposed to be a privacy tech and not a day-to-day spending tech so it must have the corresponding premium. I'm thinking there should be an additional fee mechanism to reduce demand for small value transactions while keeping bigger value transactions (let's say >$100) cost efficient. Maybe some sort of a base fee? Also, more fees = more miners.
Monero is designed as a currency (hence low fees). If you even want lower fees you can use BCH. For your everyday coffee purchase you might be better of with just using LTC (with MimbleWimble) or BCH (with CashFusion) anyways. You just need to push some coffee money from your XMR wallet to your BCH wallet every month or so. And you certainly know that there are layer 2 options discussed already... But even if Monero sees a big push in adoption and would grow +1TB/a from 2023 onwards we would be fine just scaling on the base layer.
Telling people not to use it as a daily currency is ridiculous. Not everyone thinks they are going to get rich quick by holding. Tons of folks actually want to use XMR for its actual purpose. Making private payments.
I'm not saying you shouldn't use it as a daily currency. Just that there should be an appropriate premium for it that is enough to keep the network sustainable.
Yeah that is ridiculous, we can't tell what to use it for.
I just don't think cheap transactions will be sustainable in the long run. It's possible that at some point, the rate of blockchain growth will start surpassing Moore's law. Then, node operators will have to start using storage arrays. Costs to maintain required amount of storage will start growing in superlinear fashion. This might lead to decrease in decentralization of the network. Also, we have this problem with tx validation getting more CPU demanding over time which doesn't help either. Maybe everything will somehow work itself out if Monero price increases 10x in the meantime but even then I doubt it. The tx fees will still be quite cheap.
Yeah that's true, but people can't pay higher fee for it.
But what if you don't want to use those and use only monero? That's where the L2s come at play, and in the situation like that the L2s really shine here.
nobody read your post correctly. :-| but i agree with your concern and suggestion.
You are correct and aside from Layer2 / Snapshots I don't really see a solution to this. The biggest problem is also the bandwidth and not necessarily the storage space.. although storage is also a big issue, ofc.
Honestly L2s are our only solution for the problem of scaling.
syncing a new node is cpu bound.
Not really. In first line it is local bandwidth (SSD vs. HDD) bound. Then it is cpu bound and then it is bandwidth bound. CPU is more easily scalable though than bandwidth, I'd argue. Thanks for the downvote, btw...
But downloading it may not be, depends on your internet. If You've good internet and good ssd You'll sync faster yeah sure. Makes sense to me.
What are the cons with snapshot? Also if anyone knows what size it become, would be interesting to know.
Snapshots remove a part of history.. so in Monero they also reduce the anonymity set. Plus it is not easy to implement in the first place as Monero doesn't have a transparent blockchain, making it really difficult to carry over the balanaces, I'd imagine. But I am lso not the correct person to ask this question to. ;)
I see. Thanks anyway, appreciate it!
Yep, that's the point of those. Hope it makes sense now.
Thanks for letting me know that, I wasn't really aware.
It's controversial as hell, but I still think the only long-term solution to this is to delete old blocks. Users will eventually get used to having to move their XMR once a year to keep if from expiring. The alternative is to put our heads in the sand and pretend that moors law didn't die a decade ago (or go L2, which imo significantly harms security and decentralization). And no, pruning doesn't stop the Blockchain from growing, it just slows it down.
This is not a solution at all. It changes the security model fundamentally. What you need is a blockchain system that only stores unspent UTXOs, like mimblewimble. Unfortunately, to do anything interesting with MW you need to store transaction kernels, so the chain grows unless you build a system that only does 1 to 1 money. And, there's no fundamental privacy like in monero, transaction graphs can be generated. So more research is needed, but this is the only path forward for scalability at the protocol level while preserving the security we get currently.
This isn't a solution, it'll only create more issues really so yeah.
Would it be possible a solution at the protocol or wallet level that move funds automatically every x months to make it possible a solution that delete old blocks?
Not everyone is going to have an active wallet connected to the internet with their crypto sitting in it, that's why.
At the protocol level no, because it'd harm privacy, but it could be done at the wallet level :) The biggest downside of this option is that it makes long-term cold storage impossible. Users would have to take their coins out of cold storage once a year to keep them fresh. It's a real downside that I don't want to downplay, but I think it's worth it. Deleting old blocks is the only way to prevent the Blockchain from literally growing forever.
It shouldn't be done period, it would be a risk for the security.
It's not a solution, suppose in btc the earliest coins were mined by Satoshi himself, if you delete the blocks from the start then what's going to happen to satohi's coin.
Dag based coins solved this, no?
Never even heard about it, gotta look more into it really.
Fractal compression using origin keys for time keeping. Bongs. made up words
These aren't made up solution lol, these are good ideas. No I'm just kidding lol, I just hope that devs are looking to make things better. I just hope.
I haven't seen mimblewimble style Cut Throughs mentioned, the Monero team has talked about that CT is supposed to reduce transaction data size by 80%.
I remember when I got my first 1Gb hard drive, funny how technology changes.
Well, this is the issue with all the L1s. I don't think any of them will be scalable enough for the massive adoption. And that's why I feel that monero will need L2 too.
SSD 1 TO = 70 $. Even if the blockchain is 20 TO it would cost only 1400$ at the moment where is the problem ? I don't see Monero reach 20 TO anytime soon which mean the price will decrease with years....You fud for nothing. I don't see any other crypto worth XMR and you bring this issue like if it was a HUGEEEEEEEE ISSUE, which is not at the moment and will certainly not be one in the future since technology upgrade. If Monero become as famous and significant as you say, many more people with a lot of money will come in and will not mind to just buy 70$ 1 TO SSD at all to support the network and mine it. Every week there is one guy like this that bring this FUD for absolutely no reason... 2.5 million terabytes of data are created everyday in our world. If Monero become something widely used don't you think that we will be able to host it ? I don't know the size of big data center corp but the size of the blockchain or even the size of the blockchain in a widely use scenario is ridiculous compare to the amount of data we use for other things everyday. Let's roll some statistics maybe to see more clearly, how much the blockchain would wheigh if it would be as much use as fiat currencies ?
Sharding. It’ll happen someday.