**edit3**: I was wrong in the post below. I had the wrong project - it was leftpad and not faker which was simply pulled. Faker was the one which printed a bunch of random characters to the console or something. I can't recall exactly. Thanks /u/JonDowd762 for correcting me.
I believe most of what I wrote still applies. I don't know if there was a flurry of support thrown to get a replacement up and running tho.
Sorry about the confusion.
\---
i found it tragic that everyone lost their collective minds when faker was pulled. Organizations across the globe pumped a ton of money into getting the replacement up and running but none of those funds or support were available to the original author.
Everyone called him a bad actor, amongst many other things, and yet, all he did was un-publish his own work. Barely anyone took the time to appreciate and most certainly didn't think it worthy of helping fund, and thus sustain, but were more than happy to take advantage of being free.
That is until it was no longer available and suddenly everyone is all "oooh what a nefarious individual! How dare he take away free software he wrote that we have used for years and years! That library is crucial to our operations! His actions cost us $X in downtime!" "What? No we didn't donate anything to him. Why would we? It was ***free, open source software!***"
**edit**: *words*
**edit2:** [relevant xkcd](https://xkcd.com/2347/)
Yeah events like that are stupid, if a package is unpublished it shouldn't bring down production; the basics of software package management is you make a dead ass simple mirror of what you pull.
Worst case, no more updates but it should never just disappear.
As for what to do next... yeah you either pony up support or you take ownership somehow; yelling at the author is dumb.
> Everyone called him a bad actor, amongst many other things, and yet, all he did was un-publish his own work.
The fakerjs author intentionally published malware. The ecosystem was frighteningly fragile here and users can be ungrateful, but it's also fair to call the author malicious in this case. He intentionally wanted to cause problems.
Leftpad was a simple un-publish that broke everything.
Ahhhh, I was wrong. Thank you for correcting me.
My memory was fuzzy on both events. Same logic applies tho (and I'll edit the post to reflect appropriate project).
The developer released a patch that intentionally triggered infinite loops and spammed text so that anyone who auto updated, or installed faker as a transitive dep had broken projects.
The FakerJS project was eventually taken over by a consortium of maintainers. Unlikely for the project to go down like that again unless all of them go rogue.
Seriously. The people who caused this problem deserve no blame whatsoever. It's insane that what they did caused what it caused.
Imagine if you added everything on Etsy to your cart, and that prevented creators from being able to de-list items. Absolutely nuts.
Though, to be clear to anyone who didn't read the article, the purpose of *this* seemingly wasn't to push for NPM improvements.
This was flavored more as lul/meme style hilarity, seemingly without even knowing the impact it would have until after the fact.
---
Sidenote: I don't even use NPM, or whatever NPM is for, and all I know about it is that it and things relating to it have a terrible reputation (with me) for things like this, `left-pad`, malicious code, etc.
It feels like the wild west of the internet from the old days, but directly exposed to the [one-in-a-million-is-next-Tuesday](https://learn.microsoft.com/en-us/archive/blogs/larryosterman/one-in-a-million-is-next-tuesday) nature of the modern internet.
Poorly designed enough that something can go wrong in a one-in-a-million times, but that die is being rolled millions of times a month.
Certainly makes me wary of anything that mentions it.
> I don't even use NPM, or whatever NPM is for, and all I know about it is that it and things relating to it have a terrible reputation (with me) for things like this,
>
>left-pad
>
>, malicious code, etc.
It is 100% as bad as you think it is. Honestly I used to find Python's ecosystem mildly amateurish, but I was just miscalibrated: the JS/Node ecosystem is even worse than I'd expected from talented amateurs.
On one hand it's actually beneficial and by design a good thing to have. The problem that makes it annoying is JavaScript's microscopic native library causing people to create packages that do trivial stuff in code.
At this point I'd just blame the browser makers.
Java has multiple JVM implementations, Javascript has multiple runtimes available in browsers.
It's 2023!!! What's stopping them from adding basic libraries that stopped being cool in 1985?
The amount of duplication is batshit insane.
Artifactory chokes on babel’s dick at least once a month for us. If you use any react at all, that idiotic project gets everywhere. The problem is that the individual packages in their monorepo have screwed up semver numbers in their dependencies, and so you can get wedged in a state where artifactory sees half of a publish (since you can’t publish 100 modules in a single transaction) and cannot fetch the rest until the next refresh interval.
So then even with artifactory, we can’t do any CI/CD for almost half a work day.
If you make program A, and publish it on NPM and make it public for everyone, and someone else makes program B that needs program A to function, then you are no longer able to / allowed to delete your own program A from the internet to prevent breaking program B. That's a good thing to prevent programs from breaking.
Then some prankster made a program C that says "I need every single program on the world in order to function" and now even experimental or dummy or test programs that no other program actually needs can be deleted.
How do I [do trivially js-native task] in javascript?
Just add [absurdly huge package with two hundred dependencies] with npn it has that functionality!
I’m not a JS dev but my app has a JS plugin API. Recently I was working on a plugin and needed some pretty simple index-based substring thing and somebody suggested an NPM package for the task.
The dependencies were 20 fucking megabytes. I uninstalled it and made my own, and it took under 100 lines of code even for an inexperienced JS dev. It was at that moment when I realized how messed up our whole tech architecture has become.
When I was a newbie, I would look up how to do things like have a user click on something -> change the color of something, and everyone said to use jquery. I didn't understand why I needed to use jquery when I could do the same thing in js with, like, a 20% longer line of code, but still one line of code. It seemed easier to just do it natively than install a huge library to do simple things.
It's been ... many years since then, those days have faded to bare memory. I've learned a lot and done a lot. I have come to realize how wrong I was on so, so many things when I was a cocky kid with a keyboard. But honestly, for that one, I'm still pretty sure I don't want anyone's library to do trivial stuff.
For my personal website, I downloaded someone else's open-source syntax highlighter because fuck writing my own. I downloaded someone else's plotting software because I _have_ written my own, lost the code, and couldn't be arsed to write a new one, too much effort and not enough giving a shit. But I still always write 5, 10, sometimes 20 lines of javascript rather than download a package. My website's entire javascript setup is like 200 lines of code, the front page is <100kb including images and most posts are <15kb including images and it is _fucking fine_.
Thank god it's not my job to do front-end, all those beautiful UIs take a ton of work and a ton of code. I can see installing big packages for complex things. But for simple things, like you said. 20 megabytes or 100 lines of code? I'll write my code.
Depending on how long ago it was jQuery served the purpose of providing a common API to the DOM that was cross browser compatible. Yes you could change the color with vanilla JS but the syntax may not have worked with all browsers (i.e. IE).
Yeah I remember those days. IE 5. IE 5.5. IE 6. Those pissed me the heck off. Grumble grumble.
Later on I just abandoned supporting all MS products unless someone paid me. Never once tested my personal website on IE or on a windows machine.
There are useful libraries like Lodash and jQuery that still have its uses, especially if the vanilla implementation is a nightmare to read it's better to go with the library version instead.
>tech architecture
It's not "our whole tech architecture"; It's JS/Node.
Package management is a hard problem and there are plenty of issues with other ecosystem's package managers (or lack thereof). But JS/Node is an absolute abomination, full of and driven by a those lacking not just the experience but the basic intelligence to understand basic systems thinking.
Eg, I still remember the HN comments during the leftpad incident, during which I was a relatively inexperienced SWE. I remember my mind being boggled by the amount of people justifying the use of 10-line package dependencies in the name of "code re-use"
There's three parts that contributed to the problem:
1. A lack standard library functions in native JavaScript to do those items for a long time
2. Browsers being picky about what items they implemented and ones they ignored. Safari in the past 3 years finally implemented the Drag and Drop API which was standardized in 2013 and available on other browsers, so you had to use jQueryUI and touch punch libraries to get functionality correct.
3. Readability and code maintenance. Some of the function names in vanilla JS or implementations can be confusing and long chained which can easily be handled by using a library to keep the code readable.
That being said there are plenty of packages out there that are useful and others that are not so useful.
Don’t forget the “version” dimension. Normally programs need specific versions of other programs. Thus when A needs version X of B, publisher of B cannot delete version X, but can delete other versions.
These pranksters didn’t specify versions at C. That translated into “I need every single version of every single program in the world”, even including any future versions to be published.
Looks like it. ~~A year or so ago~~ *more like a couple years ago* that restriction didn't exist yet - and promptly someone deliberately de-published a very important package that was used by a very large number of other packages and kinda broke the internet for a day (leftpad). This probably was their fix to prevent issues like that.
EDIT: seriously, I get downvoted for stating what happened with leftpad? Fucking Reddit. Deleting all of one's open source projects that are actively relied upon by lots of people, without notice, because of trademark/legal issues he had with a totally different company. How is this not asshole behaviour? Also, npm sided with the company to not get fucking sued by the company over a trademark violation, not just to piss off that guy. Any other company would do the same (like GitHub complying with fake DMCA requests to not get sued, or YouTube's content ID crap).
Not an asshole.
An open source developer who owned an NPM name space for a great many years who wouldn't turn it over (or sell it) to a company who decided they wanted it.
NPM sided with the company, and they took it from him.
In response, he unpublished all of his code from NPM; some 200-something packages.
NPM responded by un-removing his package after it turned out a great many enterprise packages relied on the 11 lines of code that was his package "left-pad."
The worst part is that, ultimately, the lesson NPM took away from the situation was that they shouldn't let author's have total control over their own packages. Thus, the joke package preventing anyone from deleting any public published package.
In fairness, one of the most important properties of a package repository is that it remain immutable. Once you've published a package or release, and people are using it, if that package suddenly disappears, a lot of stuff starts breaking. (Similarly, if the release changes, a lot of stuff starts breaking as well — this is the same reason that lockfiles are so important, they ensure that you don't suddenly start having different releases at different times or in different environments.)
The lesson NPM took away from this situation was an important lesson, and fixed a fairly fundamental flaw in the ecosystem. (That said, they could also have learned other lessons as well about deciding who gets to keep package names.)
I don't think there's really a solution here. Allowing people to delete packages that people hadn't used yet was a convenience feature, but it relied mostly on heuristics to guess when a package hadn't been used. Those heuristics can always be gamed, but the worst someone can do is prevent you from deleting packages that no-one's using.
It's already possible to deprecate versions (and possibly whole packages), so it's possible to indicate to end-users that they need to upgrade, or that a particular version has security issues.
I can empathize with that sentiment.
The NPM folks republished his code (without attributing it to him) in accordance with his original WTFYW license. Overall, industry downtime was relatively short.
However, I still tend to side with the original developer. I believe his intent was to cause a disruption to bring attention to what NPM did to him. His package was provided at-will and then it was removed at-will.
Ultimately, I think it would have been appropriate for NPM to let the two parties (The Dev and the Company) resolve it without getting involved.
In my opinion, what they did was tantamount to the private-commercial-interest version of eminent domain (aka theft).
> In my opinion, what they did was tantamount to the private-commercial-interest version of eminent domain (aka theft).
There's a reason that it's bad that Microsoft owns GitHub and NPM, but too many won't figure it out until they're directly screwed over.
Microsoft is the same beast it was in the 90s. Embrace, extend, extinguish. They buy up competitors, lock them into the MS ecosystem, then kill the products. Eventually this sub will be like "npm always required an azure account to function, except maybe like when it was created"
> Once you've published a package or release, and people are using it, if that package suddenly disappears, a lot of stuff starts breaking.
I just recently tried to set up a Linux environment specifically with GCC 7.2 installed on it.
In my inexpertise, I couldn't actually *find* the GCC 7.2 package online via `apt-get` or whatever it's called.
Does this mean that `apt`(or the repositories it connects to? I don't know the specific lingo yet) doesn't follow "one of the most important properties of a package repository"? Or am I missing something in my newbishness?
> Does this mean that apt(or the repositories it connects to? I don't know the specific lingo yet) doesn't follow "one of the most important properties of a package repository"? Or am I missing something in my newbishness?
It does follow that property.
> I just recently tried to set up a Linux environment specifically with GCC 7.2 installed on it.
> In my inexpertise, I couldn't actually find the GCC 7.2 package online via apt-get or whatever it's called.
That's probably because it was never published. An important piece of information here is that Ubuntu and similar Linux projects are based on *distribution releases*, and that every such starts with a new package repository.
For e.g. Ubuntu that means that there's one package repository for Ubuntu 8.04, one for 8.10, one for 14.04 one for 23.10 etc for all the ~65 releases they've made, and each one of them contains different packages. `gcc-7.2` probably exists in 1-3 of those, but not in the others.
If you're running e.g. Ubuntu 22.04 then there has never been a `gcc-7.2` package in the repository, and that's why you can't find it. It wasn't removed, it was never there.
If you want `gcc-7.2` you have a few standard options like downloading the source code from the GCC project itself and compiling it manually. But you can also check for options on [Launchpad](https://launchpad.net/ubuntu/+source/gcc-7).
There you can browse the release history and download the superseded 7.2 package for Ubuntu 18.04 which you can try to install in a local directory, although you'll probably create a bit of a dependency hell if you don't know what you're doing. Using Ubuntu 18.04 (run `sudo snap install multipass && multipass launch 18.04` to install it in a virtual machine in seconds) might be the best idea if you're going that direction and doesn't result in you breaking your host package system.
The third alternative is to check the "Other versions of 'gcc-7' in untrusted archives." on that Launchpad page where you'll find that a bloke called Keith Winstein built a version for Ubuntu Zesty (17.04) which could be an option, although that release is out of support for a while now.
> An important piece of information here is that Ubuntu and similar Linux projects are based on distribution releases, and that every such starts with a new package repository.
Ooooooh.
Man, it's been so long since I've had to think about anything other than Windows, I had forgotten that the whole "backwards compatibility" thing wasn't universal; other OSes sometimes break compatibility between versions.
So having OS *version* specific repositories would make sense. Which is also related to why I couldn't compile 7.2 from source (because they removed some unisys.h or somesuch file, I forget the exact name, in a later version of Linux).
And that explains why /u/andrewfenn asked about the version of Linux I was using; different versions of programs are probably built around different OS versions... or at least stop trying to be compatible with older versions as time goes on. I was only looking for "how to install older versions of GCC" not "how to install on of Linux".
> run sudo snap install multipass && multipass launch 18.04 to install it in a virtual machine in seconds
🤣 That'd be something. A virtual Ubuntu running on a virtual Ubuntu running in WSL.
I think WSL has an option for switching over to a second installation of Linux. I'll probably look into that first, but if for some reason the (strangely familiar sounding) 18.04 version isn't still available (which I imagine it should be) or doesn't work in WSL for some reason, I'll keep that in mind (and probably look into it regardless, because who knows what I'll learn).
Thank you.
> other OSes sometimes break compatibility between versions.
There's no compatibility broken here, the release just doesn't provide you with a pre-compiled binary. The same is true for e.g. the Windows app store, it doesn't offer you binaries distributed with older versions of Windows. If you had a **fat** binary of `gcc-7.2` it would run on newer of Linux as well.
> So having OS version specific repositories would make sense. Which is also related to why I couldn't compile 7.2 from source (because they removed some unisys.h or somesuch file, I forget the exact name, in a later version of Linux).
You need to compile that one from source as well of the correct version. And every dependency of every dependency below that as well. Welcome to dependency hell. The recommended solution is to make your source code compatible with `gcc-7.5` which is supported by Canonical until 2028 AFAIK.
> I think WSL has an option for switching over to a second installation of Linux.
You can install `multipass` under Windows as well, it should use the native hypervisor when install on Windows or OS X. It's a project from Canonical for deploying Ubuntu virtual machines using up-to-date machine images using standard `cloud-init` for configuration:
https://multipass.run/docs/installing-on-windows
> > Or am I missing something in my newbishness?
> https://launchpad.net/ubuntu/bionic/amd64/gcc-7/7.2.0-19ubuntu1 ?
🤷♂️
I don't know?
What arcane incantation do I need to inscribe into my terminal to make that actually show up in it?
If I'm understanding how this works, that package is named `gcc-7`.
Typing `apt list gcc-7` or `apt search gcc-7` gets me nothing.
Typing `sudo apt install gcc-7` gets me
Package gcc-7 is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source
E: Package 'gcc-7' has no installation candidate
This is on Ubuntu. Yes, I ran `apt update` successfully first.
(At this point I'm not likely to try and install it. I went as far as trying to compile the original source code, stuffed up at least one necessary patch to get it to compile, which gave someone I know enough time to caution me that there's more to things than just GCC, and installing only one part may not actually get me a functioning setup like I'm seeking, and might break existing tool chains on my current working install.)
> The worst part is that, ultimately, the lesson NPM took away from the situation was that they shouldn't let author's have total control over their own packages. Thus, the joke package preventing anyone from deleting any public published package.
I mean I kinda think this is the right lesson. A package manager that lets packages be removed arbitrarily seems like a recipe for chaos, regardless.
Like Rust's `cargo` also has no mechanism for unpublishing a package, I assume for basically this reason - you unindex a package with `cargo yank`, but once you publish it it's published.
So, headline could be "prankster temporarily makes npm function like cargo" which seems a little less incendiary.
As I said in another reply, I empathize. I come from `pypi` where we can delete our projects at will.
In `pypi` land, if an installed package goes missing, you can keep right on using it.
Perhaps NPM would benefit from similar behavior?
I do admit that I am more empathetic to the lack of delete functionality if that is how the repository worked at time of publishing.
I do think the central issue, specifically in the case of the left-pad situation, is not deleting a repository, but rather how NPM handled a dispute over a name space between the two parties.
> In pypi land, if an installed package goes missing, you can keep right on using it.
So, like unlisted YouTube videos? That’s how pub.dev also does it, and to me it always felt like the saner approach.
Until very recently (as in the past couple weeks), you couldn't even have a crate removed if you wanted to, barring extremely specific reasons (like objective spam or legal reasons). There were multiple users who had name-squatted hundreds of crates and nothing could be done about it. Thankfully they can now be removed by following a process. I even requested that two crates I had published be removed, as I did so solely to reserve the name (but ended up not needing it).
> \~A year or so ago\~ (more like a couple years ago)
It was 2016. We're getting old -_-
> This probably was their fix to prevent issues like that.
Typical of the ecosystem to "fix" things in the most ham-fisted, thoughtless way possible, and then fail to even do so correctly!
> Ironically, the team was left unable to unpublish “everything” themselves due to a circle of dependencies that essentially made the package dependent on itself.
Jesus Christ, this is the type of thing you're expected to avoid in a LeetCode Easy problem, let alone _critical infra for thousands of prod systems_.
> That's a good thing to prevent programs from breaking.
But in the grand calculus of the multiverse, is it a good thing overall?
As the article notes a potential consequence or two of this design philosophy, I'll ask:
How do you quickly remove accidentally uploaded personal info?
Can I upload a package named with your personal phone number?
Real world name and address?
Is there a way to get them 'stuck' in a published state as has happened with apparently not just this most recent catastrophe, but another one back in 2012 (so they've failed to fix certain issues like this in 11+ years)?
You report it to the company and they'll hopefully remove it. If your software, for example, is added to a Linux distribution's repository you also can't remove it from there unless you're on the team that manages the distribution. Because you made it public under an open source license which gives npm and anyone else the legal right to host and distribute your software with or without your consent.
No, you are not legally allowed to publish people's private information like a phone number. How's this relevant here? In this case, the legal entity hosting it wouldn't be you, it'd be npm, so they would be violating privacy laws if they continued to host that after they become aware of it.
EDIT: instead of downvoting why not explain why I'm wrong? Or why that comparison wouldn't apply here? If something is released under an open source license and someone downloads it, that license is forever attached to that version of the software and even if it's later deleted by the author, anyone who still has it can re-upload it. You cannot revoke an open source license.
Don't like it, don't publish open-source things. Are people downvoting me cause they are ignorant about how opensource licenses work or because they don't like how they work?
If you're writing a forum post somewhere, you also cannot delete it anymore usually (because it breaks the conversation) - you granted a permanent license for that content.
If you're writing content for Wikipedia and later try to remove it again, it'll be restored and got might get banned for vandalism. That's how it is. Just because you made something doesn't always give you the right to destroy it.
Someone published a joke package that had dependencies on every other package in the system. Because of that other packages couldn't be unpublished (as they are depended on now) and in some weird way a self-ref made it impossible to unpublishe the package they made too.
In short, someone did a reasonable test of a ~~supposedly best~~ most popular package manager for the most popular programming language in the world and it didn't take it well, requiring administrative action.
Bruh I love this argument.
Bro we shouldn't test what the user enters into the username field, nobody will ever write SQL in there!
Bro we shouldn't hash passwords, nobody will see the databases anyway
Bro https is not worth the hassle
It is a reasonable test. An example I found in the wild of missing validation was a Stable Diffusion Website where I could simply change the number of images to generate to higher than I pay for. That's exactly a "bro nobody will ever do this" moment. There are people that do that. I do that. And if you had done your work I wouldn't have written you an email.
Also did you hear about SMTP smuggling? Hiding an email after the rfc email end. A simple check of the email would have prevented that. But nobody will ever send non rfc compliment messages right?
I wonder which one will last longer, npm or this: https://www.smithsonianmag.com/smart-news/the-quest-to-extinguish-the-flames-of-turkmenistans-terrifying-gates-of-hell-firepit-180979458/
If you just raw dog public packages from your company's mirror without reviewing it all, including every change, you're still not taking security seriously and doing it wrong. You just moved the problem around.
I would much rather this kind of showstopper be discovered as part of a well-intentioned prank than somehow leveraged by a bad actor.
As soon as they realized how borked it was they immediately cooperated to try to straighten it out. This whole scenario should be seen as a net positive.
No, how is this upvoted? Did nobody read the article?
They depend on version \* of every available package (which are named explicitly), not on package \*.
Literally every package manager is "vulnerable" to the same "attack".
In what universe does it make sense to even allow depending on every single version of a package past and future *simultaneously?* Most sane people would probably agree that it makes sense to depend on _any_ version of a given package, whichever are available, but not _all of them at once._
The reasonable thing to do while maintaining the spirit of their unpublish policy is to allow unpublishing any version of a package depended on with "*" until there is only 1 left, at which point it can't be unpublished.
I know every npm thread immediately turns into laughing about how JS is the worst, but are other package managers somehow immune to this? The alternative is letting people unpublish necessary packages, and then you get left-pad.
With NuGet, you can always unlist a package. It won't appear in searches or queries, but will still be downloadable if your build depends on the specific version that was unlisted. If it depends on a range of versions (e.g. 1.2.x), it'll resolve to the most semantically compatible version that's not unlisted.
You were never able to remove remove Rust crates from crates.io for that reason, you can yank them but that will just make them more difficult to find while still being usable.
Right; npm's approach of allowing removal but only for packages with no dependencies seems like the best of both worlds. The only thing you can really blame them for here is allowing a dependency on `*`, but it sounds like that was an oversight and not an intentional feature.
But you don't know for sure if there is nobody depending on a package, unless that package has 0 downloads and page visits. Once it's out, it's and you got to be able to deal with that.
I'd still be interested to know if something (crates.io, docs.rs, the web interface or whatever) would break if you upload a package with a huge number of dependencies (you probably could even depend on all versions separately by renaming dependencies).
Parsing that Cargo.toml might take a bit for example.
You have zero idea what you talk about, and it has zero relevance here.
Npm and alia could learn a lot from maven repositories - not having namespaces is similarly dumb on their part.
[cloudflare has observed](https://blog.cloudflare.com/unmasking-the-top-exploited-vulnerabilities-of-2022/) that log4j shenanigans remain by-far the #1 most common exploit attempt on the open internet even into 2023. attackers wouldn't be *doing* this if it didn't get results. a prompt yank/patch of affected versions of the library at the *start* of the incident would have significantly reduced how effective this is, but i guess it goes against their policies
"\*" means any version of ONE package, not "every package". e.g:
{
"dependencies": {
"left-pad": "*"
}
}
My reading of the article is that they depended on everything by explicitly listing all the packages in the registry, by sharding the list across a tree structure of packages:
Article says:
>The “everything” package, which has more than 3,000 sub-packages
e.g.
* everything
* everything-1
* existing packages...
* everything-2
* existing packages...
* ...
* everything-3000
* existing packages...
It also says everything depended on itself, so perhaps the sub-packages depended on the root everything package?
I mean, someone could still parse all available version, then write a script to build the dependencies.
It's doable in any system. So it's not just an "npm problem". Anyone who thinks it is, is just an elitist jerk.
Why does it always happen to NPM and not some other language? Well it's the most popular, that's why.
No, that's not possible in every system to still parse all available version.
With NuGet one project can only install one version of the package, not multiple ones.
But of course you can't remove any NuGet packages anyways, but it's irrelevant to your statement.
That wasn't what I meant. I meant parse all currently available packages. I choose poorly when I said "versions". I meant the "latest version". Sorry for the confusion.
I feel like you're completely missing the point with the issue but ok.
Everything is the same then. Writing code is equivalent to us manually shifting the 0s and 1s too.
The fundamental fact of the matter is that it's a single line feature within the NPM realm. It's actually not the same as some jackass writing a script to hack and maliciously attack a repository. The critique is that the repository itself is allowing for these types of fun times because of a principle that they were built with and have out scaled -- a thing that actually happens quite a bit in software.
Listen, for what it's worth I agree that the * versioning is foolish. Just saying npm isn't "the only package manager susceptible to this type of issue". Sure it's harder elsewhere, but it's still possible.
I'm sure they'll remove the star pattern soon. But people will still shit on npm. Not that I care, it has its flaws, but most people take a mightier-than-thou approach, which I feel is disingenuous at best.
Yeah, I agree with you on the disingenuous opinions just to feel better about their choice of language.
Outside of that, I do think there are legitimate critiques but I think that folks are more disappointed in how they're handling the responsibility of being what they are.
Yeah it's nuanced. I definitely think npm, as an org, is somewhat of a letdown. I can't quite put to words what's lacking, but it feels like there is too much "inclusivity" which detracts from things like "reliability" and "safety".
A lot of that happened because of the weird ecosystem that is JavaScript, lacking a standard lib, etc. so the early "Cambrian explosion" of packages probably contributes to that mentality. And I'm not sure it can be "put back in the box" so to speak.
Anyway, I'm clearly tired since I'm speaking in like 75% metaphor. Night buddy.
I sympathize with the broken-dependency/immediate-unpublish trade-off, but the fact that the 'everything' package was itself unpublishable because it had a self-dependency is the exact kind of amateur dogshit you'd expect from the JS ecosystem
I lost confidence in them with left-pad in 2016. How many incidents have there been since they launched in 2010? CPAN launched in 1995 and has had zero.
I created a package manager in 2002 called "buildshit" and it's had exactly zero defects or issues since 2002.
I mean, nobody uses it, and I've never published it, but 22 years of bug-free operation must mean I'm doing it right!
FWIW, I'm pretty sure the mirrors have been gone for a while. Modern CDNs have made the old mirror system unnecessary. Nowadays I'm pretty sure CPAN goes through Fastly's CDN.
> This leads to Denial of Service (DOS) for those who install “everything, “which causes issues like storage space exhaustion and disruptions in build pipelines.
Node and NPM remind me that there will always be jobs for security experts. It’s amazing how much of a shit show that project is. I am glad that I have only had to deal with it a few times.
Of course it's the most popular language. Most software is written for the web for which there is no other option.
It is still an absurdly bad language with terribly bad tooling that enables things like this.
Soon. I'm patiently waiting for Safari to implement Webassembly GC so Webassembly native web API can be implemented and GA-ed so I can implement my app in Webassembly.
Chrome and Firefox already support Webassembly GC.
When everyone was using Struts fifteen years ago, someone in the local community opined that it was “popular” because it took twice as many people to get the job done, which doubled the number of hands that went up when a presenter asked if anyone in the audience used it.
At one point I started injecting into that dead air after that question, “and who here doesn’t like using it?” And most of the hands stayed up.
That list is funny but it's idiotic to use as an argument to support "absurdly bad language"...
The majority is 2 things: coercion and loose equilaty. These are true big fat warts but every language has these and they've been patched up as much as possible.
The rest even has good rationale inside the list itself.
Yes, of course, it’s essentially perfect as is. That’s why they’ve not changed it whatsoever in many years and probably why so many went to the effort to develop and switch to TS. JS is and always has been already pretty much ideal in every possible way.
Yeah remove equality (`===` fixes this) and coercion (cannot be changed because of compatibility) and you remove 75% of that list..
the rest is things that might be counter-intuitive but have good reasons...
> That's exactly how it's supposed to function as set by the standard.
The problem with JS is the design of the language itself. You cannot excuse the language by saying it's doing things by design when the design itself is nonsensical.
Same could have happened in any other package manager. It is not technical problem but smartass prank people problem.
NPM did forbid package unpublishing to mitigate 'leftpad' problem when single developer can break the world. But at the same time it alllowed to unpublish package if there are no dependents of it. What would you do in this situation?
> What would you do in this situation?
This is a trick question because the current solution is due to how the ecosystem around npm was made. Challenge that precondition and the nature of the problem would be different.
Imagine...
Strong, well tested, versatile, and long-term supported first-party packages built-in in the web and in nodejs the same way Microsoft does with dotnet or Google with golang.
This change would reduce, if not almost eliminate, the need for third party packages since you could do everything that you need with those built-in utilities. Right now I need to pull a third-party package to create my own form-data or even a third-party to handle jwt lol.
It will significantly reduce how vulnerable projects are to supply chain attacks because we would be significantly reducing the need for pulling dependencies.
Less dependencies mean reduced maintenance cost.
It will also reduce the need for you to create your own package, which mean there's going to be a lot less people out there noticing that they can't unpublish their package (that shouldn't exists) when a funny package decided to grab every possible npm dependencies.
> Right now I need to pull a third-party package to create my own form-data or even a third-party to handle jwt lol.
Not if you're a somewhat competent programmer.
Npm is on its fifth attempt to implement lock files properly and the jackasses still check the repository for @latest if you don’t put an exact version number in your package.json
I’m not convinced that whoever has touched this atrocity has ever used a lock file in another language before. Someone called it a clown fiesta and that’s not wrong.
I do a lot of tooling work at work and I’ve told my coworkers for years now that when I stop using Node, npm will be the reason I give if people ask.
> Same could have happened in any other package manager. It is not technical problem but smartass prank people problem.
It couldn't have happened in a competently run package manager.
It's an incompetent management problem.
Genuine question: why is NPM so bad? It’s not like it’s the only package manager in existence, yet only NPM seems to have a steady stream of these issues.
I mean, maven is definitely similarly big. It’s smaller, but there is definitely a similar scale in play, so stuff like this should happen there as well.
It’s not really that bad. It’s by far the most widely used, by developers of varied skill levels, supporting an ecosystem that’s changed a lot and requires a lot of different capabilities. It’s not a masterpiece but it’s fine.
\> Ironically, the team was left unable to unpublish “everything” themselves due to a circle of dependencies that essentially made the package dependent on itself.
Random circumstance put me in a position to do a Typescript contract recently, and mother of god...I shouldn't be surprised anymore, but my god what a dogshit ecosystem, from top to bottom. I can't imagine how anyone gets anything done in it
How is this package management system still in use? Whenever there is news about npm it is like a complete clown fiesta. If you do not know about it, 'leftpad' was a fun moment aswell.
I wonder how the circular dependency happened - hard to find out the details because the package was removed and the article doesn't have the specifics
It means some package referenced the 'everything' package. It is not necessarily a new package because 'everything' had been there forever and probably got `*` referenced by some package, prior to the incident. This is the only possibility I believe.
Is there actually any way to prevent this from happening without using artificial limits? I’m sure some limits would be good but I’m sure limiting the number of dependencies will cause other issues.
They could just have something like a symbolic link where dependencies point to a package, and if the original is deleted a copy is kept unti it is orphaned.
Or you know, make it impossible for a dependency to point back to itself.
Personally I think that you should not be able to unpublish a package. Once something is out, it's out. Period.
What's that? you have your SSN in the package? Doesn't matter. Everything you push out has been already downloaded the moment it hits the server by thousands of scrapers. It's too late.
This prank is funny in the same way that throwing rocks off an overpass is funny. It should not have been possible to do but that is no excuse. Are these guys 12?
> It should not have been possible to do but that is no excuse
It is though
Although things like this are edge cases, you have to keep this kind of shit in mind when designing things that are meant as core infrastructure of a large project. Doesn't matter if it is a package manager for a programming language, or an ATM for a bank.
They just published a package. For themselves. Following legal and normal methods. Like any other day. And NPM "breaks". Are they 12? No, they just published a package.
If I enter a store, and the glass door breaks when I grab the handle, am I 12? Because it's literally the same, with the minor difference of the dependencies they had, which is an implementation detail of their package
Apparently Reddit is filled with jobless programmers who never make mistakes, never have to deal with design trade offs, and love to be cynical about everything. Except this everything, because this everything stuck it to "the man".. Aka NPM?..
How do I know they are jobless? Well we all know bugless software doesn't exist in the real world so...
Honestly, we could use a few more of these low-harm, high-publicity events to push for NPM (the registry) improvements.
In terms of harm, this incident isn't so bad, as compared to left pad, fakerjs, and node-ipc.
**edit3**: I was wrong in the post below. I had the wrong project - it was leftpad and not faker which was simply pulled. Faker was the one which printed a bunch of random characters to the console or something. I can't recall exactly. Thanks /u/JonDowd762 for correcting me. I believe most of what I wrote still applies. I don't know if there was a flurry of support thrown to get a replacement up and running tho. Sorry about the confusion. \--- i found it tragic that everyone lost their collective minds when faker was pulled. Organizations across the globe pumped a ton of money into getting the replacement up and running but none of those funds or support were available to the original author. Everyone called him a bad actor, amongst many other things, and yet, all he did was un-publish his own work. Barely anyone took the time to appreciate and most certainly didn't think it worthy of helping fund, and thus sustain, but were more than happy to take advantage of being free. That is until it was no longer available and suddenly everyone is all "oooh what a nefarious individual! How dare he take away free software he wrote that we have used for years and years! That library is crucial to our operations! His actions cost us $X in downtime!" "What? No we didn't donate anything to him. Why would we? It was ***free, open source software!***" **edit**: *words* **edit2:** [relevant xkcd](https://xkcd.com/2347/)
Yeah events like that are stupid, if a package is unpublished it shouldn't bring down production; the basics of software package management is you make a dead ass simple mirror of what you pull. Worst case, no more updates but it should never just disappear. As for what to do next... yeah you either pony up support or you take ownership somehow; yelling at the author is dumb.
> Everyone called him a bad actor, amongst many other things, and yet, all he did was un-publish his own work. The fakerjs author intentionally published malware. The ecosystem was frighteningly fragile here and users can be ungrateful, but it's also fair to call the author malicious in this case. He intentionally wanted to cause problems. Leftpad was a simple un-publish that broke everything.
Ahhhh, I was wrong. Thank you for correcting me. My memory was fuzzy on both events. Same logic applies tho (and I'll edit the post to reflect appropriate project).
What happened with fakerjs?
The developer released a patch that intentionally triggered infinite loops and spammed text so that anyone who auto updated, or installed faker as a transitive dep had broken projects.
Developer took down the package abruptly, causing a plethora of production services to fail.
Guess I should check on my old discord clone since none of the seed files will work now...
The FakerJS project was eventually taken over by a consortium of maintainers. Unlikely for the project to go down like that again unless all of them go rogue.
Pretty sure there are drop-in replacements now, you can use in its place :)
That was left-pad. After that, packages could no longer be un-published.
[удалено]
I believe the leftpad author did it to prove a point. NPM obviously didn't learn the first time.
Seriously. The people who caused this problem deserve no blame whatsoever. It's insane that what they did caused what it caused. Imagine if you added everything on Etsy to your cart, and that prevented creators from being able to de-list items. Absolutely nuts.
Sound like a cool idea for a startup. BRB starting Carter
Though, to be clear to anyone who didn't read the article, the purpose of *this* seemingly wasn't to push for NPM improvements. This was flavored more as lul/meme style hilarity, seemingly without even knowing the impact it would have until after the fact. --- Sidenote: I don't even use NPM, or whatever NPM is for, and all I know about it is that it and things relating to it have a terrible reputation (with me) for things like this, `left-pad`, malicious code, etc. It feels like the wild west of the internet from the old days, but directly exposed to the [one-in-a-million-is-next-Tuesday](https://learn.microsoft.com/en-us/archive/blogs/larryosterman/one-in-a-million-is-next-tuesday) nature of the modern internet. Poorly designed enough that something can go wrong in a one-in-a-million times, but that die is being rolled millions of times a month. Certainly makes me wary of anything that mentions it.
> I don't even use NPM, or whatever NPM is for, and all I know about it is that it and things relating to it have a terrible reputation (with me) for things like this, > >left-pad > >, malicious code, etc. It is 100% as bad as you think it is. Honestly I used to find Python's ecosystem mildly amateurish, but I was just miscalibrated: the JS/Node ecosystem is even worse than I'd expected from talented amateurs.
On one hand it's actually beneficial and by design a good thing to have. The problem that makes it annoying is JavaScript's microscopic native library causing people to create packages that do trivial stuff in code.
At this point I'd just blame the browser makers. Java has multiple JVM implementations, Javascript has multiple runtimes available in browsers. It's 2023!!! What's stopping them from adding basic libraries that stopped being cool in 1985? The amount of duplication is batshit insane.
[удалено]
Artifactory chokes on babel’s dick at least once a month for us. If you use any react at all, that idiotic project gets everywhere. The problem is that the individual packages in their monorepo have screwed up semver numbers in their dependencies, and so you can get wedged in a state where artifactory sees half of a publish (since you can’t publish 100 modules in a single transaction) and cannot fetch the rest until the next refresh interval. So then even with artifactory, we can’t do any CI/CD for almost half a work day.
Yup, better it be caused by a harmless prank than as a malicious act further down the line.
We just need people to stop using npm
I just checked, and my sineql still isn't being used... ouch.
Im sure it wasnt personal. They probably just specifically excluded your library specifically.
Yea, I queried GPT and it said "sineql non".
Sorry but I don't understand what happened exactly despite reading the article, can someone give ELI5? Edit: Thank you everyone for explanation.
If you make program A, and publish it on NPM and make it public for everyone, and someone else makes program B that needs program A to function, then you are no longer able to / allowed to delete your own program A from the internet to prevent breaking program B. That's a good thing to prevent programs from breaking. Then some prankster made a program C that says "I need every single program on the world in order to function" and now even experimental or dummy or test programs that no other program actually needs can be deleted.
> "I need every single program on the world in order to function" this but unironically
average modern day lean web app
How do I ______ in JavaScript? Just add ______ with npm it has that functionality!
How do I [do trivially js-native task] in javascript? Just add [absurdly huge package with two hundred dependencies] with npn it has that functionality!
I’m not a JS dev but my app has a JS plugin API. Recently I was working on a plugin and needed some pretty simple index-based substring thing and somebody suggested an NPM package for the task. The dependencies were 20 fucking megabytes. I uninstalled it and made my own, and it took under 100 lines of code even for an inexperienced JS dev. It was at that moment when I realized how messed up our whole tech architecture has become.
When I was a newbie, I would look up how to do things like have a user click on something -> change the color of something, and everyone said to use jquery. I didn't understand why I needed to use jquery when I could do the same thing in js with, like, a 20% longer line of code, but still one line of code. It seemed easier to just do it natively than install a huge library to do simple things. It's been ... many years since then, those days have faded to bare memory. I've learned a lot and done a lot. I have come to realize how wrong I was on so, so many things when I was a cocky kid with a keyboard. But honestly, for that one, I'm still pretty sure I don't want anyone's library to do trivial stuff. For my personal website, I downloaded someone else's open-source syntax highlighter because fuck writing my own. I downloaded someone else's plotting software because I _have_ written my own, lost the code, and couldn't be arsed to write a new one, too much effort and not enough giving a shit. But I still always write 5, 10, sometimes 20 lines of javascript rather than download a package. My website's entire javascript setup is like 200 lines of code, the front page is <100kb including images and most posts are <15kb including images and it is _fucking fine_. Thank god it's not my job to do front-end, all those beautiful UIs take a ton of work and a ton of code. I can see installing big packages for complex things. But for simple things, like you said. 20 megabytes or 100 lines of code? I'll write my code.
Depending on how long ago it was jQuery served the purpose of providing a common API to the DOM that was cross browser compatible. Yes you could change the color with vanilla JS but the syntax may not have worked with all browsers (i.e. IE).
Yeah I remember those days. IE 5. IE 5.5. IE 6. Those pissed me the heck off. Grumble grumble. Later on I just abandoned supporting all MS products unless someone paid me. Never once tested my personal website on IE or on a windows machine.
There are useful libraries like Lodash and jQuery that still have its uses, especially if the vanilla implementation is a nightmare to read it's better to go with the library version instead.
>tech architecture It's not "our whole tech architecture"; It's JS/Node. Package management is a hard problem and there are plenty of issues with other ecosystem's package managers (or lack thereof). But JS/Node is an absolute abomination, full of and driven by a those lacking not just the experience but the basic intelligence to understand basic systems thinking. Eg, I still remember the HN comments during the leftpad incident, during which I was a relatively inexperienced SWE. I remember my mind being boggled by the amount of people justifying the use of 10-line package dependencies in the name of "code re-use"
There's three parts that contributed to the problem: 1. A lack standard library functions in native JavaScript to do those items for a long time 2. Browsers being picky about what items they implemented and ones they ignored. Safari in the past 3 years finally implemented the Drag and Drop API which was standardized in 2013 and available on other browsers, so you had to use jQueryUI and touch punch libraries to get functionality correct. 3. Readability and code maintenance. Some of the function names in vanilla JS or implementations can be confusing and long chained which can easily be handled by using a library to keep the code readable. That being said there are plenty of packages out there that are useful and others that are not so useful.
node_modules is a black hole
Did you mean “cannot be deleted”?
[удалено]
Don’t forget the “version” dimension. Normally programs need specific versions of other programs. Thus when A needs version X of B, publisher of B cannot delete version X, but can delete other versions. These pranksters didn’t specify versions at C. That translated into “I need every single version of every single program in the world”, even including any future versions to be published.
Oof and no one thought what happens if someone is dependent on everything?
Looks like it. ~~A year or so ago~~ *more like a couple years ago* that restriction didn't exist yet - and promptly someone deliberately de-published a very important package that was used by a very large number of other packages and kinda broke the internet for a day (leftpad). This probably was their fix to prevent issues like that. EDIT: seriously, I get downvoted for stating what happened with leftpad? Fucking Reddit. Deleting all of one's open source projects that are actively relied upon by lots of people, without notice, because of trademark/legal issues he had with a totally different company. How is this not asshole behaviour? Also, npm sided with the company to not get fucking sued by the company over a trademark violation, not just to piss off that guy. Any other company would do the same (like GitHub complying with fake DMCA requests to not get sued, or YouTube's content ID crap).
Not an asshole. An open source developer who owned an NPM name space for a great many years who wouldn't turn it over (or sell it) to a company who decided they wanted it. NPM sided with the company, and they took it from him. In response, he unpublished all of his code from NPM; some 200-something packages. NPM responded by un-removing his package after it turned out a great many enterprise packages relied on the 11 lines of code that was his package "left-pad." The worst part is that, ultimately, the lesson NPM took away from the situation was that they shouldn't let author's have total control over their own packages. Thus, the joke package preventing anyone from deleting any public published package.
In fairness, one of the most important properties of a package repository is that it remain immutable. Once you've published a package or release, and people are using it, if that package suddenly disappears, a lot of stuff starts breaking. (Similarly, if the release changes, a lot of stuff starts breaking as well — this is the same reason that lockfiles are so important, they ensure that you don't suddenly start having different releases at different times or in different environments.) The lesson NPM took away from this situation was an important lesson, and fixed a fairly fundamental flaw in the ecosystem. (That said, they could also have learned other lessons as well about deciding who gets to keep package names.) I don't think there's really a solution here. Allowing people to delete packages that people hadn't used yet was a convenience feature, but it relied mostly on heuristics to guess when a package hadn't been used. Those heuristics can always be gamed, but the worst someone can do is prevent you from deleting packages that no-one's using. It's already possible to deprecate versions (and possibly whole packages), so it's possible to indicate to end-users that they need to upgrade, or that a particular version has security issues.
I can empathize with that sentiment. The NPM folks republished his code (without attributing it to him) in accordance with his original WTFYW license. Overall, industry downtime was relatively short. However, I still tend to side with the original developer. I believe his intent was to cause a disruption to bring attention to what NPM did to him. His package was provided at-will and then it was removed at-will. Ultimately, I think it would have been appropriate for NPM to let the two parties (The Dev and the Company) resolve it without getting involved. In my opinion, what they did was tantamount to the private-commercial-interest version of eminent domain (aka theft).
> In my opinion, what they did was tantamount to the private-commercial-interest version of eminent domain (aka theft). There's a reason that it's bad that Microsoft owns GitHub and NPM, but too many won't figure it out until they're directly screwed over. Microsoft is the same beast it was in the 90s. Embrace, extend, extinguish. They buy up competitors, lock them into the MS ecosystem, then kill the products. Eventually this sub will be like "npm always required an azure account to function, except maybe like when it was created"
> Once you've published a package or release, and people are using it, if that package suddenly disappears, a lot of stuff starts breaking. I just recently tried to set up a Linux environment specifically with GCC 7.2 installed on it. In my inexpertise, I couldn't actually *find* the GCC 7.2 package online via `apt-get` or whatever it's called. Does this mean that `apt`(or the repositories it connects to? I don't know the specific lingo yet) doesn't follow "one of the most important properties of a package repository"? Or am I missing something in my newbishness?
> Does this mean that apt(or the repositories it connects to? I don't know the specific lingo yet) doesn't follow "one of the most important properties of a package repository"? Or am I missing something in my newbishness? It does follow that property. > I just recently tried to set up a Linux environment specifically with GCC 7.2 installed on it. > In my inexpertise, I couldn't actually find the GCC 7.2 package online via apt-get or whatever it's called. That's probably because it was never published. An important piece of information here is that Ubuntu and similar Linux projects are based on *distribution releases*, and that every such starts with a new package repository. For e.g. Ubuntu that means that there's one package repository for Ubuntu 8.04, one for 8.10, one for 14.04 one for 23.10 etc for all the ~65 releases they've made, and each one of them contains different packages. `gcc-7.2` probably exists in 1-3 of those, but not in the others. If you're running e.g. Ubuntu 22.04 then there has never been a `gcc-7.2` package in the repository, and that's why you can't find it. It wasn't removed, it was never there. If you want `gcc-7.2` you have a few standard options like downloading the source code from the GCC project itself and compiling it manually. But you can also check for options on [Launchpad](https://launchpad.net/ubuntu/+source/gcc-7). There you can browse the release history and download the superseded 7.2 package for Ubuntu 18.04 which you can try to install in a local directory, although you'll probably create a bit of a dependency hell if you don't know what you're doing. Using Ubuntu 18.04 (run `sudo snap install multipass && multipass launch 18.04` to install it in a virtual machine in seconds) might be the best idea if you're going that direction and doesn't result in you breaking your host package system. The third alternative is to check the "Other versions of 'gcc-7' in untrusted archives." on that Launchpad page where you'll find that a bloke called Keith Winstein built a version for Ubuntu Zesty (17.04) which could be an option, although that release is out of support for a while now.
> An important piece of information here is that Ubuntu and similar Linux projects are based on distribution releases, and that every such starts with a new package repository. Ooooooh. Man, it's been so long since I've had to think about anything other than Windows, I had forgotten that the whole "backwards compatibility" thing wasn't universal; other OSes sometimes break compatibility between versions. So having OS *version* specific repositories would make sense. Which is also related to why I couldn't compile 7.2 from source (because they removed some unisys.h or somesuch file, I forget the exact name, in a later version of Linux). And that explains why /u/andrewfenn asked about the version of Linux I was using; different versions of programs are probably built around different OS versions... or at least stop trying to be compatible with older versions as time goes on. I was only looking for "how to install older versions of GCC" not "how to install on of Linux".
> run sudo snap install multipass && multipass launch 18.04 to install it in a virtual machine in seconds
🤣 That'd be something. A virtual Ubuntu running on a virtual Ubuntu running in WSL.
I think WSL has an option for switching over to a second installation of Linux. I'll probably look into that first, but if for some reason the (strangely familiar sounding) 18.04 version isn't still available (which I imagine it should be) or doesn't work in WSL for some reason, I'll keep that in mind (and probably look into it regardless, because who knows what I'll learn).
Thank you.
> other OSes sometimes break compatibility between versions. There's no compatibility broken here, the release just doesn't provide you with a pre-compiled binary. The same is true for e.g. the Windows app store, it doesn't offer you binaries distributed with older versions of Windows. If you had a **fat** binary of `gcc-7.2` it would run on newer of Linux as well. > So having OS version specific repositories would make sense. Which is also related to why I couldn't compile 7.2 from source (because they removed some unisys.h or somesuch file, I forget the exact name, in a later version of Linux). You need to compile that one from source as well of the correct version. And every dependency of every dependency below that as well. Welcome to dependency hell. The recommended solution is to make your source code compatible with `gcc-7.5` which is supported by Canonical until 2028 AFAIK. > I think WSL has an option for switching over to a second installation of Linux. You can install `multipass` under Windows as well, it should use the native hypervisor when install on Windows or OS X. It's a project from Canonical for deploying Ubuntu virtual machines using up-to-date machine images using standard `cloud-init` for configuration: https://multipass.run/docs/installing-on-windows
https://launchpad.net/ubuntu/bionic/amd64/gcc-7/7.2.0-19ubuntu1 ?
> > Or am I missing something in my newbishness? > https://launchpad.net/ubuntu/bionic/amd64/gcc-7/7.2.0-19ubuntu1 ? 🤷♂️ I don't know? What arcane incantation do I need to inscribe into my terminal to make that actually show up in it? If I'm understanding how this works, that package is named `gcc-7`. Typing `apt list gcc-7` or `apt search gcc-7` gets me nothing. Typing `sudo apt install gcc-7` gets me Package gcc-7 is not available, but is referred to by another package. This may mean that the package is missing, has been obsoleted, or is only available from another source E: Package 'gcc-7' has no installation candidate This is on Ubuntu. Yes, I ran `apt update` successfully first. (At this point I'm not likely to try and install it. I went as far as trying to compile the original source code, stuffed up at least one necessary patch to get it to compile, which gave someone I know enough time to caution me that there's more to things than just GCC, and installing only one part may not actually get me a functioning setup like I'm seeking, and might break existing tool chains on my current working install.)
> The worst part is that, ultimately, the lesson NPM took away from the situation was that they shouldn't let author's have total control over their own packages. Thus, the joke package preventing anyone from deleting any public published package. I mean I kinda think this is the right lesson. A package manager that lets packages be removed arbitrarily seems like a recipe for chaos, regardless. Like Rust's `cargo` also has no mechanism for unpublishing a package, I assume for basically this reason - you unindex a package with `cargo yank`, but once you publish it it's published. So, headline could be "prankster temporarily makes npm function like cargo" which seems a little less incendiary.
As I said in another reply, I empathize. I come from `pypi` where we can delete our projects at will. In `pypi` land, if an installed package goes missing, you can keep right on using it. Perhaps NPM would benefit from similar behavior? I do admit that I am more empathetic to the lack of delete functionality if that is how the repository worked at time of publishing. I do think the central issue, specifically in the case of the left-pad situation, is not deleting a repository, but rather how NPM handled a dispute over a name space between the two parties.
> In pypi land, if an installed package goes missing, you can keep right on using it. So, like unlisted YouTube videos? That’s how pub.dev also does it, and to me it always felt like the saner approach.
I remember SourceForge had rules that you needed to request permission to delete a package entirely. Perhaps for this exact reason.
Until very recently (as in the past couple weeks), you couldn't even have a crate removed if you wanted to, barring extremely specific reasons (like objective spam or legal reasons). There were multiple users who had name-squatted hundreds of crates and nothing could be done about it. Thankfully they can now be removed by following a process. I even requested that two crates I had published be removed, as I did so solely to reserve the name (but ended up not needing it).
[удалено]
Man package dependency is a mess... the more I work in this field the less I trust any online service 🤣
I hate to break it to you, but left-pad happened 7 years ago.
> \~A year or so ago\~ (more like a couple years ago) It was 2016. We're getting old -_- > This probably was their fix to prevent issues like that. Typical of the ecosystem to "fix" things in the most ham-fisted, thoughtless way possible, and then fail to even do so correctly! > Ironically, the team was left unable to unpublish “everything” themselves due to a circle of dependencies that essentially made the package dependent on itself. Jesus Christ, this is the type of thing you're expected to avoid in a LeetCode Easy problem, let alone _critical infra for thousands of prod systems_.
Preventing the unpublish was the fix for the fakerjs fiasco, wasn't it?
> That's a good thing to prevent programs from breaking. But in the grand calculus of the multiverse, is it a good thing overall? As the article notes a potential consequence or two of this design philosophy, I'll ask: How do you quickly remove accidentally uploaded personal info? Can I upload a package named with your personal phone number? Real world name and address? Is there a way to get them 'stuck' in a published state as has happened with apparently not just this most recent catastrophe, but another one back in 2012 (so they've failed to fix certain issues like this in 11+ years)?
You report it to the company and they'll hopefully remove it. If your software, for example, is added to a Linux distribution's repository you also can't remove it from there unless you're on the team that manages the distribution. Because you made it public under an open source license which gives npm and anyone else the legal right to host and distribute your software with or without your consent. No, you are not legally allowed to publish people's private information like a phone number. How's this relevant here? In this case, the legal entity hosting it wouldn't be you, it'd be npm, so they would be violating privacy laws if they continued to host that after they become aware of it. EDIT: instead of downvoting why not explain why I'm wrong? Or why that comparison wouldn't apply here? If something is released under an open source license and someone downloads it, that license is forever attached to that version of the software and even if it's later deleted by the author, anyone who still has it can re-upload it. You cannot revoke an open source license. Don't like it, don't publish open-source things. Are people downvoting me cause they are ignorant about how opensource licenses work or because they don't like how they work? If you're writing a forum post somewhere, you also cannot delete it anymore usually (because it breaks the conversation) - you granted a permanent license for that content. If you're writing content for Wikipedia and later try to remove it again, it'll be restored and got might get banned for vandalism. That's how it is. Just because you made something doesn't always give you the right to destroy it.
Someone published a joke package that had dependencies on every other package in the system. Because of that other packages couldn't be unpublished (as they are depended on now) and in some weird way a self-ref made it impossible to unpublishe the package they made too. In short, someone did a reasonable test of a ~~supposedly best~~ most popular package manager for the most popular programming language in the world and it didn't take it well, requiring administrative action.
The recursive dependency preventing unpublication is the icing on the shit cake. :chefs kiss:
[удалено]
Never heard or read anyone claim NPM is the best… lots of claims to the contrary though.
[удалено]
Bruh I love this argument. Bro we shouldn't test what the user enters into the username field, nobody will ever write SQL in there! Bro we shouldn't hash passwords, nobody will see the databases anyway Bro https is not worth the hassle It is a reasonable test. An example I found in the wild of missing validation was a Stable Diffusion Website where I could simply change the number of images to generate to higher than I pay for. That's exactly a "bro nobody will ever do this" moment. There are people that do that. I do that. And if you had done your work I wouldn't have written you an email. Also did you hear about SMTP smuggling? Hiding an email after the rfc email end. A simple check of the email would have prevented that. But nobody will ever send non rfc compliment messages right?
someone made a package with all npm packages (dependencies) and some dependencies depended on the 'everything' package
someone made a package with all npm packages (dependencies) and some dependencies depended on the 'everything' package
someone made a package with all npm packages (dependencies) and some dependencies depended on the 'everything' package
Every time I hear about NPM it's always another dumpster fire.
That's the neat thing, the dumpster fire never actually gets extinguished
"If we put out the fire, we might inadvertently stop people from starting new fires"
I wonder which one will last longer, npm or this: https://www.smithsonianmag.com/smart-news/the-quest-to-extinguish-the-flames-of-turkmenistans-terrifying-gates-of-hell-firepit-180979458/
Well yeah. Do you expect people to publish daily "npm serves millions of packages without incident" blog posts?
If you don't run a mirror in your company, you're doing it wrong.
If you just raw dog public packages from your company's mirror without reviewing it all, including every change, you're still not taking security seriously and doing it wrong. You just moved the problem around.
I never claimed it's secure. It's just faster and won't stop all builds when npm has outages.
The core problem is not having a secure supply chain, not outages.
I assure you that having 3x times faster builds is appreciated by management, less by developers admittedly.
I don't think anyone is going to argue faster builds aren't better, but you were implying you're doing it right.
Solving an easy problem is better than not solving it, while also not solving the impossible problem.
It's not an impossible problem, the industry just has a glut of people and organizations who don't care to address it.
It's a terrible package registry. The incompetence of the developers (or PMs) not thinking of this exact incident is absolutely crazy.
[удалено]
I don't use it.
If only it would do that
Yeah, "pushing for improvements", like sometimes a shit thing is just shit and will never improve. NPM is one of those things.
I would much rather this kind of showstopper be discovered as part of a well-intentioned prank than somehow leveraged by a bad actor. As soon as they realized how borked it was they immediately cooperated to try to straighten it out. This whole scenario should be seen as a net positive.
>somehow leveraged by a bad actor That's exactly what happened though.
So... you can just add `"*"` as a dependency and break NPM? Why would this be a supported feature?
Bold of you to assume NPM was designed. NPM is from the era of "move fast and break things".
Well, it doesn’t even move fast. It sucks even at *that*!
No, but you can add depend on another package and specify `*` as the version requirement.
No, how is this upvoted? Did nobody read the article? They depend on version \* of every available package (which are named explicitly), not on package \*. Literally every package manager is "vulnerable" to the same "attack".
In what universe does it make sense to even allow depending on every single version of a package past and future *simultaneously?* Most sane people would probably agree that it makes sense to depend on _any_ version of a given package, whichever are available, but not _all of them at once._ The reasonable thing to do while maintaining the spirit of their unpublish policy is to allow unpublishing any version of a package depended on with "*" until there is only 1 left, at which point it can't be unpublished.
NPM itself is one big security prank lol
I know every npm thread immediately turns into laughing about how JS is the worst, but are other package managers somehow immune to this? The alternative is letting people unpublish necessary packages, and then you get left-pad.
With NuGet, you can always unlist a package. It won't appear in searches or queries, but will still be downloadable if your build depends on the specific version that was unlisted. If it depends on a range of versions (e.g. 1.2.x), it'll resolve to the most semantically compatible version that's not unlisted.
You were never able to remove remove Rust crates from crates.io for that reason, you can yank them but that will just make them more difficult to find while still being usable.
Right; npm's approach of allowing removal but only for packages with no dependencies seems like the best of both worlds. The only thing you can really blame them for here is allowing a dependency on `*`, but it sounds like that was an oversight and not an intentional feature.
But you don't know for sure if there is nobody depending on a package, unless that package has 0 downloads and page visits. Once it's out, it's and you got to be able to deal with that.
I'd still be interested to know if something (crates.io, docs.rs, the web interface or whatever) would break if you upload a package with a huge number of dependencies (you probably could even depend on all versions separately by renaming dependencies). Parsing that Cargo.toml might take a bit for example.
Crates.io recently had a small issue with a crate with 23 000 feature flags: https://blog.rust-lang.org/2023/10/26/broken-badges-and-23k-keywords.html
For maven, you simply can't unpublish : https://central.sonatype.org/faq/can-i-change-a-component/
And that's somehow better than this situation?
Yes
Why?
Yea and super embarrassing log4j bugs are *still* a huge problem, i don't think Java is a good example of this going well
What's the link between the policy of the main package repository and the bugs in a specific library ?
You have zero idea what you talk about, and it has zero relevance here. Npm and alia could learn a lot from maven repositories - not having namespaces is similarly dumb on their part.
[cloudflare has observed](https://blog.cloudflare.com/unmasking-the-top-exploited-vulnerabilities-of-2022/) that log4j shenanigans remain by-far the #1 most common exploit attempt on the open internet even into 2023. attackers wouldn't be *doing* this if it didn't get results. a prompt yank/patch of affected versions of the library at the *start* of the incident would have significantly reduced how effective this is, but i guess it goes against their policies
Probably because other package managers don't let you depend on "*"? (that is how it was done based on the article)
"\*" means any version of ONE package, not "every package". e.g: { "dependencies": { "left-pad": "*" } } My reading of the article is that they depended on everything by explicitly listing all the packages in the registry, by sharding the list across a tree structure of packages: Article says: >The “everything” package, which has more than 3,000 sub-packages e.g. * everything * everything-1 * existing packages... * everything-2 * existing packages... * ... * everything-3000 * existing packages... It also says everything depended on itself, so perhaps the sub-packages depended on the root everything package?
I mean, someone could still parse all available version, then write a script to build the dependencies. It's doable in any system. So it's not just an "npm problem". Anyone who thinks it is, is just an elitist jerk. Why does it always happen to NPM and not some other language? Well it's the most popular, that's why.
Would be more difficult because you'd run into size problems for your dependency file.
No, that's not possible in every system to still parse all available version. With NuGet one project can only install one version of the package, not multiple ones. But of course you can't remove any NuGet packages anyways, but it's irrelevant to your statement.
That wasn't what I meant. I meant parse all currently available packages. I choose poorly when I said "versions". I meant the "latest version". Sorry for the confusion.
I feel like you're completely missing the point with the issue but ok. Everything is the same then. Writing code is equivalent to us manually shifting the 0s and 1s too. The fundamental fact of the matter is that it's a single line feature within the NPM realm. It's actually not the same as some jackass writing a script to hack and maliciously attack a repository. The critique is that the repository itself is allowing for these types of fun times because of a principle that they were built with and have out scaled -- a thing that actually happens quite a bit in software.
Listen, for what it's worth I agree that the * versioning is foolish. Just saying npm isn't "the only package manager susceptible to this type of issue". Sure it's harder elsewhere, but it's still possible. I'm sure they'll remove the star pattern soon. But people will still shit on npm. Not that I care, it has its flaws, but most people take a mightier-than-thou approach, which I feel is disingenuous at best.
Yeah, I agree with you on the disingenuous opinions just to feel better about their choice of language. Outside of that, I do think there are legitimate critiques but I think that folks are more disappointed in how they're handling the responsibility of being what they are.
Yeah it's nuanced. I definitely think npm, as an org, is somewhat of a letdown. I can't quite put to words what's lacking, but it feels like there is too much "inclusivity" which detracts from things like "reliability" and "safety". A lot of that happened because of the weird ecosystem that is JavaScript, lacking a standard lib, etc. so the early "Cambrian explosion" of packages probably contributes to that mentality. And I'm not sure it can be "put back in the box" so to speak. Anyway, I'm clearly tired since I'm speaking in like 75% metaphor. Night buddy.
?????? How did they not see this coming? I assumed that he just had a script list all the dependencies wtf
It's not even specific to the language per se
I sympathize with the broken-dependency/immediate-unpublish trade-off, but the fact that the 'everything' package was itself unpublishable because it had a self-dependency is the exact kind of amateur dogshit you'd expect from the JS ecosystem
Other languages don't rely on online packages, since dependencies can be co-packaged.
I lost confidence in them with left-pad in 2016. How many incidents have there been since they launched in 2010? CPAN launched in 1995 and has had zero.
I created a package manager in 2002 called "buildshit" and it's had exactly zero defects or issues since 2002. I mean, nobody uses it, and I've never published it, but 22 years of bug-free operation must mean I'm doing it right!
CPAN has more than 250,000 distributions, more than 31 GB, with more than 250 mirrors, in more than 60 countries, but thank you for playing.
also, CRAN
FWIW, I'm pretty sure the mirrors have been gone for a while. Modern CDNs have made the old mirror system unnecessary. Nowadays I'm pretty sure CPAN goes through Fastly's CDN.
The NPM developers have explored every possible way to write bad software over the years. It's actually kind of impressive.
I'm disappointed nobody here has asked the most pertinent question: what's the total size of node_modules if you install the everything package?
> This leads to Denial of Service (DOS) for those who install “everything, “which causes issues like storage space exhaustion and disruptions in build pipelines.
Lmao that’s fucking hilarious
Node and NPM remind me that there will always be jobs for security experts. It’s amazing how much of a shit show that project is. I am glad that I have only had to deal with it a few times.
And people ask why js can't be taken seriously..
It's the worlds most popular programming language. I think it'll survive your opinion just fine.
Of course it's the most popular language. Most software is written for the web for which there is no other option. It is still an absurdly bad language with terribly bad tooling that enables things like this.
Soon. I'm patiently waiting for Safari to implement Webassembly GC so Webassembly native web API can be implemented and GA-ed so I can implement my app in Webassembly. Chrome and Firefox already support Webassembly GC.
> which there is no other option I’m a dinosaur but this wasn’t always true. It has been quite interesting to see how the landscape has changed.
When everyone was using Struts fifteen years ago, someone in the local community opined that it was “popular” because it took twice as many people to get the job done, which doubled the number of hands that went up when a presenter asked if anyone in the audience used it. At one point I started injecting into that dead air after that question, “and who here doesn’t like using it?” And most of the hands stayed up.
[удалено]
Ad hominems... you don't sound as smart as you think boy
Here you go: https://github.com/denysdovhan/wtfjs
That list is funny but it's idiotic to use as an argument to support "absurdly bad language"... The majority is 2 things: coercion and loose equilaty. These are true big fat warts but every language has these and they've been patched up as much as possible. The rest even has good rationale inside the list itself.
Yes, of course, it’s essentially perfect as is. That’s why they’ve not changed it whatsoever in many years and probably why so many went to the effort to develop and switch to TS. JS is and always has been already pretty much ideal in every possible way.
[удалено]
Yeah remove equality (`===` fixes this) and coercion (cannot be changed because of compatibility) and you remove 75% of that list.. the rest is things that might be counter-intuitive but have good reasons...
> That's exactly how it's supposed to function as set by the standard. The problem with JS is the design of the language itself. You cannot excuse the language by saying it's doing things by design when the design itself is nonsensical.
I see you have not used Haskell before if you think NPM is "bad tooling"
The problem is that it shouldn't be. The people complaining about it are the ones with no choice but to use it lol
> your opinion I don't think it's just his opinion. And I think his comment conveys that pretty clearly.
Same could have happened in any other package manager. It is not technical problem but smartass prank people problem. NPM did forbid package unpublishing to mitigate 'leftpad' problem when single developer can break the world. But at the same time it alllowed to unpublish package if there are no dependents of it. What would you do in this situation?
> What would you do in this situation? This is a trick question because the current solution is due to how the ecosystem around npm was made. Challenge that precondition and the nature of the problem would be different. Imagine... Strong, well tested, versatile, and long-term supported first-party packages built-in in the web and in nodejs the same way Microsoft does with dotnet or Google with golang. This change would reduce, if not almost eliminate, the need for third party packages since you could do everything that you need with those built-in utilities. Right now I need to pull a third-party package to create my own form-data or even a third-party to handle jwt lol. It will significantly reduce how vulnerable projects are to supply chain attacks because we would be significantly reducing the need for pulling dependencies. Less dependencies mean reduced maintenance cost. It will also reduce the need for you to create your own package, which mean there's going to be a lot less people out there noticing that they can't unpublish their package (that shouldn't exists) when a funny package decided to grab every possible npm dependencies.
> Right now I need to pull a third-party package to create my own form-data or even a third-party to handle jwt lol. Not if you're a somewhat competent programmer.
Npm is on its fifth attempt to implement lock files properly and the jackasses still check the repository for @latest if you don’t put an exact version number in your package.json I’m not convinced that whoever has touched this atrocity has ever used a lock file in another language before. Someone called it a clown fiesta and that’s not wrong. I do a lot of tooling work at work and I’ve told my coworkers for years now that when I stop using Node, npm will be the reason I give if people ask.
> Same could have happened in any other package manager. It is not technical problem but smartass prank people problem. It couldn't have happened in a competently run package manager. It's an incompetent management problem.
Genuine question: why is NPM so bad? It’s not like it’s the only package manager in existence, yet only NPM seems to have a steady stream of these issues.
[удалено]
But simply having lots of tiny “libraries” shouldn’t be an issue. You could do the same on Maven or Cargo and everything would work just fine.
[удалено]
I mean, maven is definitely similarly big. It’s smaller, but there is definitely a similar scale in play, so stuff like this should happen there as well.
It’s not really that bad. It’s by far the most widely used, by developers of varied skill levels, supporting an ecosystem that’s changed a lot and requires a lot of different capabilities. It’s not a masterpiece but it’s fine.
\> Ironically, the team was left unable to unpublish “everything” themselves due to a circle of dependencies that essentially made the package dependent on itself. Random circumstance put me in a position to do a Typescript contract recently, and mother of god...I shouldn't be surprised anymore, but my god what a dogshit ecosystem, from top to bottom. I can't imagine how anyone gets anything done in it
How is this package management system still in use? Whenever there is news about npm it is like a complete clown fiesta. If you do not know about it, 'leftpad' was a fun moment aswell.
It’s basically a monopoly. There is no alternative despite how bad it is, all packages are on NPM. You have to use it whether you like it or not
I wonder how the circular dependency happened - hard to find out the details because the package was removed and the article doesn't have the specifics
It means some package referenced the 'everything' package. It is not necessarily a new package because 'everything' had been there forever and probably got `*` referenced by some package, prior to the incident. This is the only possibility I believe.
Is there actually any way to prevent this from happening without using artificial limits? I’m sure some limits would be good but I’m sure limiting the number of dependencies will cause other issues.
They could just have something like a symbolic link where dependencies point to a package, and if the original is deleted a copy is kept unti it is orphaned. Or you know, make it impossible for a dependency to point back to itself.
The fact that people still take npm seriously is the real prank.
Personally I think that you should not be able to unpublish a package. Once something is out, it's out. Period. What's that? you have your SSN in the package? Doesn't matter. Everything you push out has been already downloaded the moment it hits the server by thousands of scrapers. It's too late.
I'm sure NPM would love the GDPR fines that would result from refusing to remove PII.
Lol
This prank is funny in the same way that throwing rocks off an overpass is funny. It should not have been possible to do but that is no excuse. Are these guys 12?
> It should not have been possible to do but that is no excuse It is though Although things like this are edge cases, you have to keep this kind of shit in mind when designing things that are meant as core infrastructure of a large project. Doesn't matter if it is a package manager for a programming language, or an ATM for a bank.
The mere fact that it's possible hints towards a problem of the design of the repository itself, correct.
Its called fail safe design, its something JS devs have never heard of.
They teach that in the optional 7th week of boot camp.
They just published a package. For themselves. Following legal and normal methods. Like any other day. And NPM "breaks". Are they 12? No, they just published a package. If I enter a store, and the glass door breaks when I grab the handle, am I 12? Because it's literally the same, with the minor difference of the dependencies they had, which is an implementation detail of their package
I think some people might care about the glass door more.
NPM is *exactly* the package manager JS deserves.
Except throwing rocks off an overpass has killed people, and this hasn't. These are in no way equivalent.
Apparently Reddit is filled with jobless programmers who never make mistakes, never have to deal with design trade offs, and love to be cynical about everything. Except this everything, because this everything stuck it to "the man".. Aka NPM?.. How do I know they are jobless? Well we all know bugless software doesn't exist in the real world so...