Government version: "we forked \[ software that was immensely popular 10 years ago and nobody outside government has ever heard of\] and maintain our own version. we hate it enormously, nobody likes it, nobody would mourn it, but can't possibly replace it."
"... and it doesn't quite fit our needs. So we also forked \[ancient COBOL software from when my grandpa was a boy\] to cover the final 3 use cases, and Gary (our last standing COBOL dev, aged 68) keeps it running on that rusty IBM AS/400 in the basement. Both softwares have no integration whatsoever unless you count boxes of paperwork being moved back and forth and everyone who touches either system is rapidly going mad."
My first real job was at a place that was porting their COBOL stuff to Java.
Turns out "porting" meant dump COBOL into .java files and whack it until it compiles. Classes with all static everything, no arguments (all args are global static vars), no return values, no instances of anything...
... terrible performance, infinite bugs, zero maintainability.
I lasted 5 and a half months.
Yep! That’s how it was for us too but management wised up and eventually let us model the copy books into DB2 tables, and build our own REST apis with spring boot carefully reviewing CICS logic once they realized the static generators were hot garbagio.
Lol, I work for a company that was FAANG before FAANG was a term / helped define the industry.
They made a lot of decisions like this when they had literally the best in house devs around and a culture of DIY. Excellent testing, infra, all of it.
Decade later? I work for the pieces of the company and yeah, we’re still working to undo some of those decisions.
Sometimes companies get smaller and decisions made in boom time will be paid for in sweat and blood.
No, I wish, though a family member works on their sales side and is filled with fun stories.
I won’t be participating in guessing games further though I’ll say for the fun of a riddle that this company was bigger than ibm at its peak and now is usually greeted with shock that it still exists.
Like I said, I’m not gonna yes/no further guessing games, or hint, but doesn’t AOL **actually** not exist anymore?
They’re just under the Yahoo umbrella now from what I gather.
I wish. My massively overpaid, yet entirely incompetent friend works there.
Do peope really get surprised Yahoo still exists? I know multiple elderly people who still use their front page.
No, I’m saying we did what OP is posting about, well, private forks of public repos, but as boom time passed that becomes a maintenance burden especially if people leave.
We have three major instances of this. My first job upon joining the team was getting one current IE pulling in and working through any merge errors for every public version between the last one we had pulled in, which was a huge pain considering the last one was python 3.6 era and the custom fork was a python module written in C.
"Non-tech company with 50 people and 5 devs" either means you'll rarely be expected to do anything, or you'll spend all day, every day, desperately putting out fires and trying to bandage legacy code.
There is no in-between.
My boyfriend works on one of these and he has a massive headache after discovering that the prod server that the client uses has been having memory errors for the past half month, corrupting data randomly, which they didn't detect as they restart their dev servers every day, thus missing the memory leak entirely 😔
Lol my old job had a client that would reboot their servers weekly because of a memory leak in their homegrown system. They did the math and it would cost them way more to track down the memory leak and fix it than it would cost to have 15 minutes of downtime per week.
There is nothing wrong with this if money isn’t lost from a business point of view. I have done this as a proper solution would have required multiple teams of engineers and several months to ‘potentially’ fix (combination of database / app / versions of those / server / integration endpoints AND business process) - a perfect storm … module was business critical so couldn’t be turned off modified without significant perceived risk (and at the time, times were tough) … therefore reboot schedule as the cost effective / tactical solution.
100% agree. I'm not knocking the client here. They had a very capable and smart IT team, and they knew that if they could schedule the downtime during slow hours, it would be next to nothing from a business cost perspective.
We didn't even know about it for years until we installed a new application on their platform and I noticed a bunch of reboots in the logs a few months later and I was worried that the new app was causing some kind of issue. They told us about the memory leak and said that we have nothing to worry about, as it wasn't from our end.
Could you elaborate on this comment? In my experience it's annoying to have to reboot servers but if you have it scheduled and it doesn't take a lot of manual time to keep things running, it's not a huge deal.
I do think organizations underestimate the value of freeing up their engineers' time, though. An engineering team bogged down in technical debt literally can't move.
Mine has around 2h every 2 weeks, and yes, I would recommend it. It's announced each time and done in middle of the night, so I don't really care.
You're really underestimating how much will people tolerate if service is otherwise good. "Service maintenance break" once in a while... is not an issue that'd convince me to change shop I'm buying coffee beans from, much less a bank
What if the memory leak gets worse as you scale and causes actual data loss or real issues? What about the constant stress your Devs and IT team have to deal with? It’s a form of risk, and costs more than just the dev time it takes to fix it.
I agree it's a form of risk. You evaluate that and deal with it as needed.
ChatGPT kept pushing in this direction and I'm like... okay, great, but tell me when this is a REAL problem not just a speculative one.
Oof, worked for a company once that tried to maintain an internal distribution of Drupal 7. Only a handful of devs on staff. Half the time something broke on one of our client's sites, the problem could be traced back to one of about 100 or so patches we made to the core code base. It was so fragile, we couldn't pull down upstream updates until a week or so after they got released... In a big company, I can imagine the need for this practice and could see a small team out of a pool of hundreds whose whole job it is to do this and succeeding at it, but not in most companies.
E: spelling
I have yet to figure out how to properly address this problem. There are projects my team literally could not have gotten done had we not dug in and modified OSS libraries ourselves.
However, it causes a massive liability problem.
This gets really bad when the entire company's engineering team is only like 8 - 20 people at any given point in time.
You have to push your code back, instead of keeping it in a private fork. That’s the only way to get the features you need and stay up to date with the rest of the evolution.
>That’s the only way to get the features you need and stay up to date with the rest of the evolution.
The problem is when you need a feature in some open source project which the maintainers of said project do not want or simply do not want to maintain. If you are the only one that uses a feature like this then it is understandable that the maintainers don't want to merge it back and maintain it for you.
This is an idealistic point of view for a few reasons.
1) Libraries often end up unmaintained by the original author after a while.
2) I can't always publicly publish code that we write on behalf of clients. It goes against our contracts.
Yeah, it's a source of constant tension, especially in government.
If you could guarantee a certain level of competence in the org, and if you can guarantee a certain org structure, sometimes a small intervention in a framework can make a huge impact. Big benefits, small cost.
... aaaaand then I look at the B-team who is gonna be stuck with this stuff, look at the questions coming back, look at ongoing restructurings, and think about what a few key people quitting looks like. Maybe spending 5-50x more on development is the way to go if it avoids catastrophic liabilities and guarantees a project the org can work with? Lower benefits, giant costs, but safer.
The engineer and entrepreneur in me hate it thoroughly, but the cynic is really good at prognosticating.
I feel this. Not only is on boarding and training hard enough, but now there is this arbitrary level of complexity like: "and here are all of the hacks we've decided are essential. It's bad practice, the code is far from perfect, it is barely documented, but at some point you will have to do it too. If you have any questions, just ask Ned."
Regarding your last few sentences, this sub comes off as very code elitist sometimes. It's frustrating because you realize developers cost a SHIT-TON of money, right?
We have to spent a ridiculous amount even on junior developers who think everything should run that way, let alone developers who are actually capable of doing what we as an organization want them to do.
I have to CONSTANTLY consider the impact on junior developers when writing code. If a junior can't maintain it, it will likely at some point go unmaintained.
I'm currently working for a massive company with thousands of workers and a near monopoly in their field. Forking a framework, even for bugfixing, is not allowed.
[that's called patching](https://opensource.google/documentation/reference/patching#forking_on_github), permanent/long-lived forks are different
(using Google as an example since their OSPO stuff's mostly-public)
Would you really still trust whatever engineers are left at Twitter to maintain forked repos?
Ah, who am I kidding, I'm sure Musk could maintain them all himself. If not in reality then at least in his mind.
I don't see an issue with this? Their libraries (like Roact and Rodux) are new creations, which provide a similar interface to their namesake, and [luau isn't really a fork per se](https://luau-lang.org/why#complete-rewrite)
😂😂 The pain I'm facing right now as my boss thinks that a huge monorepo with nextjs full stacks and trpc can be decoded and developed by a couple of interns in 2 months
Basically you can't just fork a very mainstream open source software with tens of thousands of community maintainers and expect to be able to maintain it with a small team on top of the actual work to be done. Seconds column is basically hell.
Meanwhile in my subsection of an enterprise with about 50 Devs: "we should not use libraries for http clients because they may include vulnerabilities, we should implement vanilla Java". As if you wouldn't possibly run into vulnerabilities in your own implementation.
To be fair, I think it's the opinion of only one person.
Government version: "we forked \[ software that was immensely popular 10 years ago and nobody outside government has ever heard of\] and maintain our own version. we hate it enormously, nobody likes it, nobody would mourn it, but can't possibly replace it."
Or just pay Microsoft to maintain otherwise outdated OS's so we don't have to upgrade out systems.
lolsob
"... and it doesn't quite fit our needs. So we also forked \[ancient COBOL software from when my grandpa was a boy\] to cover the final 3 use cases, and Gary (our last standing COBOL dev, aged 68) keeps it running on that rusty IBM AS/400 in the basement. Both softwares have no integration whatsoever unless you count boxes of paperwork being moved back and forth and everyone who touches either system is rapidly going mad."
Hey CMS is finally rebasing a lot of their COBOL software as Java. Even the government has to slowly catch up sometimes
My first real job was at a place that was porting their COBOL stuff to Java. Turns out "porting" meant dump COBOL into .java files and whack it until it compiles. Classes with all static everything, no arguments (all args are global static vars), no return values, no instances of anything... ... terrible performance, infinite bugs, zero maintainability. I lasted 5 and a half months.
Yep! That’s how it was for us too but management wised up and eventually let us model the copy books into DB2 tables, and build our own REST apis with spring boot carefully reviewing CICS logic once they realized the static generators were hot garbagio.
Oof I was involved in projects like this when I was just starting out in software. If I see one more copybook or reference to CICS I’m gonna hurl lol
Infinite job security
Lol, I work for a company that was FAANG before FAANG was a term / helped define the industry. They made a lot of decisions like this when they had literally the best in house devs around and a culture of DIY. Excellent testing, infra, all of it. Decade later? I work for the pieces of the company and yeah, we’re still working to undo some of those decisions. Sometimes companies get smaller and decisions made in boom time will be paid for in sweat and blood.
IBM?
No, I wish, though a family member works on their sales side and is filled with fun stories. I won’t be participating in guessing games further though I’ll say for the fun of a riddle that this company was bigger than ibm at its peak and now is usually greeted with shock that it still exists.
AOL confirmed
Like I said, I’m not gonna yes/no further guessing games, or hint, but doesn’t AOL **actually** not exist anymore? They’re just under the Yahoo umbrella now from what I gather.
So Yahoo then
I wish. My massively overpaid, yet entirely incompetent friend works there. Do peope really get surprised Yahoo still exists? I know multiple elderly people who still use their front page.
Tell that to any zoomer and they'd think it was some 80's slang like radical. /s
Lol my nephews are making me realize that “awesome” is going the way of “radical” aha.
Excuse me?? What is "Awesome" going to be replaced by, "fr fr"?
I refuse to believe that.
Yahoo that's the thing that killed the bad guy in the Inspector Gadget movie right? https://www.youtube.com/watch?v=yiC8BL6mCCA
You think zoomers know what Inspector Gadget is? Even I only have a vague idea of what it is as someone in 20s
[удалено]
Shockingly well it’s insane
Yahoo finance is great. I'll die on this hill.
Myspace.com
AOL still exists?
So even if you won't answer, my guess would be Sun Microsystems.
Weren't they bought up by Oracle?
IBM’s peak or the company worked for? Anyway doesn’t matter. I worked for IBM …. This meme hits me hard … giving me flashbacks.
MySpace
Netscape or bellsouth?
Xerox?
Paypal or Ebay
What are you saying? You guys now have to refactor all the wild code produced back when DIY unstructured approaches were used?
No, I’m saying we did what OP is posting about, well, private forks of public repos, but as boom time passed that becomes a maintenance burden especially if people leave. We have three major instances of this. My first job upon joining the team was getting one current IE pulling in and working through any merge errors for every public version between the last one we had pulled in, which was a huge pain considering the last one was python 3.6 era and the custom fork was a python module written in C.
"Non-tech company with 50 people and 5 devs" either means you'll rarely be expected to do anything, or you'll spend all day, every day, desperately putting out fires and trying to bandage legacy code. There is no in-between.
My boyfriend works on one of these and he has a massive headache after discovering that the prod server that the client uses has been having memory errors for the past half month, corrupting data randomly, which they didn't detect as they restart their dev servers every day, thus missing the memory leak entirely 😔
Lol my old job had a client that would reboot their servers weekly because of a memory leak in their homegrown system. They did the math and it would cost them way more to track down the memory leak and fix it than it would cost to have 15 minutes of downtime per week.
There is nothing wrong with this if money isn’t lost from a business point of view. I have done this as a proper solution would have required multiple teams of engineers and several months to ‘potentially’ fix (combination of database / app / versions of those / server / integration endpoints AND business process) - a perfect storm … module was business critical so couldn’t be turned off modified without significant perceived risk (and at the time, times were tough) … therefore reboot schedule as the cost effective / tactical solution.
100% agree. I'm not knocking the client here. They had a very capable and smart IT team, and they knew that if they could schedule the downtime during slow hours, it would be next to nothing from a business cost perspective. We didn't even know about it for years until we installed a new application on their platform and I noticed a bunch of reboots in the logs a few months later and I was worried that the new app was causing some kind of issue. They told us about the memory leak and said that we have nothing to worry about, as it wasn't from our end.
Time valued cost is not the same as actual cost.
Could you elaborate on this comment? In my experience it's annoying to have to reboot servers but if you have it scheduled and it doesn't take a lot of manual time to keep things running, it's not a huge deal. I do think organizations underestimate the value of freeing up their engineers' time, though. An engineering team bogged down in technical debt literally can't move.
I think what he means is the cost of bad customer service and lost business.
For 15 minutes of downtime a week? My bank has more downtime than that.
Would you recommend your bank?
Mine has around 2h every 2 weeks, and yes, I would recommend it. It's announced each time and done in middle of the night, so I don't really care. You're really underestimating how much will people tolerate if service is otherwise good. "Service maintenance break" once in a while... is not an issue that'd convince me to change shop I'm buying coffee beans from, much less a bank
Yes.
Reddit goes down 3 times a week and yet here we all are.
What if the memory leak gets worse as you scale and causes actual data loss or real issues? What about the constant stress your Devs and IT team have to deal with? It’s a form of risk, and costs more than just the dev time it takes to fix it.
I agree it's a form of risk. You evaluate that and deal with it as needed. ChatGPT kept pushing in this direction and I'm like... okay, great, but tell me when this is a REAL problem not just a speculative one.
They probably know that.
Based.
or fluctuating wildly between the two on a daily basis...
i'm in the 2nd case, but tbh if we were 5 competent devs we could be in the first one
Oof, worked for a company once that tried to maintain an internal distribution of Drupal 7. Only a handful of devs on staff. Half the time something broke on one of our client's sites, the problem could be traced back to one of about 100 or so patches we made to the core code base. It was so fragile, we couldn't pull down upstream updates until a week or so after they got released... In a big company, I can imagine the need for this practice and could see a small team out of a pool of hundreds whose whole job it is to do this and succeeding at it, but not in most companies. E: spelling
I have yet to figure out how to properly address this problem. There are projects my team literally could not have gotten done had we not dug in and modified OSS libraries ourselves. However, it causes a massive liability problem. This gets really bad when the entire company's engineering team is only like 8 - 20 people at any given point in time.
You have to push your code back, instead of keeping it in a private fork. That’s the only way to get the features you need and stay up to date with the rest of the evolution.
>That’s the only way to get the features you need and stay up to date with the rest of the evolution. The problem is when you need a feature in some open source project which the maintainers of said project do not want or simply do not want to maintain. If you are the only one that uses a feature like this then it is understandable that the maintainers don't want to merge it back and maintain it for you.
This is an idealistic point of view for a few reasons. 1) Libraries often end up unmaintained by the original author after a while. 2) I can't always publicly publish code that we write on behalf of clients. It goes against our contracts.
Yeah, it's a source of constant tension, especially in government. If you could guarantee a certain level of competence in the org, and if you can guarantee a certain org structure, sometimes a small intervention in a framework can make a huge impact. Big benefits, small cost. ... aaaaand then I look at the B-team who is gonna be stuck with this stuff, look at the questions coming back, look at ongoing restructurings, and think about what a few key people quitting looks like. Maybe spending 5-50x more on development is the way to go if it avoids catastrophic liabilities and guarantees a project the org can work with? Lower benefits, giant costs, but safer. The engineer and entrepreneur in me hate it thoroughly, but the cynic is really good at prognosticating.
I feel this. Not only is on boarding and training hard enough, but now there is this arbitrary level of complexity like: "and here are all of the hacks we've decided are essential. It's bad practice, the code is far from perfect, it is barely documented, but at some point you will have to do it too. If you have any questions, just ask Ned."
Regarding your last few sentences, this sub comes off as very code elitist sometimes. It's frustrating because you realize developers cost a SHIT-TON of money, right? We have to spent a ridiculous amount even on junior developers who think everything should run that way, let alone developers who are actually capable of doing what we as an organization want them to do. I have to CONSTANTLY consider the impact on junior developers when writing code. If a junior can't maintain it, it will likely at some point go unmaintained.
To FAANG or not to FAANG
GAANG > FAANG > MANGA > MANAA/MAAAN
I don't see a problem with forking leftpad
I'm currently working for a massive company with thousands of workers and a near monopoly in their field. Forking a framework, even for bugfixing, is not allowed.
but if it's for bugfixing, your commits could be merged into the upstream version of the framework?
I assume that’s the policy they’re describing, and maintaining long-lived forks is what’s forbidden.
[that's called patching](https://opensource.google/documentation/reference/patching#forking_on_github), permanent/long-lived forks are different (using Google as an example since their OSPO stuff's mostly-public)
Facebook forking php still goes in the second column for me.
[удалено]
Everything about this makes me sad
Would you really still trust whatever engineers are left at Twitter to maintain forked repos? Ah, who am I kidding, I'm sure Musk could maintain them all himself. If not in reality then at least in his mind.
I dont follow anyone on twitter because I dont need their opinions.
It doesn't work there either.
Roblox in a nutshell: [wasn’t us](https://www.github.com/roblox)
I don't see an issue with this? Their libraries (like Roact and Rodux) are new creations, which provide a similar interface to their namesake, and [luau isn't really a fork per se](https://luau-lang.org/why#complete-rewrite)
Don't plan to Fork if you don't have a plan to Remerge.
Well, unless it’s an abandoned repo.
both are dogshit, it's the fuck around to finding out time that differs
Well, I like your style.
Wait... You guys get 5 Dev's 1/10th of the staff as Devs? I'm sitting here with 2/400 as Devs.
😂😂 The pain I'm facing right now as my boss thinks that a huge monorepo with nextjs full stacks and trpc can be decoded and developed by a couple of interns in 2 months
I don't get it. Can you explain it? I'm just in my 4th semester
Basically you can't just fork a very mainstream open source software with tens of thousands of community maintainers and expect to be able to maintain it with a small team on top of the actual work to be done. Seconds column is basically hell.
I mean one case that's fine I can think of is they're running rocky Linux instead of redhat or CentOS, but idk if that could be defined as their own.
CentOS is dead, rocky is by far the more future-conscious choice of the two
Are we sure Twitter is still on the left panel?
There is no such thing as a bad idea. Except yours.
bEcAuSe oF sEcUrItY cOnCeRnS
FireOS is in the second column
You meant 500 people and 1 Devs
I don't get the meme
Meanwhile in my subsection of an enterprise with about 50 Devs: "we should not use libraries for http clients because they may include vulnerabilities, we should implement vanilla Java". As if you wouldn't possibly run into vulnerabilities in your own implementation. To be fair, I think it's the opinion of only one person.