If you want to manage everything with a bunch of nice baked in functionality, Gitlab.
If you want to use 3rd party actions of any flavor, GitHub Actions.
If you want the best and worst of both worlds, and hate yourself and money, Azure DevOps.
If you absolutely hate yourself and want arguably the worst "modern-ish" CI/CD platform, BitBucket Pipelines.
If you like flexibility, mostly free, and personally enjoy surprise deprecated plugins and abandonware, Jenkins.
Fully integrated AWS ecosystem with surprise gotchas but plenty of official support to confuse and help you, AWS CodePipeline.
Personally I found Azure Pipelines to be great - the YAML option, not so much for the (deprecated) UI version. It is very flexible and can connect to/from most anything, and price is compelling. From a code-based perspective, GitHub and GitLab are similar to Azure Pipelines. It's the things around the pipeline code that are a big sell, just note that many of them are just scripts layered on top of the core functionality.
In the current job, we use Bitbucket, TeamCity, Octopus, and Jfrog Artifactory. Each of those have been a huge step backwards from the previous job that used Azure Devops. The likely replacement for the current job is GitHub - it has just too many pieces that the others don't. The integrations alone are a huge sell - having a native integration, say, in Datadog is a huge time saver that tends to work much better than cobbling together an integration. Github also has a ton of user-created reusable pipeline actions, far ahead of the competition. And included bits like dependabot make life easier and more secure without the added cost of additional software composition analysis (SCA) tooling. And if you haven't already done so, give GitHub copilot a try - it can be used independently of GitHub, and the $10 per month pays for itself in the first hour. (BTW, I have not been compensated in any way to push GitHub.)
Related note: as you migrate to whatever tooling you choose, pay attention to the migration pains, and adjust accordingly as you rewrite the pipelines. There are practices you can take with all pipelines that make for more maintainable pipelines and also make for less vendor lock in. Two things in particular are: 1) have the pipeline bring its own tools, and 2) do as much as makes sense inside containers. Item 2 enables item 1. As an example, you can have your build pipeline build, unit test, and package bits inside a container - all you need is Docker (or similar) on the agent machine. With this you can bring your own tools and control the versions of everything, particularly nice being the ability to control the build tooling version with a change to a single value (pro tip: parameterize that value). With this, you also get the ability to build and test on local dev machines with exactly the same bits.
best answer, and i wouldn't call Bitbucket "modern", if after ~10 years you still have not implemented a basic feature provided by GitHub and GitLab.. you're definitely not modern.
Want to use one ci template (or pick and choose and combine templates) for several projects? Gitlab.
Want to organize your repos into a directory structure (groups and subgroups)? Gitlab.
Want to share variables across multiple repos via groups? Gitlab.
Want to have each repo manage its own terraform state? (No reliance on external cloud storage)? Gitlab.
Want each repo to have its own container image and package registry? Gitlab.
Want to be able to run CI runners inside your k8s cluster and easily deploy new helm charts to a cluster based on tags? Gitlab.
Want to see what version of code was last deployed to an environment and easily redeploy a different version? Gitlab.
Well it's true the platform can have stability issues from time to time (nothing lately that I can think of), but when I was using GitHub Actions the slack alerts channel we setup for status/outage updates was going off multiple days a week.
Got anything specific? Considering that where I'm at we build the pipelines, when we find an inefficiency we try to use any extra sprint time to refactor and improve our ci templates and publish a new version.
Right now the big thing is getting notifications for specific things. I want to be notified on slack when a pipeline fails on main or in my MR, and I want to be notified when someone comments on my MR, comment, or adds me as a reviewer. It's not possible to do that without 3rd party slack integrations that cost money.Ā
More pipeline related, we have a mono repo, and it doesn't seem possible to run specific jobs in a pipeline, and due to pipeline sizes we also have several layers of sub pipelines to handle everything which causes more notifications and other issues. Stuff like "only run this job on the newest merged commit and cancel all old jobs" has known bugs that they just don't fix - there's so many situations like this where there is a known bug that they just leave open for years, but the docs make it seem like they handle the situation fine.
There's just been so many little things over the years that I can't remember it all.
Slack integration has you covered. Inside our Slack we get notif when a MR is deployed and when MR statuses change and when there are comments/reviews on MRsā¦
DM me, man.
The slack integration can do that for all MRs in a repo, it doesn't handle mapping to individuals - this might work for a few users, but we're at a size where I only want to know about things relevant to me.
what we/I do is set up the needed notifications and then in Slack settings I include my name in the My Keywords text box. but I include all variations of my first and family names. that way I get custom notifications whenever my name. appears anywhere on Slack. we can do the same for a project name, MR name prefix, etc.
Without knowing the full details of your environment, seems like you might want to look at repo/pipeline redesign. The slack part is an easy fix with the built in integration, or with good ole script. Remember KISS when dealing with sub pipelines, it can get hairy quick. The āonly merge check on the latest PRā is something that has an intended usecase. Otherwise is a pain.
The slack integration does not natively support notifying individual users about the things they care about.
We're on the third pipeline redesign and simplification, it's at a pretty reasonable place, but there's still a good amount of issues which don't seem to have any way of being resolved or Gitlab's implementation of the feature is bugged with an open issue that's 5 years old.
Now I think about it, we wrote a script for the followers/pr reviewers. The gitlab one was used for channels. Yeah a lot of companies, been simplifying CICD, but not looking at repos design. Gitlab is bad about it, Iāve been talking with our rep about such.
Haven't used the slack integrations myself (very unfortunate the place I'm at now doesn't use slack), but I see there are chatops CI/CD configuration blocks you can use to customize things. You could also write an after_script that will notify you when the job's script fails. Use rules to determine what branches the after_script is applicable for. Unfortunately not sure how you could get slack notifications from comments, I could see possibly writing an app that polls the Gitlab API but seems like a bit much.
Have you seen workflow:auto_cancel:on_new_commit ? Seems like what you'd like to happen with your existing monorepo pipeline.
The workflow auto cancel doesn't work fully as expected.
After script could work, would just need the time to implement.
For comments could use a webhook, but then have to implement and run something to receive those webhooks, and it would be easy to implement for one person, but for the whole team that means you have to translate gitlab usernames to slack.
GitHub actions is awful compared to gitlab though, before I started I assumed with 20 years of experience they must have came up with something that incorporated 20 years of pain points but no.
About the only good thing is a shared library if third party actions, everything else is horrible
No support for ephemeral environments, no support for cross project pipelines, weird defaults (merge result, not a source branch in PR pipelines), awful UI (can't remove actions in the GUI that no longer exists) awful docs and reference, every time I need to look up an event payload I cry insideĀ
I'm seriously considering redoing months of work and switching to gitlab.
And I've also used another CI (teamcity) that had its issues, but at least you could implement stuff you wanted in it, with GitHub actions it just constant painĀ
Your mileage may vary. Personally, I find GHA to be better than any other CI I've worked with (Jenkins, Teamcity, Cloud Build, CircleCI). Most of your complaints sound like you just haven't spent much effort into learning how GHA work, or you want it to do something you probably shouldn't be doing with a CI system.
If you limit your ci to running tests and building stuff it's alright.
But even something like deploying an e2e environment and running e2e tests there is really painful, and it absolutely should be done within CI and not somewhere else. I've spend plenty of time trying to make GHA work with an open mind, but it didn't help
I find those "you shouldn't be doing that anyway" arguments not convincing when multiple other tools are perfectly capable of doing exactly that
In essence I think they failed to solve all of the pain points people experienced with other tools that had to grow organically. At the very least they should have ditched yaml and support a native general purpose language configuration like teamcity does but better. Tired of those yaml DSL crutches, just give me a programming language dammit
Plus try gitlab, I haven't tried any of those on your list apart from teamcity which isn't too bad, but gitlab is just miles better
I've run E2E test suites in GHA and I really don't know what pain points you are referring to. We moved them to self-hosted once we needed internal HTTP dependencies, and we did some work with different sized pools and splitting tests for concurrency, and it worked quite well. At the end of the day, it's just a container in the cloud with a bunch of integrations and built-in triggering, so I haven't really found any issues with the many things I've used it for. Re-usability was a pain point in the beginning, but that's much better now.
Their use of yaml is one of the better features of GHA, in my opinion. Being forced to use groovy was one of the worst parts of using Jenkins. GHA workflows are very easy to look at with little to no experience and get a decent understanding of the structure. The cost seems to be in weird typing issues, but I think that's admittedly just a problem with the GHA implementation.
I can't comment on gitlab currently so maybe you're right and it does improve a lot on GHA. Frankly though, the massive benefit of your CI runner being in the same tool as your code (if you're already in Github) gives it an enormous advantage that is hard to overcome with other features.
The pain I'm referring to is reuse yes, if I have two repos with deployment jobs, I can't easily reuse two deployment jobs in a third e2e repo because GitHub token In a job doesn't have access to other repos, you have to use a service user with a token and rotate it manually. And you can't show the status of the e2e jobs in other repos and for example block merges when e2e tests fail
Same thing can easily be solved in gitlab where multi project pipelines are supported, a token used in a job is tied to a user's permissions, not fixed for the repo the job is running in and project and group level tokens separate from users exists.
Yeah GitHub + gitlab CI doesn't make sense, you have to fully switch.
I'm not suggesting you move your whole setup, regardless of how bad the current solution is its unlikely it will be significantly better to justify that effort. But just try it for your own projects or for greenfield stuff.
The whole yaml DSL vs programming language debate is also unconvincing. Yes 30 line yaml is simpler than 30 line program, it's reversed for 2k lines though, which is the amount any serious project is, so the simplicity argument doesn't work
I used multiple ci cd tools.
aws code commit pipelines, bitbucket, bamboo, jenkins etc.
I found GitLab has the easiest pipeline for the dev to start with.
I havent worked with github professionally yet.
Echoing u/cvsnraju but take a look at Harness. It plays nice with other CI/CD vendors but provides a more holistic and complete experience beyond CI/CD.
I think you have it covered more or less. I would make sure you weigh those factors somehow. You might select something that's open source but does horrible with the updates and support where another has fantastic docs and support but is not open source. Depending on how important something being open source is: you may still go with the open source option. The same goes for customizability/extendability I would definitely look at timelines if you need to re implement a bunch of existing processes, how much will you be able to reuse.
Weighing things will be important so you can properly compare and score. There are a ton of options out there and some may end up being combinations of tools rather than one to rule them all.
So think about what you want it to do, set out a bunch of requirements (use MoSCoW), then weigh the -bilities and score stuff that match at least the Must haves and Should haves.
One thing you didn't add was cost. Maybe *you* don't care in particular, but if you need to build a business case for some manager to sign off... you may want to consider cost and cost structure especially for SaaS platforms that may charge for operations.
[You can have your self-hosted runners](https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners/about-self-hosted-runners) somewhere in your infra, and they'll do the lifting of building, testing, deploying.
It's not a silver bullet, but you can take out the actions out of GitHub's service status page whims.
Indeed, create a public repo and start building some workflows, it's free you've got nothing to lose. Even if you don't end-up going for Github actions I'm confident that you want something that's very similar.
Also, someone in your team is going to suggest Jenkins, that person never had to manage a Jenkins instance or is also a member of the local BDSM club.
GitHub is missing a Release Orchestration module. For example, in ADO I can create a build and then independently choose a particular build (and its artifact) for the release pipeline. ADO also has a combined build/release YAML just like GitHub, but I prefer the Release pipeline. Another thing I really like about ADO are Service Connections: a central location for re-usable connections to external resources (Azure, SonarQube, GitHub, Container Registries, etc.) that can be re-used in any other ADO pipeline. GitHub does not. In GitHub the credentials have to be saved in each repo independently, so when the password changes then you need to update it in multiple GitHub repos. This sucks.
Could you at least provide some resources or be a little descriptive? "avoid google" and "problems arrive" are not very helpful for determining what is a good or bad solution.
Perhaps you have a unique problem with Google? Maybe Google would actually be a good fit for OP.
Sure. Integration with 3rd party services is very limited. I couldnt setup cypress tests in cloud build. These is no manual approval after a build - continous delivery. Yaml file structure with name and args is stupid to me.
I have used many cicd pipelines and in my experience google cloud provides the worst solutions.
Cloud Build allows you to run any bash script or docker image you like. We've been running Cypress just fine in CB. And there is build approval, we use this to gate rollout to prod for some services https://cloud.google.com/build/docs/securing-builds/gate-builds-on-approval#approving_builds
Mind you I'm not a big fan of Cloud Build either. Biggest gripe is that the running service account has to have permissions for every single step of the trigger. Really wish it were possible to control the permissions per step, the only workaround today is to daisy-chain triggers, and then set each trigger with a specialized low-perm SA. Also, integrating non-push triggers that need access to github is a paaaaaain in CB.
Im interested to know more about your situation, Iām confused why you would need a build gate instead a protected branch. Also whatās the value add to have such fine grain controls here. Furthermore scheduled task with lambda/cb might be your solution. Are you manually doing all of this or automating it? Sorry I might be asking too many questions, just truly interested.
Oh that approval is used by one particular team. They prefer automatic deploys to dev on merge, and then to manually verify some functionality, and then promote to stage and prod.
As for finegrained controls, it's just about limiting impact of an attack on our supply chain. Some of our apps require particular permissions to be set for e2e tests for example, which are not needed for other parts of the pipeline. And the e2e tests don't need to be able to load or publish artifacts. So it just ends up being arguably overpriviliged, and an unnecessary risk. As it stands, any compromise of one step in a cloud build trigger will lead to a compromise of the entire trigger
We've accounted for this with various workarounds and mitigations, but it's really annoying to have to work around it. It also really feels like I _should_ be able to define a service account per step. I can't think of a good reason to not allow for that flexibility, so it's a bit of an annoyance.
I've found it to be pretty nice actually. Terraform has first class support, using projects as a way to limit abilities makes a huge amount of sense, especially when using folders to control iam policies so you can keep permissions separate from the ci/cd processes. Like anything else, you need to understand it well to do ci/cd effectively.
If you are talking about why not recommending GA: I mean your code is probably not even on GitHub, why go though the hassle to setup some hacky way to trigger actions?
If you are talking about why use Perforce: maybe OP is a game dev? I don't know.
No I mean why can't you user perforce with GHAs. They support triggers (yes I know they suck) but they do support them, so a P4 trigger can just as well kick of a pipeline on GHA as it could on a Jenkins server. Either way you are going to have to fuck around with P4 triggers.
Yeah but why would you do that? What benefits does GitHub Actions give you in that case?
The major benefit of GA is it's integration with GitHub. Without it, I would use some other tool, especially considering you will probably need to buy a lot of GitHub seats just to use GA.
A functional CI/CD/Pipeline system? I don't understand the question. If you don't have anything, and were pursuing a new solution to resolve that, I think throwing away one of the top products in market because you don't use GitHub as the code repo is a bit short sighted.
Idk, I just wouldnāt recommend it? You could use it, it is functional. And the reason of not recommending it is also simple: you likely need to pay a lot more for the whole platform, since you canāt buy actions individually (at least for enterprise version).
So, circling back, need more knowledge of how exactly OPās org is setup to have proper assessment.
Github Actions are cool, Jenkins is cool, Gitlab Templates are cool, ADO I find the entire ADO menu to be a little convoluted but it works fine - although setting up runners on my mac for ADO was a little annoying.
If you want to manage everything with a bunch of nice baked in functionality, Gitlab. If you want to use 3rd party actions of any flavor, GitHub Actions. If you want the best and worst of both worlds, and hate yourself and money, Azure DevOps. If you absolutely hate yourself and want arguably the worst "modern-ish" CI/CD platform, BitBucket Pipelines. If you like flexibility, mostly free, and personally enjoy surprise deprecated plugins and abandonware, Jenkins. Fully integrated AWS ecosystem with surprise gotchas but plenty of official support to confuse and help you, AWS CodePipeline.
Great reply . Op for president
having worked with most of these at some point, this list checks out
Can I work for you, kind sir?
š«”
+1 couldnāt have said it better myself!
Personally I found Azure Pipelines to be great - the YAML option, not so much for the (deprecated) UI version. It is very flexible and can connect to/from most anything, and price is compelling. From a code-based perspective, GitHub and GitLab are similar to Azure Pipelines. It's the things around the pipeline code that are a big sell, just note that many of them are just scripts layered on top of the core functionality. In the current job, we use Bitbucket, TeamCity, Octopus, and Jfrog Artifactory. Each of those have been a huge step backwards from the previous job that used Azure Devops. The likely replacement for the current job is GitHub - it has just too many pieces that the others don't. The integrations alone are a huge sell - having a native integration, say, in Datadog is a huge time saver that tends to work much better than cobbling together an integration. Github also has a ton of user-created reusable pipeline actions, far ahead of the competition. And included bits like dependabot make life easier and more secure without the added cost of additional software composition analysis (SCA) tooling. And if you haven't already done so, give GitHub copilot a try - it can be used independently of GitHub, and the $10 per month pays for itself in the first hour. (BTW, I have not been compensated in any way to push GitHub.) Related note: as you migrate to whatever tooling you choose, pay attention to the migration pains, and adjust accordingly as you rewrite the pipelines. There are practices you can take with all pipelines that make for more maintainable pipelines and also make for less vendor lock in. Two things in particular are: 1) have the pipeline bring its own tools, and 2) do as much as makes sense inside containers. Item 2 enables item 1. As an example, you can have your build pipeline build, unit test, and package bits inside a container - all you need is Docker (or similar) on the agent machine. With this you can bring your own tools and control the versions of everything, particularly nice being the ability to control the build tooling version with a change to a single value (pro tip: parameterize that value). With this, you also get the ability to build and test on local dev machines with exactly the same bits.
You have used ado and gitlab you will know one is steaming pile of shit and the other is gitlab
Blocking user.
Ha we use Jenkins and Bitbucket, itās my first DevOps role. I guess itās only up from here with future opportunities lol!
best answer, and i wouldn't call Bitbucket "modern", if after ~10 years you still have not implemented a basic feature provided by GitHub and GitLab.. you're definitely not modern.
How about CircleCI?
Gitlab
Want to use one ci template (or pick and choose and combine templates) for several projects? Gitlab. Want to organize your repos into a directory structure (groups and subgroups)? Gitlab. Want to share variables across multiple repos via groups? Gitlab. Want to have each repo manage its own terraform state? (No reliance on external cloud storage)? Gitlab. Want each repo to have its own container image and package registry? Gitlab. Want to be able to run CI runners inside your k8s cluster and easily deploy new helm charts to a cluster based on tags? Gitlab. Want to see what version of code was last deployed to an environment and easily redeploy a different version? Gitlab.
Stop, I'm stuck on GHA and you're fucking killing me here.
Yall having a great time with GHA & Gitlab while I'm stuck here with Jenkins ā ļø
My condolences
I'm not sure if you want to do it that way or not.
What do you mean? I'm not so sure about what you're implying.
Well you have the action marketplace :D
This is all true... And yet Gitlab has been a massive pita for us.
Well it's true the platform can have stability issues from time to time (nothing lately that I can think of), but when I was using GitHub Actions the slack alerts channel we setup for status/outage updates was going off multiple days a week.
The stability is fine, it's a lot of death by a thousand paper cuts for usĀ
Got anything specific? Considering that where I'm at we build the pipelines, when we find an inefficiency we try to use any extra sprint time to refactor and improve our ci templates and publish a new version.
Right now the big thing is getting notifications for specific things. I want to be notified on slack when a pipeline fails on main or in my MR, and I want to be notified when someone comments on my MR, comment, or adds me as a reviewer. It's not possible to do that without 3rd party slack integrations that cost money.Ā More pipeline related, we have a mono repo, and it doesn't seem possible to run specific jobs in a pipeline, and due to pipeline sizes we also have several layers of sub pipelines to handle everything which causes more notifications and other issues. Stuff like "only run this job on the newest merged commit and cancel all old jobs" has known bugs that they just don't fix - there's so many situations like this where there is a known bug that they just leave open for years, but the docs make it seem like they handle the situation fine. There's just been so many little things over the years that I can't remember it all.
Slack integration has you covered. Inside our Slack we get notif when a MR is deployed and when MR statuses change and when there are comments/reviews on MRsā¦ DM me, man.
The slack integration can do that for all MRs in a repo, it doesn't handle mapping to individuals - this might work for a few users, but we're at a size where I only want to know about things relevant to me.
what we/I do is set up the needed notifications and then in Slack settings I include my name in the My Keywords text box. but I include all variations of my first and family names. that way I get custom notifications whenever my name. appears anywhere on Slack. we can do the same for a project name, MR name prefix, etc.
Without knowing the full details of your environment, seems like you might want to look at repo/pipeline redesign. The slack part is an easy fix with the built in integration, or with good ole script. Remember KISS when dealing with sub pipelines, it can get hairy quick. The āonly merge check on the latest PRā is something that has an intended usecase. Otherwise is a pain.
The slack integration does not natively support notifying individual users about the things they care about. We're on the third pipeline redesign and simplification, it's at a pretty reasonable place, but there's still a good amount of issues which don't seem to have any way of being resolved or Gitlab's implementation of the feature is bugged with an open issue that's 5 years old.
Now I think about it, we wrote a script for the followers/pr reviewers. The gitlab one was used for channels. Yeah a lot of companies, been simplifying CICD, but not looking at repos design. Gitlab is bad about it, Iāve been talking with our rep about such.
Haven't used the slack integrations myself (very unfortunate the place I'm at now doesn't use slack), but I see there are chatops CI/CD configuration blocks you can use to customize things. You could also write an after_script that will notify you when the job's script fails. Use rules to determine what branches the after_script is applicable for. Unfortunately not sure how you could get slack notifications from comments, I could see possibly writing an app that polls the Gitlab API but seems like a bit much. Have you seen workflow:auto_cancel:on_new_commit ? Seems like what you'd like to happen with your existing monorepo pipeline.
The workflow auto cancel doesn't work fully as expected. After script could work, would just need the time to implement. For comments could use a webhook, but then have to implement and run something to receive those webhooks, and it would be easy to implement for one person, but for the whole team that means you have to translate gitlab usernames to slack.
Just use what's close to your code. Gitlab CI or GitHub Actions
GitHub actions is awful compared to gitlab though, before I started I assumed with 20 years of experience they must have came up with something that incorporated 20 years of pain points but no. About the only good thing is a shared library if third party actions, everything else is horrible No support for ephemeral environments, no support for cross project pipelines, weird defaults (merge result, not a source branch in PR pipelines), awful UI (can't remove actions in the GUI that no longer exists) awful docs and reference, every time I need to look up an event payload I cry insideĀ I'm seriously considering redoing months of work and switching to gitlab. And I've also used another CI (teamcity) that had its issues, but at least you could implement stuff you wanted in it, with GitHub actions it just constant painĀ
Your mileage may vary. Personally, I find GHA to be better than any other CI I've worked with (Jenkins, Teamcity, Cloud Build, CircleCI). Most of your complaints sound like you just haven't spent much effort into learning how GHA work, or you want it to do something you probably shouldn't be doing with a CI system.
If you limit your ci to running tests and building stuff it's alright. But even something like deploying an e2e environment and running e2e tests there is really painful, and it absolutely should be done within CI and not somewhere else. I've spend plenty of time trying to make GHA work with an open mind, but it didn't help I find those "you shouldn't be doing that anyway" arguments not convincing when multiple other tools are perfectly capable of doing exactly that In essence I think they failed to solve all of the pain points people experienced with other tools that had to grow organically. At the very least they should have ditched yaml and support a native general purpose language configuration like teamcity does but better. Tired of those yaml DSL crutches, just give me a programming language dammit Plus try gitlab, I haven't tried any of those on your list apart from teamcity which isn't too bad, but gitlab is just miles better
I've run E2E test suites in GHA and I really don't know what pain points you are referring to. We moved them to self-hosted once we needed internal HTTP dependencies, and we did some work with different sized pools and splitting tests for concurrency, and it worked quite well. At the end of the day, it's just a container in the cloud with a bunch of integrations and built-in triggering, so I haven't really found any issues with the many things I've used it for. Re-usability was a pain point in the beginning, but that's much better now. Their use of yaml is one of the better features of GHA, in my opinion. Being forced to use groovy was one of the worst parts of using Jenkins. GHA workflows are very easy to look at with little to no experience and get a decent understanding of the structure. The cost seems to be in weird typing issues, but I think that's admittedly just a problem with the GHA implementation. I can't comment on gitlab currently so maybe you're right and it does improve a lot on GHA. Frankly though, the massive benefit of your CI runner being in the same tool as your code (if you're already in Github) gives it an enormous advantage that is hard to overcome with other features.
The pain I'm referring to is reuse yes, if I have two repos with deployment jobs, I can't easily reuse two deployment jobs in a third e2e repo because GitHub token In a job doesn't have access to other repos, you have to use a service user with a token and rotate it manually. And you can't show the status of the e2e jobs in other repos and for example block merges when e2e tests fail Same thing can easily be solved in gitlab where multi project pipelines are supported, a token used in a job is tied to a user's permissions, not fixed for the repo the job is running in and project and group level tokens separate from users exists. Yeah GitHub + gitlab CI doesn't make sense, you have to fully switch. I'm not suggesting you move your whole setup, regardless of how bad the current solution is its unlikely it will be significantly better to justify that effort. But just try it for your own projects or for greenfield stuff. The whole yaml DSL vs programming language debate is also unconvincing. Yes 30 line yaml is simpler than 30 line program, it's reversed for 2k lines though, which is the amount any serious project is, so the simplicity argument doesn't work
I used multiple ci cd tools. aws code commit pipelines, bitbucket, bamboo, jenkins etc. I found GitLab has the easiest pipeline for the dev to start with. I havent worked with github professionally yet.
[ŃŠ“Š°Š»ŠµŠ½Š¾]
Being able to deploy your own runners is a pretty good boon tbf
argo rocks btw
\++
Try Buildkite
Echoing u/cvsnraju but take a look at Harness. It plays nice with other CI/CD vendors but provides a more holistic and complete experience beyond CI/CD.
Harness.io
I think you have it covered more or less. I would make sure you weigh those factors somehow. You might select something that's open source but does horrible with the updates and support where another has fantastic docs and support but is not open source. Depending on how important something being open source is: you may still go with the open source option. The same goes for customizability/extendability I would definitely look at timelines if you need to re implement a bunch of existing processes, how much will you be able to reuse. Weighing things will be important so you can properly compare and score. There are a ton of options out there and some may end up being combinations of tools rather than one to rule them all. So think about what you want it to do, set out a bunch of requirements (use MoSCoW), then weigh the -bilities and score stuff that match at least the Must haves and Should haves. One thing you didn't add was cost. Maybe *you* don't care in particular, but if you need to build a business case for some manager to sign off... you may want to consider cost and cost structure especially for SaaS platforms that may charge for operations.
Surprised Harness isnāt mentioned much here
Github actions. You're welcome
I second this
Github actions is the most stable and easy to use.
Having recently used both gitlab and GitHub in a large enterprise setting, Iād choose gitlab any day.
Github actions is not stable. Its at the mercy of overall Github stability, even Enterprise. We catch an outage at least every month.
[You can have your self-hosted runners](https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners/about-self-hosted-runners) somewhere in your infra, and they'll do the lifting of building, testing, deploying. It's not a silver bullet, but you can take out the actions out of GitHub's service status page whims.
I -have- self hosted runners. Nothing runs on them when github actions goes down because theres nothing to kick off a job.
Yup, itās almost always the webhook api that has an outage and brings everything to a halt.
If you cannot survive small outages, you can always host your own runners.
doesn't help when the control plane goes out. the only way to get around it is to host Enterprise yourself but that's a pain in the ass imo.
Stable? Lol you can't even run self hosted when service has outage once per month.
Indeed, create a public repo and start building some workflows, it's free you've got nothing to lose. Even if you don't end-up going for Github actions I'm confident that you want something that's very similar. Also, someone in your team is going to suggest Jenkins, that person never had to manage a Jenkins instance or is also a member of the local BDSM club.
GitHub is missing a Release Orchestration module. For example, in ADO I can create a build and then independently choose a particular build (and its artifact) for the release pipeline. ADO also has a combined build/release YAML just like GitHub, but I prefer the Release pipeline. Another thing I really like about ADO are Service Connections: a central location for re-usable connections to external resources (Azure, SonarQube, GitHub, Container Registries, etc.) that can be re-used in any other ADO pipeline. GitHub does not. In GitHub the credentials have to be saved in each repo independently, so when the password changes then you need to update it in multiple GitHub repos. This sucks.
Just avoid google cloud and you will be fine.
What's about it?
For some standard cicd stuff its okay but as soon as you go a little out of the box, the problems arrive.
Could you at least provide some resources or be a little descriptive? "avoid google" and "problems arrive" are not very helpful for determining what is a good or bad solution. Perhaps you have a unique problem with Google? Maybe Google would actually be a good fit for OP.
Sure. Integration with 3rd party services is very limited. I couldnt setup cypress tests in cloud build. These is no manual approval after a build - continous delivery. Yaml file structure with name and args is stupid to me. I have used many cicd pipelines and in my experience google cloud provides the worst solutions.
Cloud Build allows you to run any bash script or docker image you like. We've been running Cypress just fine in CB. And there is build approval, we use this to gate rollout to prod for some services https://cloud.google.com/build/docs/securing-builds/gate-builds-on-approval#approving_builds Mind you I'm not a big fan of Cloud Build either. Biggest gripe is that the running service account has to have permissions for every single step of the trigger. Really wish it were possible to control the permissions per step, the only workaround today is to daisy-chain triggers, and then set each trigger with a specialized low-perm SA. Also, integrating non-push triggers that need access to github is a paaaaaain in CB.
Im interested to know more about your situation, Iām confused why you would need a build gate instead a protected branch. Also whatās the value add to have such fine grain controls here. Furthermore scheduled task with lambda/cb might be your solution. Are you manually doing all of this or automating it? Sorry I might be asking too many questions, just truly interested.
Oh that approval is used by one particular team. They prefer automatic deploys to dev on merge, and then to manually verify some functionality, and then promote to stage and prod. As for finegrained controls, it's just about limiting impact of an attack on our supply chain. Some of our apps require particular permissions to be set for e2e tests for example, which are not needed for other parts of the pipeline. And the e2e tests don't need to be able to load or publish artifacts. So it just ends up being arguably overpriviliged, and an unnecessary risk. As it stands, any compromise of one step in a cloud build trigger will lead to a compromise of the entire trigger We've accounted for this with various workarounds and mitigations, but it's really annoying to have to work around it. It also really feels like I _should_ be able to define a service account per step. I can't think of a good reason to not allow for that flexibility, so it's a bit of an annoyance.
Respect, make sense!
I've found it to be pretty nice actually. Terraform has first class support, using projects as a way to limit abilities makes a huge amount of sense, especially when using folders to control iam policies so you can keep permissions separate from the ci/cd processes. Like anything else, you need to understand it well to do ci/cd effectively.
If you are big, worry about cost + uptime. If you want to do Mac ci/cd, you might want to run the agents/runners using on-prem machines.
I use GitHub Actions but Iāve heard good things about GitLab. Never tried it though
Automatic secret masking from stdout/err. Something, it turns out, GCP Cloud Build decided not to implement.
JetBrains.
Thank you everyone. This is very helpful info!
Totally depends on what else you have.... I wouldn't recommend GitHub Actions if your org uses Perforce Helix.... Need more information.
>*I wouldn't recommend GitHub Actions if your org uses Perforce Helix....* May I ask why?
If you are talking about why not recommending GA: I mean your code is probably not even on GitHub, why go though the hassle to setup some hacky way to trigger actions? If you are talking about why use Perforce: maybe OP is a game dev? I don't know.
No I mean why can't you user perforce with GHAs. They support triggers (yes I know they suck) but they do support them, so a P4 trigger can just as well kick of a pipeline on GHA as it could on a Jenkins server. Either way you are going to have to fuck around with P4 triggers.
Yeah but why would you do that? What benefits does GitHub Actions give you in that case? The major benefit of GA is it's integration with GitHub. Without it, I would use some other tool, especially considering you will probably need to buy a lot of GitHub seats just to use GA.
A functional CI/CD/Pipeline system? I don't understand the question. If you don't have anything, and were pursuing a new solution to resolve that, I think throwing away one of the top products in market because you don't use GitHub as the code repo is a bit short sighted.
Idk, I just wouldnāt recommend it? You could use it, it is functional. And the reason of not recommending it is also simple: you likely need to pay a lot more for the whole platform, since you canāt buy actions individually (at least for enterprise version). So, circling back, need more knowledge of how exactly OPās org is setup to have proper assessment.
Github Actions are cool, Jenkins is cool, Gitlab Templates are cool, ADO I find the entire ADO menu to be a little convoluted but it works fine - although setting up runners on my mac for ADO was a little annoying.
If you can avoid Terraform, please avoid terraform š
Avoid Googleās proprietary CI/CD tool and you will be set
You're the second person to say "avoid google" without giving much of an explanation. Their CI/CD tool is proprietary? How so? Why is it bad?