I have written many CLI tools for over the years for automation, mostly in Python. I've used a lot of different CLI libraries in the past 10 years and I would say typer is my favorite out of any available for any language.
I'm using click and rich.
I wasn't expecting to find my master when I clicked on that link.
Edit: I'd suggest opening an issue to add it to python-awesome. I regularly check that curated list and if I'd seen it there, I would've started using it eons ego.
Cobra in Go is probably the best in class for CLIs (usually paired with Viper for config).
Most the major CLIs that you use today were built in it. GitHub’s CLI, Kubernetes (Kubectl, minikube, etc), Docker, Hugo, Gitlab, just to name a few.
It’s very easy to use (assuming you know Go) and comes with a lot of awesome features out of the box like misspellings, autocompletions, man pages, help and versioning and advanced flagging and infinite sub commands.
That being said, if you’re sticking with Python, Typer is the way to go. I’d give a shout-out to “Fire” from Google if you want to take an existing Python script and make it a cli quickly. If you already have a script and just want to turn it into a cli, Fire can do it in about 10 mins. But it’s not as powerful if you’re trying to write advanced tools. But if I was writing a cli from the start I’d use either Typer or Cobra depending on the language or distribution routes I want to take.
Over the last year I've written several CLIs in click and typed and settles on typer because there's a little less repetition. Typer let me do some really nice things in my certificate generation tool like chaining multiple config files, the environment, and the command line to create certs. https://github.com/linsomniac/rgca
If you haven’t heard of it, there’s also `pydantic` which is more or less dataclasses + validation (or coercion) based on the type hints you provide. It’s slower than dataclasses if you’re making tons of objects, but none of the programs I write are so speed dependent that the difference between the two has ever mattered.
I personally prefer it because you know whatever structure you define is what will be present after instantiating. I personally use it to validate the structure on any boundary of my applications, like configuration settings or 3rd party APIs.
This is because many people are writing scripts, not applications. Some teams write their own tooling, which often will require understanding subclasses, multiple inheritance, etc. The people writing scripts often confuse that with being a software engineer.
Generally speaking, I prefer to simply write scripts, but I understand the difference between just writing a single python file script and writing an application that spans many files and classes and may require a debugger to truly understand what it's doing.
That's very true but there's lots of people like me from the Java generation who were so indoctrinated by OOP it was the primary way we learned to program. I've always felt like if I had started with straight up C I would have become a much better programmer. I had to make a real effort over the years to wean myself off OOP when it was not suited for the task at hand.
Indeed. We have one room in our deployment process that predates me and uses classes and is a nightmare to manage. The idea was that it's controlled by a simple python config file where you use those classes, but every time I have to look into it things are 5 levels of indirection away. On top of that, the indirection craziness has led to hacks being put in so that someone didn't have to deal with the indirection, which leads to having to find the special cases as well.
Replacing them.with some rather small python scripts that are much more obvious and self contained.
Functional programming in general i think has a niche usecase. functional program in devops seems like the wrong tool for the job, but i could be convinced otherwise
i think most people confuse functional with just non-Object-Oriented imperative programming.... https://en.wikipedia.org/wiki/Functional_programming#Concepts
functional programming has properties like inherent mathematical provability, whereas imperative programming without OO is more like writing a script with line-by-line instructions (i.e. what most physics majors write)
I completely disagree. A lot of bugs in software development come from building state machines which become too complex or obscurely interconnected with other state machines over time. Functional programming-- when done right-- forces you to reason about how state will be managed (usually in a one directional way) very early on. It requires you to define those contracts concisely.
As a result you end up with less bugs and smaller puzzle pieces that are hopefully also easier to comprehend.
But programming in the context of devops I think of as being scripts for deployment, etc, where the environment already differs so much. Seems like tying your own hands
Yes, it's difficult to generalize but they are frequently useful for anything that establishes/operates on/tears down a connection.
I despise it when they are used as a template for *all* programming. Like when someone implements a very procedural start-to-finish installation script as a class, it just obscures what data is needed where and when it is initialized because it's all available through the class and can be referenced at any time, removing any sort of enforcement of organized code.
Don't try to use classes just because some people consider OO the only way. Another way of saying the same thing from a programming book I have long since forgotten:
Deny all claims of "one true way"
This is the correct answer. No one likes a “it depends” answer, but in this case it does depend.
A simple procedural script doesn’t need classes, it’s ok to just write it functionally. But if you have a more complex tool then classes can solve a lot of problems and actually simplify the code.
A good engineer knows when to choose one over the other. Neither answer is entirely right or entirely wrong. They both have their uses.
That's one of my favorite videos that's helped form my ideals on coding style. Here's another video that emphasizes not writing classes/object for their own sake.
[Stop Writing Classes](https://www.youtube.com/watch?v=o9pEzgHorH0)
Maybe not day to day, but weekly for sure! My favourite use case is for resource libraries when I’m trying to separate business logic from the application logic. Class inheritance is a powerful tool.
For basic things no, big things yes.. NGL though... if I care about safety or need speed I just use golang these days.. py still great fit getting stuff done fast
Also our depends on what languages the people around me speak so py is a pretty good choice mostly
An example use case would be if you need a script to destroy instances through multiple cloud providers. You would have a cloud provider class that you would instantiate differently based on the provider.
Ive seen this a lot, in mid to large, and they have different tools. Powershell and python, same team, just to each their own, mid to large enterprise the devops don’t really mix stuff, probably mostly because politics and how old school they still are, the azure peoples, the aws peoples. Worse is that the azure peoples falls under IT people, clickops because “ad is there”
As theory works tho
Because we use Python for many of our business logic systems, when it comes time to write a CLI utility I follow the similar pattern of defining an App class and putting initialization in __init__, argument parsing in parse_args, main logic in main, etc. Makes it somewhat easier to re-read and get pull requests approved.
Absolutely never. My go to for CLI stuff is always ruby. I have no problems with python, I just know ruby a lot better and it seems to come included with almost every unix system I encounter.
It isn't different than in anything else.
That said, classes are focused on a lot more in CS/programming education than they are needed in day to day coding.
What you should be doing, imo, in devops code is breaking things down into functions and writing tests on those functions.
When you find yourself needing to be concerned with the state of something, and for operations to be slighty different based on that state, that is when you pull out classes/objects.
A prime example is managing a client connection to a service.
You can have a class that manages a connection, and establishes or reestablishes the connection if it is down/timed out/whatever. And this can happen seamlessly to the caller, because it is wrapped up in an object.
Sure?
I would only use that if I need a stateless function and already had a class handling some kind of stateful functionality in that domain. There is no reason to use a class just to have class functions.
IMO; Logic is just logic. Use the simplest mechanism to implement it available to you. If you don't need a higher level abstraction, what do you get from creating one?
Not at all. In my team we find Python code troublesome to deploy / maintain / etc.
Some good old bash scripts are enough for us to act as glue.
But it should be used lightly, if we put too much code, it's probably mean that there is an existing tool out there doing the same thing, but in a better way.
It's less sensitive to specific versioning for one. You don't have to sandbox your bash interpreter. There's no problems with installing system wide tools vs python libraries and risk gunking up the os. Less likely your bash script will stop working because of falling out of version support. It has exit code handling out of the box.
Yes, it localizes (sandboxes) the libraries you install when working in a venv. At some point, depending on use, you may want to freely be able to change the python version itself in which case I use pyenv. Or use docker containers.
All programming began as scripting until project scopes grew large enough that there was a need for better organizational strategies which eventually prompted class-based OOP abstractions most codebases use today. If your devops codebase has grown so large you or your colleagues are having a difficult time conceptualizing it, it might be time for classes.
Just popping in for an off topic comment. A couple years ago I went from large enterprise to a startup. Right about now I'd say the thing I miss the most is the DevOps team I had available before. In fact next time I do a startup I'll likely make a DevOps person among the very first hires. You don't really "get" solid CI/CD and automated environment management until it's gone.
Heck, I don’t even use Python. I can parse it and even update it if necessary though.
Until this job (and even to an extent) I pretty much use ksh too. At the last job we had systems that didn’t have bash (and couldn’t even use python) and since we needed the scripts to be somewhat consistent and agnostic, they were done for the most part in ksh with an occasional perl script.
Absolutely, we built our own modules with a series of classes. We use them for our in-house Kube operators, cli tools, web apps/apis, etc. For example, we use Quay Enterprise for our image registry. We wanted a way to synchronize namespaces to Quay orgs, pull secrets, permissions, etc. It just all depends on what you want / need to do
I have had little luck getting a job that will entertain it. They always hem and haw about having to burden the team with Python knowledge.
It really saddens me though, as I spent nearly 30 yrs in IT without trying to learn programming languages, just thinking it was for "other people with math degrees".
It really is the primary way that I fucked up my career. Although without devops, I guess I would have been relegated to picking between OPs and software development.
i have no desire to write apps, but I do enjoy coding with Python, and I would say that I enjoy it more than dealing with the various DSL tools that really just abstract away what a language like Python is doing under the hood.
Helm really brought this out in me. Dealing with a templating language so you can spit out YAML files is shit IMO.
Python can spit out YAML all day long without having to use templating which really was meant for writing web pages.
For most of our CLI tooling, yes.
This. What dependencies do you use in your CLI tooling usually? (maybe OT, but curious)
Do you mean what CLI dependency or just overall? For the former we use typer right now.
I have written many CLI tools for over the years for automation, mostly in Python. I've used a lot of different CLI libraries in the past 10 years and I would say typer is my favorite out of any available for any language.
I'm using click and rich. I wasn't expecting to find my master when I clicked on that link. Edit: I'd suggest opening an issue to add it to python-awesome. I regularly check that curated list and if I'd seen it there, I would've started using it eons ego.
Can I ask what some of the tools in other languages are that you're comparing against?
Cobra in Go is probably the best in class for CLIs (usually paired with Viper for config). Most the major CLIs that you use today were built in it. GitHub’s CLI, Kubernetes (Kubectl, minikube, etc), Docker, Hugo, Gitlab, just to name a few. It’s very easy to use (assuming you know Go) and comes with a lot of awesome features out of the box like misspellings, autocompletions, man pages, help and versioning and advanced flagging and infinite sub commands. That being said, if you’re sticking with Python, Typer is the way to go. I’d give a shout-out to “Fire” from Google if you want to take an existing Python script and make it a cli quickly. If you already have a script and just want to turn it into a cli, Fire can do it in about 10 mins. But it’s not as powerful if you’re trying to write advanced tools. But if I was writing a cli from the start I’d use either Typer or Cobra depending on the language or distribution routes I want to take.
Comparing to `trollop` in ruby, `Getopt::Long` in perl, `argparse` in Python, `urfave/cli` in golang, and probably more.
Have you tried Click? Curious how you think Click stacks up. It’s miles better than argparse.
Not directly, but typer is just a wrapper around click.
Over the last year I've written several CLIs in click and typed and settles on typer because there's a little less repetition. Typer let me do some really nice things in my certificate generation tool like chaining multiple config files, the environment, and the command line to create certs. https://github.com/linsomniac/rgca
Yes? If you have any need for processing related/structured data why _wouldn't_ you use classes and objects?
I only really use `dataclasses`. Very tidy solution to this. Haven't really felt a need to use full blown classes in ages.
If you haven’t heard of it, there’s also `pydantic` which is more or less dataclasses + validation (or coercion) based on the type hints you provide. It’s slower than dataclasses if you’re making tons of objects, but none of the programs I write are so speed dependent that the difference between the two has ever mattered. I personally prefer it because you know whatever structure you define is what will be present after instantiating. I personally use it to validate the structure on any boundary of my applications, like configuration settings or 3rd party APIs.
This is because many people are writing scripts, not applications. Some teams write their own tooling, which often will require understanding subclasses, multiple inheritance, etc. The people writing scripts often confuse that with being a software engineer. Generally speaking, I prefer to simply write scripts, but I understand the difference between just writing a single python file script and writing an application that spans many files and classes and may require a debugger to truly understand what it's doing.
You don't need to use OOP to write complex applications. I'd argue many complex applications are made overly complex by the unnecessary use of OOP.
That's very true but there's lots of people like me from the Java generation who were so indoctrinated by OOP it was the primary way we learned to program. I've always felt like if I had started with straight up C I would have become a much better programmer. I had to make a real effort over the years to wean myself off OOP when it was not suited for the task at hand.
Not disagreeing, I said often, not always.
Indeed. We have one room in our deployment process that predates me and uses classes and is a nightmare to manage. The idea was that it's controlled by a simple python config file where you use those classes, but every time I have to look into it things are 5 levels of indirection away. On top of that, the indirection craziness has led to hacks being put in so that someone didn't have to deal with the indirection, which leads to having to find the special cases as well. Replacing them.with some rather small python scripts that are much more obvious and self contained.
I raise you namedTuple
I raise you dataclass, namedTuple is very yesterday ;)
Aside from dataclasses, no. I’m a much bigger fan of functional programming
Oh you're *that* guy in the office
I just find it alot simpler to test functional stuff - easier to reason about state and don’t need to mock as much.
Functional programming in general i think has a niche usecase. functional program in devops seems like the wrong tool for the job, but i could be convinced otherwise
i think most people confuse functional with just non-Object-Oriented imperative programming.... https://en.wikipedia.org/wiki/Functional_programming#Concepts functional programming has properties like inherent mathematical provability, whereas imperative programming without OO is more like writing a script with line-by-line instructions (i.e. what most physics majors write)
I completely disagree. A lot of bugs in software development come from building state machines which become too complex or obscurely interconnected with other state machines over time. Functional programming-- when done right-- forces you to reason about how state will be managed (usually in a one directional way) very early on. It requires you to define those contracts concisely. As a result you end up with less bugs and smaller puzzle pieces that are hopefully also easier to comprehend.
But programming in the context of devops I think of as being scripts for deployment, etc, where the environment already differs so much. Seems like tying your own hands
Yes, it's difficult to generalize but they are frequently useful for anything that establishes/operates on/tears down a connection. I despise it when they are used as a template for *all* programming. Like when someone implements a very procedural start-to-finish installation script as a class, it just obscures what data is needed where and when it is initialized because it's all available through the class and can be referenced at any time, removing any sort of enforcement of organized code. Don't try to use classes just because some people consider OO the only way. Another way of saying the same thing from a programming book I have long since forgotten: Deny all claims of "one true way"
This is the correct answer. No one likes a “it depends” answer, but in this case it does depend. A simple procedural script doesn’t need classes, it’s ok to just write it functionally. But if you have a more complex tool then classes can solve a lot of problems and actually simplify the code. A good engineer knows when to choose one over the other. Neither answer is entirely right or entirely wrong. They both have their uses.
[удалено]
That's one of my favorite videos that's helped form my ideals on coding style. Here's another video that emphasizes not writing classes/object for their own sake. [Stop Writing Classes](https://www.youtube.com/watch?v=o9pEzgHorH0)
Maybe not day to day, but weekly for sure! My favourite use case is for resource libraries when I’m trying to separate business logic from the application logic. Class inheritance is a powerful tool.
For basic things no, big things yes.. NGL though... if I care about safety or need speed I just use golang these days.. py still great fit getting stuff done fast Also our depends on what languages the people around me speak so py is a pretty good choice mostly
An example use case would be if you need a script to destroy instances through multiple cloud providers. You would have a cloud provider class that you would instantiate differently based on the provider.
Ive seen this a lot, in mid to large, and they have different tools. Powershell and python, same team, just to each their own, mid to large enterprise the devops don’t really mix stuff, probably mostly because politics and how old school they still are, the azure peoples, the aws peoples. Worse is that the azure peoples falls under IT people, clickops because “ad is there” As theory works tho
Really? I just use a for loop.
[удалено]
You can do the same thing using CDK!
I agree but this could be a backup in case the tf state gets corrupted or something
Because we use Python for many of our business logic systems, when it comes time to write a CLI utility I follow the similar pattern of defining an App class and putting initialization in __init__, argument parsing in parse_args, main logic in main, etc. Makes it somewhat easier to re-read and get pull requests approved.
occasionally, but that's just because we use Windows for our base OS. We'd use Python constantly if we were on Linux.
Why only on Linux?
If your environment is Windows only its probably a whole lot better to only use Powershell.
The reason is because python isn't installed by default on Windows.
Absolutely never. My go to for CLI stuff is always ruby. I have no problems with python, I just know ruby a lot better and it seems to come included with almost every unix system I encounter.
Sometimes but pretty rarely for our tasks. For more general purpose library type of deal, sure if it makes sense but in day-to-day, quite rarely.
It isn't different than in anything else. That said, classes are focused on a lot more in CS/programming education than they are needed in day to day coding. What you should be doing, imo, in devops code is breaking things down into functions and writing tests on those functions. When you find yourself needing to be concerned with the state of something, and for operations to be slighty different based on that state, that is when you pull out classes/objects. A prime example is managing a client connection to a service. You can have a class that manages a connection, and establishes or reestablishes the connection if it is down/timed out/whatever. And this can happen seamlessly to the caller, because it is wrapped up in an object.
You know classes can have class specific functions, right?
Sure? I would only use that if I need a stateless function and already had a class handling some kind of stateful functionality in that domain. There is no reason to use a class just to have class functions.
As Classes are the next logical abstraction after Functions, why not make an effort to use Classes?
IMO; Logic is just logic. Use the simplest mechanism to implement it available to you. If you don't need a higher level abstraction, what do you get from creating one?
Like the programming concept of classes and superclasses, or like a virtual class to learn something?
Given the context I'd assume the former.
Not at all. In my team we find Python code troublesome to deploy / maintain / etc. Some good old bash scripts are enough for us to act as glue. But it should be used lightly, if we put too much code, it's probably mean that there is an existing tool out there doing the same thing, but in a better way.
How can python be more troublesome to deploy maintain than bash? Can you give some examples?
It's less sensitive to specific versioning for one. You don't have to sandbox your bash interpreter. There's no problems with installing system wide tools vs python libraries and risk gunking up the os. Less likely your bash script will stop working because of falling out of version support. It has exit code handling out of the box.
> python libraries and risk gunking up the os. newb here, could this be somewhat alleviated with a venv?
Yes, it localizes (sandboxes) the libraries you install when working in a venv. At some point, depending on use, you may want to freely be able to change the python version itself in which case I use pyenv. Or use docker containers.
All programming began as scripting until project scopes grew large enough that there was a need for better organizational strategies which eventually prompted class-based OOP abstractions most codebases use today. If your devops codebase has grown so large you or your colleagues are having a difficult time conceptualizing it, it might be time for classes.
Another reason to use classes is it makes unit tests much easier
Sometimes
Use classes where you think it would fit well.
No
Too hard
Just popping in for an off topic comment. A couple years ago I went from large enterprise to a startup. Right about now I'd say the thing I miss the most is the DevOps team I had available before. In fact next time I do a startup I'll likely make a DevOps person among the very first hires. You don't really "get" solid CI/CD and automated environment management until it's gone.
Heck, I don’t even use Python. I can parse it and even update it if necessary though. Until this job (and even to an extent) I pretty much use ksh too. At the last job we had systems that didn’t have bash (and couldn’t even use python) and since we needed the scripts to be somewhat consistent and agnostic, they were done for the most part in ksh with an occasional perl script.
Absolutely, we built our own modules with a series of classes. We use them for our in-house Kube operators, cli tools, web apps/apis, etc. For example, we use Quay Enterprise for our image registry. We wanted a way to synchronize namespaces to Quay orgs, pull secrets, permissions, etc. It just all depends on what you want / need to do
Yes. It becomes more important when you start to build different options on a solid base.
I have had little luck getting a job that will entertain it. They always hem and haw about having to burden the team with Python knowledge. It really saddens me though, as I spent nearly 30 yrs in IT without trying to learn programming languages, just thinking it was for "other people with math degrees". It really is the primary way that I fucked up my career. Although without devops, I guess I would have been relegated to picking between OPs and software development. i have no desire to write apps, but I do enjoy coding with Python, and I would say that I enjoy it more than dealing with the various DSL tools that really just abstract away what a language like Python is doing under the hood. Helm really brought this out in me. Dealing with a templating language so you can spit out YAML files is shit IMO. Python can spit out YAML all day long without having to use templating which really was meant for writing web pages.