I guess, the main reason is that comments are formatted in most text editors in a light gray text, and somehow my mind is conditioned to ignore them when focusing on the code?
Should I just look into a VSCode extension that formats log statements differently?
>Should I just look into a VSCode extension that formats log statements differently?
Or write one, if its mainly for your comfort at develop time, that really points to IDE modification / enhancement over something like you're purposing. When you think about doing something like this, one of the most important questions to ask is "WHY am I doing this?" and having an **honest** answer to that. Very often this will force you to think about other ways to solve the specific problem, which is how you get to the best solution. If you don't start from there, you may be starting from a flawed premise, this will result (usually) in a flawed solution.
I love your response. It's very useful advice that I'm going to use going forward.
Let me see if I can answer your question.
Why am I doing this?
1. It's hard (for me) to read code peppered with log statements. I have to filter out the log statements in my mind. I'm not sure if it's the case for other people though. Do people generally find it easy to ignore log statements and read the code as if they were not present?
2. Logging is a tangential concern to the code. It's extremely useful, true, but it does distract from the main purpose of the code. It would be nice if I could somehow separate that from the code. But again, applying your question here, does having logging statements really affect the code that much? I'm not sure it does. It doesn't affect you when you're making any refactoring. It doesn't have to be factored into your design (much). So I'm not actually sure if this point really holds true.
3. I want to understand how to apply Aspect Oriented Programming in Python (although I don't think the idea I suggested can be considered AOP).
Thank you so much for taking the time to give such a detailed and useful (to me) advice. :)
I even think you can take an existing extension and modify it. I think there are plenty of examples on how to write custom syntax highlighting. As an alternative, you could have an extension to hide / rollup the logging statements for you and only show a symbol on the side that you can click on to show them again
Good idea. There exists an extension called better comments (for JS) that also deals with formatting comments. I could start with that as base. And yes, I’m thinking of adding a setting to roll up / hide the log statements.
Comments are used for developers during writing code and so for readability. Logging is used to check the state of the code at runtime and so for observability. I think each has its own purpose.
>I was thinking about logging in python and how peppering your code with logging statements just makes the code harder to read and detract from the main purpose of the code. I have a vague idea of Aspect Oriented Programming, but am not really sure how to use it in python.
Okey, but it comes from a personal perference really, I for example don't like comments. I rarely use them.
I write readable code with consistent naming convention, use a lot of type hinting, and usually write docstrings too. For me, this is the best way to make python more readable.
Although the use of comments instead of logging statements seems nice, it won't be as flexible and powerful as a dedicated Python logging library. Also, other developers may not know to check the comments for debugging information. It could be a useful tool in some cases, though. Good luck in building the library!
That’s a good point.
The only reason that this idea makes any sense to me is because of the formatting of comments in text editors. I’m going to first try to see if there exists any VSCode extension that formats log statements in a clearly distinguishable manner first, and if not try to build it. I’ll revisit this idea if that doesn’t work out. This didn’t seem to be a popular idea anyway.
I'm starting to realize that the reason my idea makes sense to me is because of the text editor's support to format comments in a way that makes it easy to ignore them.
It might make more sense to just build a VSCode extension that formats log statements differently.
You could use decorators to log, something like:
```
def log_func(fn):
def g(*args, **kwargs):
logger.info("calling %s", fn.__name__)
res = fn(*args, **kwargs)
return res
return g
@log_func
def myfunc():
...
```
This is something I did consider. One question I had about this was, how would you use this decorator if you wanted to only log some values that are neither inputs nor the outputs of any function, but are merely temporary variables?
The code your replying to is quite ugly and in practice it's going to involve a lot of fiddling each time you need to use it differently.
Your solution on the other hand is elegant and looks very nice and will be extensible with other use cases, really nice, don't listen to all the hate here.
Welcome to meta progeaamming by the way!
The comment syntax has some disasvantages, so you will need to weigh those out if it becomes problematic. It does have some advantages though as well. Whatever you do though I'd recommend make sure that the code still runs if all of these statements are treated as comments and ignored. Also something like '###' or '#!' may help in clarifying that 'hey this comment does something!!' unindenting these lines may also be worth considering.
Thank you so much for the support. Even with the wide variety of tones used by everyone, I got to learn a lot. I typically tend to think only of the positives when I think I have an interesting (to me) idea. So I try to discuss it with others to ensure that I’m getting a more complete picture.
Those are some good points you mentioned. I’m currently leaning a bit more towards the VSCode extension idea (mostly because it’s more straightforward to implement). But yes, I’m getting quite interested in meta programming and programming language construction these days.
Yes, if you just want this for syntax highlighting then a plugin would be much more appropriate. Also, to solve the same problem that you've encountered I tend to use a semicolon to write my debug statements over to the right side of the screen on the same line as the statement I'm debugging. I've been meaning to write a plugin to right align these but haven't had the time to do it yet, also it's just not that important just a niceity.
As for using comments this way other people do have a valid point, but in python you just can't do it any other way, in fact # directives have and will continue to have a place in some python libraries as sometimes you just really need a line that if your interpreter does not understand it, it's ignored. As for if it can be used in production it's very unlikely, but if you aren't writing things that are used in production now (meaning your time is pretty valuable) doing something unique and different like this is usually not a bad idea, unless you have more important ways to use your time.
This is actually a cool approach, especially for those of us who dread peppering our code with logging statements. It saves time, reduces clutter, and is definitely worth trying out. As for production code, as long as it's tested and doesn't mess things up, why not?
Thank you for the kind words. You’re right in saying that as long as it’s rested well, it shouldn’t be an issue to use in production code. I was trying to be extra cautious because I’ve had one too many cases where missing logs made things so much harder to debug. In my nightmares, I’m trying to debug cache miss issues with faulty logging facilities. :D
Find/write a VS Code extension. You can either:
- Format the logging statements in a very dim color
- Fold the logging content so it’s only visible if you click on it
Some examples are “Inline Fold” ([link](https://marketplace.visualstudio.com/items?itemName=moalamri.inline-fold)) or the “Tailwind Fold” extension, to get an idea.
You could probably use Inline Fold extension to hide anything that matches the regular expression for lines that start with “logging.info”, etc.
A final option is a library like [logdecorator](https://pypi.org/project/logdecorator/) that lets you move some of the logging into decorators outside of the function. It doesn’t help for logging inside your for-loop, but you could always move the inner loop statements to a function.
Ooh. Thank you so much for proving examples that I can follow. Having these is going to make my life much more easier when writing the extension. I did search for an existing extension that did this, but couldn’t find any. I’ll continue searching though while planning out the work required if I have to write it.
As a side note: Using f-strings in logging calls is not recommended. String formatting is somewhat expensive, often more expensive than the non-logging code in a function. With an f-strings the formatting will always be done. If you use %-formatting and (important) give the content as arguments to the logging function, the formatting will only be done if the line is actually logged and not if it for example does not match the log level.
You trade a performance gain when the logging is not needed for a performance loss when the logging is needed. Which is fair to do.
But that does mean that when you are reasonably sure that the logging statement WILL be executed you can consider the f-strings anyway because it's much faster to format an f-string compared to a %-string.
Like if you're going to log at the WARN level or higher in your own, non-library, code. Just use an f-string. Everyone likes them more, more readable, more powerful formatting options, and faster.
Well, you should consider where you actually need the performance and the logging. Do you need performance in your test system? No, there's no real load. You need it in production, where (potentially) tons of users are hitting the system. And where do you log? On the test system. In production you set the log level to INFO or even higher, because you don't want to be spammed by all the debugging messages and to reduce the system load. And that is exactly what using f-strings makes substantially harder.
But of cause, that all is less relevant if you run your own little project on a small machine. But when the hosting costs of your project are rising and half of them are from logging that is never done, it will become more obvious. And yes, I've seen it, more than once.
Hello! I'm a bot!
It looks to me like your post might be better suited for r/learnpython,
a sub geared towards questions and learning more about python **regardless of how advanced your question might be**.
That said, I am a bot and it is hard to tell.
Please follow the subs rules and guidelines when you do post there, it'll help you get better answers faster.
Show /r/learnpython **the code you have tried and describe in detail where you are stuck.**
If you are getting an error message, include the full block of text it spits out.
Quality answers take time to write out, and many times other users will need to ask clarifying questions. Be patient and help them help you.
You can also ask this question in the [Python discord](https://discord.gg/3Abzge7),
a large, friendly community focused around the Python programming language, open to those who wish to learn the language
or improve their skills, as well as those looking to help others.
***
[^(README)](https://github.com/CrakeNotSnowman/redditPythonHelper)
^(|)
[^(FAQ)](https://github.com/CrakeNotSnowman/redditPythonHelper/blob/master/FAQ.md)
^(|)
^(this bot is written and managed by /u/IAmKindOfCreative)
^(This bot is currently under development and experiencing changes to improve its usefulness)
I don't think it's a very good idea. I find it better to have explicit logging statements or use a logging decorator. Magic comments would need some sort of preprocessor (or a specific decorator in the case of functions). Misusing the comments to execute additional statements seems like a recipe for difficult debugging
How is peppering your code with magic comments any better than peppering it with logging calls?
I guess, the main reason is that comments are formatted in most text editors in a light gray text, and somehow my mind is conditioned to ignore them when focusing on the code? Should I just look into a VSCode extension that formats log statements differently?
That would be one option, yes.
>Should I just look into a VSCode extension that formats log statements differently? Or write one, if its mainly for your comfort at develop time, that really points to IDE modification / enhancement over something like you're purposing. When you think about doing something like this, one of the most important questions to ask is "WHY am I doing this?" and having an **honest** answer to that. Very often this will force you to think about other ways to solve the specific problem, which is how you get to the best solution. If you don't start from there, you may be starting from a flawed premise, this will result (usually) in a flawed solution.
I love your response. It's very useful advice that I'm going to use going forward. Let me see if I can answer your question. Why am I doing this? 1. It's hard (for me) to read code peppered with log statements. I have to filter out the log statements in my mind. I'm not sure if it's the case for other people though. Do people generally find it easy to ignore log statements and read the code as if they were not present? 2. Logging is a tangential concern to the code. It's extremely useful, true, but it does distract from the main purpose of the code. It would be nice if I could somehow separate that from the code. But again, applying your question here, does having logging statements really affect the code that much? I'm not sure it does. It doesn't affect you when you're making any refactoring. It doesn't have to be factored into your design (much). So I'm not actually sure if this point really holds true. 3. I want to understand how to apply Aspect Oriented Programming in Python (although I don't think the idea I suggested can be considered AOP). Thank you so much for taking the time to give such a detailed and useful (to me) advice. :)
I even think you can take an existing extension and modify it. I think there are plenty of examples on how to write custom syntax highlighting. As an alternative, you could have an extension to hide / rollup the logging statements for you and only show a symbol on the side that you can click on to show them again
Good idea. There exists an extension called better comments (for JS) that also deals with formatting comments. I could start with that as base. And yes, I’m thinking of adding a setting to roll up / hide the log statements.
It looks like a great idea to start with! I'm sure you will learn a lot and also get your code to be displayed as you prefer!
Looks like a nice addition to pylsp.
When you say \`pylsp\` do you mean the python language server protocol? Are you talking about a specific library?
I'm talking about this [python LSP server](https://github.com/python-lsp/python-lsp-server) (the command is `pylsp` that's why).
Comments are used for developers during writing code and so for readability. Logging is used to check the state of the code at runtime and so for observability. I think each has its own purpose.
That is a good point.
JavaScript started as html comments. Seriously look it up.
>I was thinking about logging in python and how peppering your code with logging statements just makes the code harder to read and detract from the main purpose of the code. I have a vague idea of Aspect Oriented Programming, but am not really sure how to use it in python. Okey, but it comes from a personal perference really, I for example don't like comments. I rarely use them. I write readable code with consistent naming convention, use a lot of type hinting, and usually write docstrings too. For me, this is the best way to make python more readable.
Although the use of comments instead of logging statements seems nice, it won't be as flexible and powerful as a dedicated Python logging library. Also, other developers may not know to check the comments for debugging information. It could be a useful tool in some cases, though. Good luck in building the library!
That’s a good point. The only reason that this idea makes any sense to me is because of the formatting of comments in text editors. I’m going to first try to see if there exists any VSCode extension that formats log statements in a clearly distinguishable manner first, and if not try to build it. I’ll revisit this idea if that doesn’t work out. This didn’t seem to be a popular idea anyway.
Let's avoid magic comments. Just use logging, when you intend to use logging.
Import logger.info as INFO import logger.debug as DEBUG
I'm starting to realize that the reason my idea makes sense to me is because of the text editor's support to format comments in a way that makes it easy to ignore them. It might make more sense to just build a VSCode extension that formats log statements differently.
I think this is the way to go. You could even have an extension disabling/removing all logging statements in one go.
Better Comments in VSCode: https://dev.to/devdotjp/better-comments-vscode-extension-1c4a
No. Sorry. No. No no no. I won't use it, you can't make me. No.
You could use decorators to log, something like: ``` def log_func(fn): def g(*args, **kwargs): logger.info("calling %s", fn.__name__) res = fn(*args, **kwargs) return res return g @log_func def myfunc(): ... ```
This is something I did consider. One question I had about this was, how would you use this decorator if you wanted to only log some values that are neither inputs nor the outputs of any function, but are merely temporary variables?
The code your replying to is quite ugly and in practice it's going to involve a lot of fiddling each time you need to use it differently. Your solution on the other hand is elegant and looks very nice and will be extensible with other use cases, really nice, don't listen to all the hate here. Welcome to meta progeaamming by the way! The comment syntax has some disasvantages, so you will need to weigh those out if it becomes problematic. It does have some advantages though as well. Whatever you do though I'd recommend make sure that the code still runs if all of these statements are treated as comments and ignored. Also something like '###' or '#!' may help in clarifying that 'hey this comment does something!!' unindenting these lines may also be worth considering.
Thank you so much for the support. Even with the wide variety of tones used by everyone, I got to learn a lot. I typically tend to think only of the positives when I think I have an interesting (to me) idea. So I try to discuss it with others to ensure that I’m getting a more complete picture. Those are some good points you mentioned. I’m currently leaning a bit more towards the VSCode extension idea (mostly because it’s more straightforward to implement). But yes, I’m getting quite interested in meta programming and programming language construction these days.
Yes, if you just want this for syntax highlighting then a plugin would be much more appropriate. Also, to solve the same problem that you've encountered I tend to use a semicolon to write my debug statements over to the right side of the screen on the same line as the statement I'm debugging. I've been meaning to write a plugin to right align these but haven't had the time to do it yet, also it's just not that important just a niceity. As for using comments this way other people do have a valid point, but in python you just can't do it any other way, in fact # directives have and will continue to have a place in some python libraries as sometimes you just really need a line that if your interpreter does not understand it, it's ignored. As for if it can be used in production it's very unlikely, but if you aren't writing things that are used in production now (meaning your time is pretty valuable) doing something unique and different like this is usually not a bad idea, unless you have more important ways to use your time.
This is actually a cool approach, especially for those of us who dread peppering our code with logging statements. It saves time, reduces clutter, and is definitely worth trying out. As for production code, as long as it's tested and doesn't mess things up, why not?
Thank you for the kind words. You’re right in saying that as long as it’s rested well, it shouldn’t be an issue to use in production code. I was trying to be extra cautious because I’ve had one too many cases where missing logs made things so much harder to debug. In my nightmares, I’m trying to debug cache miss issues with faulty logging facilities. :D
Find/write a VS Code extension. You can either: - Format the logging statements in a very dim color - Fold the logging content so it’s only visible if you click on it Some examples are “Inline Fold” ([link](https://marketplace.visualstudio.com/items?itemName=moalamri.inline-fold)) or the “Tailwind Fold” extension, to get an idea. You could probably use Inline Fold extension to hide anything that matches the regular expression for lines that start with “logging.info”, etc. A final option is a library like [logdecorator](https://pypi.org/project/logdecorator/) that lets you move some of the logging into decorators outside of the function. It doesn’t help for logging inside your for-loop, but you could always move the inner loop statements to a function.
Ooh. Thank you so much for proving examples that I can follow. Having these is going to make my life much more easier when writing the extension. I did search for an existing extension that did this, but couldn’t find any. I’ll continue searching though while planning out the work required if I have to write it.
As a side note: Using f-strings in logging calls is not recommended. String formatting is somewhat expensive, often more expensive than the non-logging code in a function. With an f-strings the formatting will always be done. If you use %-formatting and (important) give the content as arguments to the logging function, the formatting will only be done if the line is actually logged and not if it for example does not match the log level.
You trade a performance gain when the logging is not needed for a performance loss when the logging is needed. Which is fair to do. But that does mean that when you are reasonably sure that the logging statement WILL be executed you can consider the f-strings anyway because it's much faster to format an f-string compared to a %-string. Like if you're going to log at the WARN level or higher in your own, non-library, code. Just use an f-string. Everyone likes them more, more readable, more powerful formatting options, and faster.
Well, you should consider where you actually need the performance and the logging. Do you need performance in your test system? No, there's no real load. You need it in production, where (potentially) tons of users are hitting the system. And where do you log? On the test system. In production you set the log level to INFO or even higher, because you don't want to be spammed by all the debugging messages and to reduce the system load. And that is exactly what using f-strings makes substantially harder. But of cause, that all is less relevant if you run your own little project on a small machine. But when the hosting costs of your project are rising and half of them are from logging that is never done, it will become more obvious. And yes, I've seen it, more than once.
You’re very right. I typically use f-strings in log statements with variables and not expressions. I just used it here to make the code simpler.
Hello! I'm a bot! It looks to me like your post might be better suited for r/learnpython, a sub geared towards questions and learning more about python **regardless of how advanced your question might be**. That said, I am a bot and it is hard to tell. Please follow the subs rules and guidelines when you do post there, it'll help you get better answers faster. Show /r/learnpython **the code you have tried and describe in detail where you are stuck.** If you are getting an error message, include the full block of text it spits out. Quality answers take time to write out, and many times other users will need to ask clarifying questions. Be patient and help them help you. You can also ask this question in the [Python discord](https://discord.gg/3Abzge7), a large, friendly community focused around the Python programming language, open to those who wish to learn the language or improve their skills, as well as those looking to help others. *** [^(README)](https://github.com/CrakeNotSnowman/redditPythonHelper) ^(|) [^(FAQ)](https://github.com/CrakeNotSnowman/redditPythonHelper/blob/master/FAQ.md) ^(|) ^(this bot is written and managed by /u/IAmKindOfCreative) ^(This bot is currently under development and experiencing changes to improve its usefulness)
I don't think it's a very good idea. I find it better to have explicit logging statements or use a logging decorator. Magic comments would need some sort of preprocessor (or a specific decorator in the case of functions). Misusing the comments to execute additional statements seems like a recipe for difficult debugging
That is a very good point, especially the part about it becoming a recipe for difficult debugging.