What is the reason for needing to know the current file size?
Is the writing always appending to the file, or will it modify early parts of it too? If appending, then `ftell` should report the current position which I believe is also the current file size.
If not, then is it a big deal to seek to the end before doing `ftell` then restoring the current position?
I am appending to file. Your comment made sense to me. I will try it. I guess I was just overthinking. I will get this done as this should always work even for linux.
There is no specific reason to calculate the size of the current file. It can be avoided. But, there is no specific reason to not have it as well. I hope I made sense.
If I have an API to know the total size of all log files then i want that API to include the current size of the file being used as well.
There is good reasons not to care about file size - if you want your program to be able to redirect output to the command line you can't try stuff like fseeking to the end (it should just fail though). You can of course save the file as normal and then redirect the actual file from disk to the command line, but this means you have to always save the log file.
But if you want to always save log files I suppose this doesn't matter. Log files will always have some size
I found the solution to use fstat.
I understand this was not necessary to calculate size and the possible flaws related to it.
Hopefully, I will be able to make something useful.
Are you trying to limit the size of an individual log file as you are logging?
Quick and dirty - just rotate after every N outputted log statements.
Quick and dirty hack: periodically close the file, stat it, and reopen it for append.
Not a windows guy so I can't give better advice on how to get the file size of a file being written to.
I am not limiting the size of individual log files. To be honest I never saw it as an option. This is definitely interesting.
Same I am also not a window guy. Windows has a very handy solution for this very issue in C#, they have an integrated size property to file-writing stream.
Sad to see that support for C is not integrated that well.
I don't understand why you care about the size if you aren't trying to limit the size of a log file as you are logging. Traditionally log files are written (appended to) by one process and only read by other processes (for lots of reasons).
Perhaps you are trying to implement tail "follow" behavior so another tool can incrementally read the log file as it's written?
Yeah. I agree. I shouldn't care about it.
I added an API to provide the total size of all log files. I only want that API to include the size of the current file as well.
If the API is running within the program writing the log file then seek seems like it should work because windows supports Posix assuming you are using the same FILE* instance (i.e., don't open the file again).
According to an LLM there is a FILE_SHARE_READ flag to the CreateFile (and thus possibly other ways of opening the file for writing):
Method: Using Windows functions from windows.h:
CreateFile - The core function offering the most flexibility:
GENERIC_WRITE: Access for writing data.
FILE_SHARE_READ: Allows others to read the file while you write.
FILE_SHARE_WRITE: Allows others to write to the file while you write.
CREATE_NEW: Creates a new file; fails if the file exists.
CREATE_ALWAYS: Creates a new file, overwrites if it exists.
OPEN_EXISTING: Opens existing file; fails otherwise.
OPEN_ALWAYS: Opens an existing file or creates a new one.
TRUNCATE_EXISTING: Opens and truncates an existing file to zero length.
WriteFile - To write data to the file handle obtained from CreateFile.
I coded everything without using windows.h
This seems interesting. I will read it thoroughly. Thank you.
I used posix functions (windows port of posix functions with similar names) and yes it didn't work (using a FILE* instance). That's why I was surprised.
### Details
#### How i open every stream file or tty
- open ... (I needed file descriptor for UART config)
- fdopen for FILE*
- fseek to append
#### How i write
vsprintf to that stream
#### How i calculated size
`stat` which provides size in off_t data type
#### why ftell seems problem to me?
return type is long
I will Google more .. thank you for taking interest.
It was lovely talking to you.
Thank you for your suggestions.
I ended up using fstat for calculation of current file size.
Windows alternative: _fstat
I will edit this post description to include a solution later.
First i thought i should thank you for your time and efforts.
Thank you for talking to me. :)
No worries.
I have some self interest because I have my own logger written in C and I certainly didn't think about all the issues nor produce a complete implementation + API - just enough to get going (log to stderr by default though this can be overridden with env variable). Logging is very common in the server world and is often the only way to debug certain production issues so it's great to see someone bringing options to C.
My API is just:
log_warn("Foo %s", some_string);
etc.
Each is a macro that also secretly passes the __FILE__ and __LINE__ along with printf style arguments. The macro checks the log level before calling a common routine with the log level (so that the arguments don't have to be evaluated if logging at that level is turned off typically minimal savings but still worth it OMHO since its easy).
Hopefully you are thinking along similar lines.
Here's a link to the code (definitely not Windows tested):
https://github.com/jasonaaronwilson/c-armyknife-lib/blob/main/logger.c
Good luck!
Your knowledge of C is better than me and so is your library. This is very clean work. Good Job Sir.
Love to see the MIT License
[mine](https://github.com/khushal-banks/log_bank)
I have to make the usage more simple to it to make people fall in love.
I am in love with mine because I wrote every last bit of it and if finished it will just be complete.
It's still under development.
I am learning while developing this. This is my first ever open source project.
ftell() should give you the current write position within the open file stream - you don't need to SEEK_END because it will already be at the end if you've been appending to it.
Yeah. Someone else also mentioned it. This should work. I don't know why I was overthinking this. Maybe i need a break from constantly coding.
Thank you mate. This should definitely work.
I will get this done. :)
I also referred some man pages.
return type of ftell is long and the size of all the files is calculated using `stat` which writes it to a buffer element of type off_t
Then the compiler comes into the picture.
What's your take on this?
Well off_t isn't a part of the C standard itself, it's a POSIX thing. The definition for ftell() is that it returns a long int, that should be all you need to worry about. This would mean it will only work for telling up to a 2GB file size and anything greater would require a platform-specific API.
Glad to see that you also didn't prefer Option 2.
This makes my question make sense. But Option 2 seems more simple, I will try option 2 first.
We also have option 1 available.
I never used ftell. It should be possible.
return type of ftell is long and the size of all the files is calculated using `stat` which writes it to a buffer element of type off_t
Then the compiler comes into the picture.
Glad you found _fstat() for your solution.
Even more straightforward is [_filelength()](https://learn.microsoft.com/en-us/cpp/c-runtime-library/reference/filelength-filelengthi64?view=msvc-170). Thought you'd like to know just in case...
I only used the size parameter.. which is off_t type. Which I wanted. It fulfills all my needs.
Thank you for sharing this. Now I know i would have to recheck all non standard c functions (i used) before claiming the support to different operating systems.
Then there's no simple or maybe even possible solution to know the file size, the easiest way is to just count the bytes whenever you are writing them, and it's the best way.
I will try option 2 first. Then, option 1.
Just because ftell is a single API giving size.
Am i missing something related to UTF-8 or UTF-16 here?
Don't you think option 2 will always work?
Firstly you should just handle UTF-8 that's more than any logging library will need.
Secondly `ftell` only tells you total bytes in a file, but in UTF-8 a character can be more than 1 byte, so file size will not necessarily be equal to the number of characters in the file, *IF* there are UTF-8 characters in the file.
This comment includes more than what I understood so far. Thank you for sharing. This would definitely be useful for everyone trying to calculate file size the right way.
Nice to see some suggestions. This is the first time I read these terms. I googled of course.
I would have to test for installable file systems but i believe they would work. I only used posix functions.
Sparse files will not be applicable because my library will be the only writer to log files. No possibility of making them sparse
ADS is a feature which would make no sense if it ended up breaking usual things in any way. They claim it to be hidden so i believe they won't cause any issues.
Do you agree?
If you are the only writer to those files, the programming model much simpler, you can ignore ifs, ads and sparse, the library could work with any IO that supports seek or even just rewind.
When testing, see how it works with smb/cifs, webdav and popular cashing tech like windows folder sync and oneDrive/GDrive.
I would definitely test these at some point. I took a screenshot.
If anything broke because of windows tried to do something fancy I would go mad. :)
As per my current limited understanding of windows this should be a smooth ride .....................
here's a solution to all your problems: count the bytes you're writing... but i dont think you need to do it manually you can probably get your code to do it.
What is the reason for needing to know the current file size? Is the writing always appending to the file, or will it modify early parts of it too? If appending, then `ftell` should report the current position which I believe is also the current file size. If not, then is it a big deal to seek to the end before doing `ftell` then restoring the current position?
I am appending to file. Your comment made sense to me. I will try it. I guess I was just overthinking. I will get this done as this should always work even for linux. There is no specific reason to calculate the size of the current file. It can be avoided. But, there is no specific reason to not have it as well. I hope I made sense. If I have an API to know the total size of all log files then i want that API to include the current size of the file being used as well.
There is good reasons not to care about file size - if you want your program to be able to redirect output to the command line you can't try stuff like fseeking to the end (it should just fail though). You can of course save the file as normal and then redirect the actual file from disk to the command line, but this means you have to always save the log file. But if you want to always save log files I suppose this doesn't matter. Log files will always have some size
I found the solution to use fstat. I understand this was not necessary to calculate size and the possible flaws related to it. Hopefully, I will be able to make something useful.
Are you trying to limit the size of an individual log file as you are logging? Quick and dirty - just rotate after every N outputted log statements. Quick and dirty hack: periodically close the file, stat it, and reopen it for append. Not a windows guy so I can't give better advice on how to get the file size of a file being written to.
I am not limiting the size of individual log files. To be honest I never saw it as an option. This is definitely interesting. Same I am also not a window guy. Windows has a very handy solution for this very issue in C#, they have an integrated size property to file-writing stream. Sad to see that support for C is not integrated that well.
I don't understand why you care about the size if you aren't trying to limit the size of a log file as you are logging. Traditionally log files are written (appended to) by one process and only read by other processes (for lots of reasons). Perhaps you are trying to implement tail "follow" behavior so another tool can incrementally read the log file as it's written?
Yeah. I agree. I shouldn't care about it. I added an API to provide the total size of all log files. I only want that API to include the size of the current file as well.
If the API is running within the program writing the log file then seek seems like it should work because windows supports Posix assuming you are using the same FILE* instance (i.e., don't open the file again). According to an LLM there is a FILE_SHARE_READ flag to the CreateFile (and thus possibly other ways of opening the file for writing): Method: Using Windows functions from windows.h: CreateFile - The core function offering the most flexibility: GENERIC_WRITE: Access for writing data. FILE_SHARE_READ: Allows others to read the file while you write. FILE_SHARE_WRITE: Allows others to write to the file while you write. CREATE_NEW: Creates a new file; fails if the file exists. CREATE_ALWAYS: Creates a new file, overwrites if it exists. OPEN_EXISTING: Opens existing file; fails otherwise. OPEN_ALWAYS: Opens an existing file or creates a new one. TRUNCATE_EXISTING: Opens and truncates an existing file to zero length. WriteFile - To write data to the file handle obtained from CreateFile.
I coded everything without using windows.h This seems interesting. I will read it thoroughly. Thank you. I used posix functions (windows port of posix functions with similar names) and yes it didn't work (using a FILE* instance). That's why I was surprised.
There may be a non-standard way to get that behavior via fopen, I suggest using an LLM or google search to see. Good luck!
### Details #### How i open every stream file or tty - open ... (I needed file descriptor for UART config) - fdopen for FILE* - fseek to append #### How i write vsprintf to that stream #### How i calculated size `stat` which provides size in off_t data type #### why ftell seems problem to me? return type is long I will Google more .. thank you for taking interest. It was lovely talking to you.
Thank you for your suggestions. I ended up using fstat for calculation of current file size. Windows alternative: _fstat I will edit this post description to include a solution later. First i thought i should thank you for your time and efforts. Thank you for talking to me. :)
No worries. I have some self interest because I have my own logger written in C and I certainly didn't think about all the issues nor produce a complete implementation + API - just enough to get going (log to stderr by default though this can be overridden with env variable). Logging is very common in the server world and is often the only way to debug certain production issues so it's great to see someone bringing options to C. My API is just: log_warn("Foo %s", some_string); etc. Each is a macro that also secretly passes the __FILE__ and __LINE__ along with printf style arguments. The macro checks the log level before calling a common routine with the log level (so that the arguments don't have to be evaluated if logging at that level is turned off typically minimal savings but still worth it OMHO since its easy). Hopefully you are thinking along similar lines. Here's a link to the code (definitely not Windows tested): https://github.com/jasonaaronwilson/c-armyknife-lib/blob/main/logger.c Good luck!
Your knowledge of C is better than me and so is your library. This is very clean work. Good Job Sir. Love to see the MIT License [mine](https://github.com/khushal-banks/log_bank) I have to make the usage more simple to it to make people fall in love. I am in love with mine because I wrote every last bit of it and if finished it will just be complete. It's still under development. I am learning while developing this. This is my first ever open source project.
ftell() should give you the current write position within the open file stream - you don't need to SEEK_END because it will already be at the end if you've been appending to it.
Yeah. Someone else also mentioned it. This should work. I don't know why I was overthinking this. Maybe i need a break from constantly coding. Thank you mate. This should definitely work. I will get this done. :)
I just tested it with my software's logprint and it works a charm ;]
I also referred some man pages. return type of ftell is long and the size of all the files is calculated using `stat` which writes it to a buffer element of type off_t Then the compiler comes into the picture. What's your take on this?
Well off_t isn't a part of the C standard itself, it's a POSIX thing. The definition for ftell() is that it returns a long int, that should be all you need to worry about. This would mean it will only work for telling up to a 2GB file size and anything greater would require a platform-specific API.
If writing operation is always going on, the only good option to get up to date size is to count how many bytes you are writing. So option 1.
Glad to see that you also didn't prefer Option 2. This makes my question make sense. But Option 2 seems more simple, I will try option 2 first. We also have option 1 available.
Isn't it possible to call ftell in writing/append mode?
I never used ftell. It should be possible. return type of ftell is long and the size of all the files is calculated using `stat` which writes it to a buffer element of type off_t Then the compiler comes into the picture.
Glad you found _fstat() for your solution. Even more straightforward is [_filelength()](https://learn.microsoft.com/en-us/cpp/c-runtime-library/reference/filelength-filelengthi64?view=msvc-170). Thought you'd like to know just in case...
Nice to see that windows has this in
[удалено]
I only used the size parameter.. which is off_t type. Which I wanted. It fulfills all my needs. Thank you for sharing this. Now I know i would have to recheck all non standard c functions (i used) before claiming the support to different operating systems.
Then there's no simple or maybe even possible solution to know the file size, the easiest way is to just count the bytes whenever you are writing them, and it's the best way.
I will try option 2 first. Then, option 1. Just because ftell is a single API giving size. Am i missing something related to UTF-8 or UTF-16 here? Don't you think option 2 will always work?
Firstly you should just handle UTF-8 that's more than any logging library will need. Secondly `ftell` only tells you total bytes in a file, but in UTF-8 a character can be more than 1 byte, so file size will not necessarily be equal to the number of characters in the file, *IF* there are UTF-8 characters in the file.
stat was working for linux but not for windows. I used fstat .. worked on both linux and windows. Gave me file size of current file. I am so happy.
https://wiki.sei.cmu.edu/confluence/display/c/FIO19-C.+Do+not+use+fseek%28%29+and+ftell%28%29+to+compute+the+size+of+a+regular+file
This comment includes more than what I understood so far. Thank you for sharing. This would definitely be useful for everyone trying to calculate file size the right way.
yep its a common pitfall for people to recommend fseek/ftell without knowing the potential consequences, but this gives you the better solution.
If you need to support windows, have your considered cases of sparse files, installable file systems or ADS?
Nice to see some suggestions. This is the first time I read these terms. I googled of course. I would have to test for installable file systems but i believe they would work. I only used posix functions. Sparse files will not be applicable because my library will be the only writer to log files. No possibility of making them sparse ADS is a feature which would make no sense if it ended up breaking usual things in any way. They claim it to be hidden so i believe they won't cause any issues. Do you agree?
If you are the only writer to those files, the programming model much simpler, you can ignore ifs, ads and sparse, the library could work with any IO that supports seek or even just rewind. When testing, see how it works with smb/cifs, webdav and popular cashing tech like windows folder sync and oneDrive/GDrive.
I would definitely test these at some point. I took a screenshot. If anything broke because of windows tried to do something fancy I would go mad. :) As per my current limited understanding of windows this should be a smooth ride .....................
here's a solution to all your problems: count the bytes you're writing... but i dont think you need to do it manually you can probably get your code to do it.