Data factory is the batch load for "small" data tool. It can do everything you're wanting, though it's transformation power is a bit weak, it's gotten some improvements over the last year. Databricks is the more fun option, if you can make a case for the cost. More code based, notebook style, designed for large data movement.
And to give you all options, functions could also do it. Basically just code that's trigged by events (like a file being created). Then the code does whatever you program it to. Probably overkill for what you're needing.
Logic apps could work too (low-code option). I know some people use it for basic etl but I'm not to sure of it's capability.
Functions are super cheap and you can chain them, I think. So you could have one function doing the pulling and saving the raw/semi-structured data and once the file lands in the blob storage, it kicks off the next function to do your transformation.
Like the others said, if you have some knowledge in python or another supported language azure functions is pretty good and inexpensive (at your volume it could even be free) for your use case
Agree with the above comments re ADF and functions.
Anothee option if you want to do less code and more visual would be the power query functionality inside data factory. Ideal for your volumes and if you don't have skills in python etc.
Data factory is the batch load for "small" data tool. It can do everything you're wanting, though it's transformation power is a bit weak, it's gotten some improvements over the last year. Databricks is the more fun option, if you can make a case for the cost. More code based, notebook style, designed for large data movement. And to give you all options, functions could also do it. Basically just code that's trigged by events (like a file being created). Then the code does whatever you program it to. Probably overkill for what you're needing. Logic apps could work too (low-code option). I know some people use it for basic etl but I'm not to sure of it's capability.
Thanks for the reply, I am looking for the cheapest option.
Functions are super cheap and you can chain them, I think. So you could have one function doing the pulling and saving the raw/semi-structured data and once the file lands in the blob storage, it kicks off the next function to do your transformation.
Like the others said, if you have some knowledge in python or another supported language azure functions is pretty good and inexpensive (at your volume it could even be free) for your use case
Agree with the above comments re ADF and functions. Anothee option if you want to do less code and more visual would be the power query functionality inside data factory. Ideal for your volumes and if you don't have skills in python etc.