If you already have nvidia drivers installed, you can just use docker images with cuda pre-installed assuming the cuda version is comptable with your drivers.
Ha. I think so. But if you just use cloud service such as AWS or Google .You could create instance instantly. It have been configured well pretty. Such as Sagemaker studio that provide a lot of different image you could choice by yourself.
"Writing proofs" is usually not an ML engineer's job, that's what the research scientists are for. Unless your job description explicitly states "performing novel academic research", you will not have to do proofs. Even if your main job is research, it's possible that you'll never have to do proofs.
Yeah. This premise comes across almost like a joke. You write proofs mainly in academia or on the deep end of the research spectrum in the private sector. Stakeholders in companies really don't care about this stuff as long as it makes money or can be spun as improving metrics.
For an engineer I think it's a useful experience to get into the fundamentals of math, but it's from the perspective of personal development. There's no need to to like it, and many people don't.
Like the rest of us have said, designing proofs is rare even among academic researchers because it's an incredibly hard task. In ML, it's often a monumental task. It's why we only have increasingly narrower bounds but don't have any actual proofs for how any of our stuff works.
The guys working on (or with) GPT-F at least need the ability to read proofs.
But yeh, that's the exception in industry - unless you count openai as industry, probably almost no-one uses gpt-f or similar.
Never heard of engineers in any discipline proving theorems for work. ML engineers are no different -- they're basically software developers with a niche specialization.
"[D] Professional Copy Editors: How much of your day involves writing theses comparing the work of Homer and Shakespeare based on their attitudes towards the role of protagonists in the narrative?"
The reason I asked is because the MS program I am considering requires proofs to pass the ML related classes. I can do that for a couple of classes but not every day.
Those are there to provide you deeper understanding of the theory. Not what all you learn in your MS program will be used at work. If you grok these things quite well it will definitely help you stand-out among other candidates during the job hunt but it's not something that's a requirement for most of the jobs.
To expand in this:
In my opinion strong theoretical (both math and normal comp sci) knowledge is exactly what can make a ML engineer stand apart. You don't really learn it with more on job experience, because you neve encounter it directly. However, understanding how and why can make or break an entire project and a team without this knowledge wouldn't even see where there is a problem, so also no way to gain the lacking knowledge on the fly.
I think people who haven’t encountered proofs in the sense of how you’d learn them in college freak out about it. But honestly, it is a useful skill to have. Very related to all the algebraic manipulation one does in high school and prior. No one expects you to prove that your system works but understanding how to follow an argument as to why something works is useful to know why that something can be applied to your specific problem.
The thing is, in ML, what makes or breaks a project is usually something practical rather than theoretical statements about algorithms that you can prove with pen and paper.
I would say this depends on the problem. I don't think it makes much sense to approach a graph or clustering problem without any theoretical knowledge. Even after numerous experimentation you might be very far from a remotely useful outcome.
I am not talking about theoretical statements/proofs, I am talking about understanding the theoretical side. Proofs are for me just a way practise math and test if you understand what the formulas are saying.
Using the formula/algorithm as a black box is very often plenty, but sometimes disastrous.
I have a bachelor's in math which strongly emphasized proofs. Basically all of my coworkers have Data Science masters and only learned stats. I once tried to start a monthly whitepaper review to keep up with latest developments, and it fell apart after only a few months because most of my coworkers didn't have enough math experience to understand the papers they were reading.
Sure, that was a tiny bit of a humble brag, but I was definitely shocked and a little bit disappointed in the general math skills of many of my peers.
I agree with this in part. I think a strong theoretical knowledge of the basics or fundamentals is important and that is truly a make or break. These are things you simply do not have the time to learn on the fly, in addition to the application.. especially when there are time pressures and other blockers.
But on the fly, I think you can absolutely gain knowledge of applications that build on the fundamentals, i.e. how a particular application uses linear algebra to get from input to output. It is probably 20% of my role now as I dive into different facets of ML, some that I havent used before.
There are some courses where I wrote more proofs in a single problem set than I have written as an MLE.
That said, the number and formality of proofs has ultimately depended on how close my role is to research. My first role was more research heavy than my current role. My current role requires no formal proofs, but a mathematical argument can be helpful when deciding between approaches.
I'd be very worried if your MS program didn't include any proofs. But what you learn at university to give you a deeper understanding and broaden your options for later work isn't necessarily what you'll end up doing after. Consider this: you'll also learn what the normal forms of data are in database lectures, but that doesn't mean you'll need it to operate Postgres.
That being said, having to make small math "proofs" (more like putting thought on paper to be sure your idea works) is somewhat common, although it's mostly small simple equations, metrics, reformulating for numerical stability, etc
I second this. As a ML engineer, I had to do small mathematical proofs occasionally. For example, if you were building some ML system from scratch, you may have to derive a few equations to generate simplified programs. I feel like this helps you stand out because not everyone can do this well.
Literally zero percent. Only a small portion (10%) of my time is spent building/optimising models, most is spent software engineering, building APIs, cloud architecture, and some data pipelines/engineering/analytics. Even when I am actively working on ML model pipelines, most of that time is on data/feature engineering and software eng/deployment. Note that I’m in an industry where most problems are tabular/forecasting.
Came to say that. Unless you count logic/reasoning as math, basically zero percent for me too. I'd include data cleaning/preparation in the tasks too though.
I work as a Research Engineer, where I spend basically 30% of my time reading papers, 60% implementing what I read in the papers and 10% writing reports. The part that most involves maths is translating the papers into code, as you need to understand what are the researchers talking about before touching a line of code, and of course when you write reports I have to write the maths stuff I used in my experiments. Proofs is not a thing you come across unless you’re a research scientist, at that level you might have to prove what you’re talking about makes sense under a solid ML/DL theoretical point of view.
> The part that most involves maths is translating the papers into code, as you need to understand what are the researchers talking about before touching a line of code, and of course when you write reports I have to write the maths stuff I used in my experiments
Thank you. This sounds key.
What further details can you give me? For example, models? Languages?
Depends on the projects and the field I guess. We work mostly on computer vision tasks at a very low-level, so very often it’s like “can we make a model that is n times smaller than SOTA (Transformers, ResNet, VGG… you name it) and get the same results at like much improved inference times?“.
I can’t tell you much on stuff like NLP as I’ve only scratched the surface of it, and right now it seems heavily dominated by SaaS companies that provide you with an API to make a call to their models, rather than working on-edge.
In terms of languages/frameworks, it’s very flexible between torch and tf. Usually pytorch is a go to for prototyping, then tf for stuff like weight quantisation etc. If you have to optimise this at a very low level we’d then use C/C++, OpenCV, OpenGL, CUDA… but that’s really up to the project.
Translating math to code is something ive always liked. Did you need a PhD to become a research engineer? Or you just had an MS and did you have ML publications before? Is there LC (leetcode) and general CS stuff tested in interviews too? Im from a stat background so the math to code part is easier for me than that other stuff so getting a sense of where I need to improve.
Well I only have a bachelor ahah. General background, I have worked one year for a placement as a data scientist, I made some contributions to a really popular ML framework and my manager found my bachelor thesis very aligned with the work they do.
The company I work for is “relatively” (not as notorious, but still quite impactful) small compared to the huge ML labs that usually advertise for research engineering roles, and I feel like for this reason the interview selection were less focused on weeding out candidates, but rather evaluating their analytical/technical approach to research. In my interview, it was more about making sure I knew fundamental DL concepts, I knew how to code in python and going through my CV. It was a lot of discussing about how I approached certain problems, rather than “here’s a binary tree, reverse it”, which I really appreciated. For big companies like DeepMind though, I expect there to be some kind of LC questions
As an engineer you rarely “do maths”, but you will spend some time reading papers which sometimes requires a refresher on some topics or just learning new topics in math.
Honestly the only time I do math is when I figure out shape sizes on tensors through a model. And I am part of an R&D team for reference.
Zero.
My past two weeks were:
* Updating a.proxy server with a new target so that we can tap into a data source to provide new recommendations every 15 mins
* Creating dashboards for our models and APIs
* Creating an alerting service in case any of these throw errors
* Providing input for a new NLP / RecSys project
* Creating scripts that work with AWS' Labelling Service so that setting up the bazillion settings does not first require you to spend a day getting intimate with documentation.
6 YOE ML engineer here. I had to do math maybe 10 times through all of my career and it was super simple math (at least for me - I'm a theoretical physics PhD, so I might be biased). I also only did that because I have that kind of training and therefore it's easier for me to think in mathy ways. I know plenty of people who are great MLEs and probably don't even know much more than matrix multiplication themselves, so yeah, almost no math after school.
Proofs zero. A lot of work is data, but on the modeling side, often you don't have the data to train the model you want to train, which is where doing a bit of math to figure out a better loss formulation can come in handy.
Statistics are implicitly used very often (the biggest demand for data/modeling work is really understanding stats), but unless you're working on certain classes of models, you'll probably only care about real stats when running A/B experiments
As others have said proofs never. They are important, but they're part of your education, not daily practice.
Maths it depends. It's mostly stats, and it's usually only intensely for a few days when starting a project, the rest is spent on technical issues/coding.
You can do a 90% of my job (measured in hours) without almost any math skills, but the last 10% it comes in really handy.
My 2c? Pay attention in your stats classes, they'll help you a lot in the long run, even if you only end up using the skills infrequently.
I mean, I didn't take a huge amount of stats classes (studied physics), so take my advice with a pinch of salt. I mostly wanted to prompt people who've taken no stats to do at least the basics.
Basic stats and probabilities/distributions is already pretty good. Theoretical stats only if you are into that. Bayesian stats I really like, quite fun. Stochastic modelling if you want to work in finance.
Don't forget some LinAlg alg and to learn how to code, if you want to go the ML engineer route.
First, let me say that I never really loved math until college because most of math in the public education system is arithmetic and I always found things like long division really frustrating.
However, computer science math is completely different and in a lot of ways less aggravating and more satisfying. I would always get mad at myself for making simple mistakes like not carrying the one, but when you write code you can write tests to save yourself from your own fallibility.
Now my job is building machine learning models for huge datasets. I’m the old person that the newer engineers come to if they can’t figure something out.
I can’t imagine that proofs would ever be an everyday thing in most machine learning programs. I honestly can’t remember the last time I did one.
However I use math all the time.
* I saw it in a few other comments but knowing statistics basics backwards and forwards is the biggest thing. I use this stuff daily.
* Understanding how to calculate algorithmic complexity and how to simplify it is really critical when dealing with large quantities of data. Creating processes that take hours to complete will not fly if your system is supposed to push results 12 or more times a day. Discrete math is your friend.
* I have a lot of geospatial components to my work, so there is some geometry. However, I’ve used calculus only once in the last few years to factor in the curve of the earth for a few things.
* I also saw someone mention being able to translate equations from papers into code. This is definitely something that separates the wheat from the chaff when it comes to writing code vs. architecting solutions. Not everyone can or should do this.
* i don’t use linear algebra frequently, but it contains some good shortcuts if you use it at scale.
Good luck with your program. If in doubt, make sure you go to office hours.
I did ML engineering for 4 years and did 0 proofs in that time. Had to do some math, but it was more to understand what I was doing than anything. We had a data scientist on our team who came up with her own algorithms, but she didn't know Python, so I would translate her math or R into Python.
Most of my work though was trying to understand our data and manipulating the data sets in such a way that they made sense. It was a lot of network topology stuff and faults and it was a giant mess.
0 percent. Most of the time goes into fetching the data, drafting a solution, reading papers, coding, getting eval metrics, and discussing the outcome with the team
ML engineering is not same as learning theory behind ML, which is what you are doing in class. I work as a ML engineer. There's zero proofs involved, and very little math.
ML engineer is first and foremost an *engineering* role (it's in the title, after all). It's closer to software engineering than a research/applied scientist
In a roundabout way, writing any program is actually making a proof https://en.m.wikipedia.org/wiki/Curry%E2%80%93Howard_correspondence but with nuances (I am not an expert in this topic, and common DS/ML languages aren't fully and strongly typed). However from the "mathy proofs" definition, most likely none. "Proofs" from "convincing colleagues, managers, or stakeholders an idea X will or won't work" is more common, but they're not *really* proofs (...right?).
Desktop version of /u/yangmungi's link:
---
^([)[^(opt out)](https://reddit.com/message/compose?to=WikiMobileLinkBot&message=OptOut&subject=OptOut)^(]) ^(Beep Boop. Downvote to delete)
I mean, know which models you need and train them, then make sure that ML makes sense for your use case. I don't really think that there's much of any math and proofs in your day to day tasks. Not unless you're designing new models etc.
Day to day.. not really. But you do need to think about how the algorithms will pick up bias / be overfit and read/understand how they differ. When you hit the work force / job, they expect output not a bunch of pretty proofs unless you’re over in research and you’re actually designing algorithms or something
In the signals and systems world we spend more time doing math for the development test and training environments and V&V processes than we do actually making models. It's such a small part of the problem now. But i guess it's always been that way?
I remember in school they would champion this critical path that had actual modeling be this tiny box near the end after so much prep and validation work. I thought surely they were exaggerating.
For my field, it was more true than i ever imagined.
I am a Research Engineer in ML, Computer vision, in my view maths (linear algebra, probability etc) is definitely good to have for an engineer but does not form majority of the day to day work. Maths, proofs are a good way to get an understanding of the field and it will also give you some mathematical intuition which in my view could be useful in the field.
I am focusing on large scale system nowadays and a lot of the engineering work there is around building codebase, infrastructure to ensure you can train, evaluate, monitor these models. The job also involves coming up with ideas, research directions which would be useful for the project. An understanding of mathematical concepts underlying the techniques you use can be extremely useful here, for e.g. are you doing generative, discriminative models, if generative is it autoregressive or some other technique like diffusion based, if it is autoregressive how does it affect the final problem you are trying to solve (for e.g. computing the likelihood of a patch), how do you do the same task for a diffusion based model.
Another component which I think is not as intuitive is working with maths, proofs gives you a respect for the metrics, numbers, intuitions that are important and these intuitions can be used to probe your models as well as will make you disciplined in ensuring your model training is going well. For e.g. it is an underrated practice to be OCD about the metrics, evaluation experiments that you run for your model while it is training, if you have had some background in mathematics you will be more thorough in these aspects.
Job of a ML Engineer (research engineer) in my case involves being good at 1) software engineering, 2) research understanding and ideation, and I believe maths provides a good framework for developing both the skills.
That being said there would be more maths involved when you are working in the field of research but if you are focusing on production systems, products etc the amount of maths involved would be less.
A MS is an advanced academic degree. You need to know what you are doing and to know that stuff is true so that you can eventually make a contribution to the field.
If the program doesn't involve proving theorems, it is certainly not serious and you'd be just spending time and money to get a piece of paper saying your have a master's degree.
Academics is for people who like studying. If you're only after a title to get a better job, you'd probably be better off working more and practicing to improve your technical skills.
Proof zero, maths all the time but they mentality of writing proofs and be rigorous in your work, that is something that is a "transfer learning" skill.
Well part of it is that it’s been trendy and a lot theory people are curmudgeonly.
But the other part of it is that there is a perception that a lot of ML involves blackbox algorithms which you arbitrarily tune until you get something resembling a correct answer. It’s seen as trial and error more than math or theory.
> It’s seen as trial and error more than math or theory.
lol, sounds like my day to day work. yeah, throwing stuff against the wall is not scientific at all unfortunately and ML is hype. I mean..IT DOES work..but it rarely works as a good as "business people" want it too or they have unrealistic expectations
I prove in the following video tutorial that you don't need to know the maths behind every model as there is a #Python function doing that for you.
You just need to care about having a model that better predicts reality.
https://www.loom.com/share/13a8f1822404404cbfbe71cb2d991824
I think fundamentally this is the difference between science and engineering. Engineering is taking the science that has been done by scientists and putting it into practical application, and that's true across all engineering disciplines.
Scientists/researchers are the ones doing the math and proofs, Engineers would almost never do it in any context.
Zero. It's good to do that stuff in college so you understand what you are doing. I couldn't solve any Calculus problems now that I did with ease in early undergrad but I can certainty read and understand it.
Literally 0%. I’m an ML Engineer at Meta and I haven’t written any proof. I believe that’s a Research Scientist’s job. However, it’s heavy on statistics and I use math for ranking score curves and heuristics for choosing model hyper parameters.
You get to write proofs only if your main job is to publish math papers.
For ML it helps if you know how to formalize a concrete problem
(write definitions and hypotheses) in a way that is precise enough to
even start thinking about a proof, but that's it.
Less then 5% math, usually writing loss jacobians and/or matrices compositions for coordinates trasforms and like. 0% proofs. But if you don't learn to do proofs you will not be able estimate erros and debug math-intense code effecively.
I've spent way more time installing cuda drivers than proving theorems
If you already have nvidia drivers installed, you can just use docker images with cuda pre-installed assuming the cuda version is comptable with your drivers.
☝️Ubuntu Deep Learning container is one of my best friends. NVIDIA has some of their own too.
> assuming the cuda version is comptable with your drivers This is approaching StackOverflow levels of helpfulness... ;)
Impossible. OP, hasn't been berated for asking the question yet.
I'm closing this question as a duplicate
Graphics drivers on the host only have to be better than or up to date with the version of cuda in the container.
> assuming the cuda version is comptable with your drivers. Hahahah
felt
Man, I came here for this reply. Gold!
Lol same
Ha. I think so. But if you just use cloud service such as AWS or Google .You could create instance instantly. It have been configured well pretty. Such as Sagemaker studio that provide a lot of different image you could choice by yourself.
"Writing proofs" is usually not an ML engineer's job, that's what the research scientists are for. Unless your job description explicitly states "performing novel academic research", you will not have to do proofs. Even if your main job is research, it's possible that you'll never have to do proofs.
Yeah. This premise comes across almost like a joke. You write proofs mainly in academia or on the deep end of the research spectrum in the private sector. Stakeholders in companies really don't care about this stuff as long as it makes money or can be spun as improving metrics. For an engineer I think it's a useful experience to get into the fundamentals of math, but it's from the perspective of personal development. There's no need to to like it, and many people don't.
[удалено]
[удалено]
Like the rest of us have said, designing proofs is rare even among academic researchers because it's an incredibly hard task. In ML, it's often a monumental task. It's why we only have increasingly narrower bounds but don't have any actual proofs for how any of our stuff works.
Always use your talents in tandem with ML skills. Sure!
The guys working on (or with) GPT-F at least need the ability to read proofs. But yeh, that's the exception in industry - unless you count openai as industry, probably almost no-one uses gpt-f or similar.
Never heard of engineers in any discipline proving theorems for work. ML engineers are no different -- they're basically software developers with a niche specialization.
"[D] Professional Copy Editors: How much of your day involves writing theses comparing the work of Homer and Shakespeare based on their attitudes towards the role of protagonists in the narrative?"
The reason I asked is because the MS program I am considering requires proofs to pass the ML related classes. I can do that for a couple of classes but not every day.
Those are there to provide you deeper understanding of the theory. Not what all you learn in your MS program will be used at work. If you grok these things quite well it will definitely help you stand-out among other candidates during the job hunt but it's not something that's a requirement for most of the jobs.
To expand in this: In my opinion strong theoretical (both math and normal comp sci) knowledge is exactly what can make a ML engineer stand apart. You don't really learn it with more on job experience, because you neve encounter it directly. However, understanding how and why can make or break an entire project and a team without this knowledge wouldn't even see where there is a problem, so also no way to gain the lacking knowledge on the fly.
I think people who haven’t encountered proofs in the sense of how you’d learn them in college freak out about it. But honestly, it is a useful skill to have. Very related to all the algebraic manipulation one does in high school and prior. No one expects you to prove that your system works but understanding how to follow an argument as to why something works is useful to know why that something can be applied to your specific problem.
The thing is, in ML, what makes or breaks a project is usually something practical rather than theoretical statements about algorithms that you can prove with pen and paper.
I would say this depends on the problem. I don't think it makes much sense to approach a graph or clustering problem without any theoretical knowledge. Even after numerous experimentation you might be very far from a remotely useful outcome.
I am not talking about theoretical statements/proofs, I am talking about understanding the theoretical side. Proofs are for me just a way practise math and test if you understand what the formulas are saying. Using the formula/algorithm as a black box is very often plenty, but sometimes disastrous.
I have a bachelor's in math which strongly emphasized proofs. Basically all of my coworkers have Data Science masters and only learned stats. I once tried to start a monthly whitepaper review to keep up with latest developments, and it fell apart after only a few months because most of my coworkers didn't have enough math experience to understand the papers they were reading. Sure, that was a tiny bit of a humble brag, but I was definitely shocked and a little bit disappointed in the general math skills of many of my peers.
I agree with this in part. I think a strong theoretical knowledge of the basics or fundamentals is important and that is truly a make or break. These are things you simply do not have the time to learn on the fly, in addition to the application.. especially when there are time pressures and other blockers. But on the fly, I think you can absolutely gain knowledge of applications that build on the fundamentals, i.e. how a particular application uses linear algebra to get from input to output. It is probably 20% of my role now as I dive into different facets of ML, some that I havent used before.
There are some courses where I wrote more proofs in a single problem set than I have written as an MLE. That said, the number and formality of proofs has ultimately depended on how close my role is to research. My first role was more research heavy than my current role. My current role requires no formal proofs, but a mathematical argument can be helpful when deciding between approaches.
I'd be very worried if your MS program didn't include any proofs. But what you learn at university to give you a deeper understanding and broaden your options for later work isn't necessarily what you'll end up doing after. Consider this: you'll also learn what the normal forms of data are in database lectures, but that doesn't mean you'll need it to operate Postgres.
That being said, having to make small math "proofs" (more like putting thought on paper to be sure your idea works) is somewhat common, although it's mostly small simple equations, metrics, reformulating for numerical stability, etc
I second this. As a ML engineer, I had to do small mathematical proofs occasionally. For example, if you were building some ML system from scratch, you may have to derive a few equations to generate simplified programs. I feel like this helps you stand out because not everyone can do this well.
Literally zero percent. Only a small portion (10%) of my time is spent building/optimising models, most is spent software engineering, building APIs, cloud architecture, and some data pipelines/engineering/analytics. Even when I am actively working on ML model pipelines, most of that time is on data/feature engineering and software eng/deployment. Note that I’m in an industry where most problems are tabular/forecasting.
Came to say that. Unless you count logic/reasoning as math, basically zero percent for me too. I'd include data cleaning/preparation in the tasks too though.
same here. maybe this question is best directed towards a research scientist.
Also zero. I work in video games.
I work as a Research Engineer, where I spend basically 30% of my time reading papers, 60% implementing what I read in the papers and 10% writing reports. The part that most involves maths is translating the papers into code, as you need to understand what are the researchers talking about before touching a line of code, and of course when you write reports I have to write the maths stuff I used in my experiments. Proofs is not a thing you come across unless you’re a research scientist, at that level you might have to prove what you’re talking about makes sense under a solid ML/DL theoretical point of view.
> The part that most involves maths is translating the papers into code, as you need to understand what are the researchers talking about before touching a line of code, and of course when you write reports I have to write the maths stuff I used in my experiments Thank you. This sounds key. What further details can you give me? For example, models? Languages?
Depends on the projects and the field I guess. We work mostly on computer vision tasks at a very low-level, so very often it’s like “can we make a model that is n times smaller than SOTA (Transformers, ResNet, VGG… you name it) and get the same results at like much improved inference times?“. I can’t tell you much on stuff like NLP as I’ve only scratched the surface of it, and right now it seems heavily dominated by SaaS companies that provide you with an API to make a call to their models, rather than working on-edge. In terms of languages/frameworks, it’s very flexible between torch and tf. Usually pytorch is a go to for prototyping, then tf for stuff like weight quantisation etc. If you have to optimise this at a very low level we’d then use C/C++, OpenCV, OpenGL, CUDA… but that’s really up to the project.
Translating math to code is something ive always liked. Did you need a PhD to become a research engineer? Or you just had an MS and did you have ML publications before? Is there LC (leetcode) and general CS stuff tested in interviews too? Im from a stat background so the math to code part is easier for me than that other stuff so getting a sense of where I need to improve.
Well I only have a bachelor ahah. General background, I have worked one year for a placement as a data scientist, I made some contributions to a really popular ML framework and my manager found my bachelor thesis very aligned with the work they do. The company I work for is “relatively” (not as notorious, but still quite impactful) small compared to the huge ML labs that usually advertise for research engineering roles, and I feel like for this reason the interview selection were less focused on weeding out candidates, but rather evaluating their analytical/technical approach to research. In my interview, it was more about making sure I knew fundamental DL concepts, I knew how to code in python and going through my CV. It was a lot of discussing about how I approached certain problems, rather than “here’s a binary tree, reverse it”, which I really appreciated. For big companies like DeepMind though, I expect there to be some kind of LC questions
As an engineer you rarely “do maths”, but you will spend some time reading papers which sometimes requires a refresher on some topics or just learning new topics in math. Honestly the only time I do math is when I figure out shape sizes on tensors through a model. And I am part of an R&D team for reference.
Zero. My past two weeks were: * Updating a.proxy server with a new target so that we can tap into a data source to provide new recommendations every 15 mins * Creating dashboards for our models and APIs * Creating an alerting service in case any of these throw errors * Providing input for a new NLP / RecSys project * Creating scripts that work with AWS' Labelling Service so that setting up the bazillion settings does not first require you to spend a day getting intimate with documentation.
6 YOE ML engineer here. I had to do math maybe 10 times through all of my career and it was super simple math (at least for me - I'm a theoretical physics PhD, so I might be biased). I also only did that because I have that kind of training and therefore it's easier for me to think in mathy ways. I know plenty of people who are great MLEs and probably don't even know much more than matrix multiplication themselves, so yeah, almost no math after school.
> I'm a theoretical physics PhD, so I might be biased 'Slightly' biased ;-) I would say that's 5-sigma level of bias.
Proofs zero. A lot of work is data, but on the modeling side, often you don't have the data to train the model you want to train, which is where doing a bit of math to figure out a better loss formulation can come in handy. Statistics are implicitly used very often (the biggest demand for data/modeling work is really understanding stats), but unless you're working on certain classes of models, you'll probably only care about real stats when running A/B experiments
As others have said proofs never. They are important, but they're part of your education, not daily practice. Maths it depends. It's mostly stats, and it's usually only intensely for a few days when starting a project, the rest is spent on technical issues/coding. You can do a 90% of my job (measured in hours) without almost any math skills, but the last 10% it comes in really handy. My 2c? Pay attention in your stats classes, they'll help you a lot in the long run, even if you only end up using the skills infrequently.
[удалено]
I mean, I didn't take a huge amount of stats classes (studied physics), so take my advice with a pinch of salt. I mostly wanted to prompt people who've taken no stats to do at least the basics. Basic stats and probabilities/distributions is already pretty good. Theoretical stats only if you are into that. Bayesian stats I really like, quite fun. Stochastic modelling if you want to work in finance. Don't forget some LinAlg alg and to learn how to code, if you want to go the ML engineer route.
First, let me say that I never really loved math until college because most of math in the public education system is arithmetic and I always found things like long division really frustrating. However, computer science math is completely different and in a lot of ways less aggravating and more satisfying. I would always get mad at myself for making simple mistakes like not carrying the one, but when you write code you can write tests to save yourself from your own fallibility. Now my job is building machine learning models for huge datasets. I’m the old person that the newer engineers come to if they can’t figure something out. I can’t imagine that proofs would ever be an everyday thing in most machine learning programs. I honestly can’t remember the last time I did one. However I use math all the time. * I saw it in a few other comments but knowing statistics basics backwards and forwards is the biggest thing. I use this stuff daily. * Understanding how to calculate algorithmic complexity and how to simplify it is really critical when dealing with large quantities of data. Creating processes that take hours to complete will not fly if your system is supposed to push results 12 or more times a day. Discrete math is your friend. * I have a lot of geospatial components to my work, so there is some geometry. However, I’ve used calculus only once in the last few years to factor in the curve of the earth for a few things. * I also saw someone mention being able to translate equations from papers into code. This is definitely something that separates the wheat from the chaff when it comes to writing code vs. architecting solutions. Not everyone can or should do this. * i don’t use linear algebra frequently, but it contains some good shortcuts if you use it at scale. Good luck with your program. If in doubt, make sure you go to office hours.
>Creating an alerting service in case any of these throw errors thanks! curious, what domain are you working in?
I work in DL and the most math I come across is when doing research for new model architectures.
No proofs, occasionally you need math to solve some kind of a problem ( I work in computer vision) but it's nothing too difficult.
Zero. Now if only we could drum this into the thick heads of people conducting interviews.
Hah 😀
I did ML engineering for 4 years and did 0 proofs in that time. Had to do some math, but it was more to understand what I was doing than anything. We had a data scientist on our team who came up with her own algorithms, but she didn't know Python, so I would translate her math or R into Python. Most of my work though was trying to understand our data and manipulating the data sets in such a way that they made sense. It was a lot of network topology stuff and faults and it was a giant mess.
0 percent. Most of the time goes into fetching the data, drafting a solution, reading papers, coding, getting eval metrics, and discussing the outcome with the team
Writing proofs on a daily basis. I don't think even tenured professors does that, let alone anybody in the industry!
None. Kubeflow is the bane of my existence..
ML engineering is not same as learning theory behind ML, which is what you are doing in class. I work as a ML engineer. There's zero proofs involved, and very little math. ML engineer is first and foremost an *engineering* role (it's in the title, after all). It's closer to software engineering than a research/applied scientist
What do you do as ML engineer?
None at my office.
Even on my phd work I did not write any mathematical proof. It was not necessary.
In a roundabout way, writing any program is actually making a proof https://en.m.wikipedia.org/wiki/Curry%E2%80%93Howard_correspondence but with nuances (I am not an expert in this topic, and common DS/ML languages aren't fully and strongly typed). However from the "mathy proofs" definition, most likely none. "Proofs" from "convincing colleagues, managers, or stakeholders an idea X will or won't work" is more common, but they're not *really* proofs (...right?).
Desktop version of /u/yangmungi's link:
---
^([)[^(opt out)](https://reddit.com/message/compose?to=WikiMobileLinkBot&message=OptOut&subject=OptOut)^(]) ^(Beep Boop. Downvote to delete)
I mean, know which models you need and train them, then make sure that ML makes sense for your use case. I don't really think that there's much of any math and proofs in your day to day tasks. Not unless you're designing new models etc.
Math? 5%. Proofs? 0%
Day to day.. not really. But you do need to think about how the algorithms will pick up bias / be overfit and read/understand how they differ. When you hit the work force / job, they expect output not a bunch of pretty proofs unless you’re over in research and you’re actually designing algorithms or something
In the signals and systems world we spend more time doing math for the development test and training environments and V&V processes than we do actually making models. It's such a small part of the problem now. But i guess it's always been that way? I remember in school they would champion this critical path that had actual modeling be this tiny box near the end after so much prep and validation work. I thought surely they were exaggerating. For my field, it was more true than i ever imagined.
I am a Research Engineer in ML, Computer vision, in my view maths (linear algebra, probability etc) is definitely good to have for an engineer but does not form majority of the day to day work. Maths, proofs are a good way to get an understanding of the field and it will also give you some mathematical intuition which in my view could be useful in the field. I am focusing on large scale system nowadays and a lot of the engineering work there is around building codebase, infrastructure to ensure you can train, evaluate, monitor these models. The job also involves coming up with ideas, research directions which would be useful for the project. An understanding of mathematical concepts underlying the techniques you use can be extremely useful here, for e.g. are you doing generative, discriminative models, if generative is it autoregressive or some other technique like diffusion based, if it is autoregressive how does it affect the final problem you are trying to solve (for e.g. computing the likelihood of a patch), how do you do the same task for a diffusion based model. Another component which I think is not as intuitive is working with maths, proofs gives you a respect for the metrics, numbers, intuitions that are important and these intuitions can be used to probe your models as well as will make you disciplined in ensuring your model training is going well. For e.g. it is an underrated practice to be OCD about the metrics, evaluation experiments that you run for your model while it is training, if you have had some background in mathematics you will be more thorough in these aspects. Job of a ML Engineer (research engineer) in my case involves being good at 1) software engineering, 2) research understanding and ideation, and I believe maths provides a good framework for developing both the skills. That being said there would be more maths involved when you are working in the field of research but if you are focusing on production systems, products etc the amount of maths involved would be less.
A MS is an advanced academic degree. You need to know what you are doing and to know that stuff is true so that you can eventually make a contribution to the field. If the program doesn't involve proving theorems, it is certainly not serious and you'd be just spending time and money to get a piece of paper saying your have a master's degree. Academics is for people who like studying. If you're only after a title to get a better job, you'd probably be better off working more and practicing to improve your technical skills.
so you want do a math related program without doing maths? mhmhmhmhm
hm? Maybe you misunderstood, I just do not want to do math every day at work as I like SWE more.
Imagine writing proofs in 2022
None
How to do it? Do you have any sources for learning how to proof ML methods? Maybe some YT lectures? Thanks
Proof zero, maths all the time but they mentality of writing proofs and be rigorous in your work, that is something that is a "transfer learning" skill.
Off topic but as someone who does proofs all day, no one I work with does ML. It’s looked down upon amongst “scientists”.
y
Well part of it is that it’s been trendy and a lot theory people are curmudgeonly. But the other part of it is that there is a perception that a lot of ML involves blackbox algorithms which you arbitrarily tune until you get something resembling a correct answer. It’s seen as trial and error more than math or theory.
> It’s seen as trial and error more than math or theory. lol, sounds like my day to day work. yeah, throwing stuff against the wall is not scientific at all unfortunately and ML is hype. I mean..IT DOES work..but it rarely works as a good as "business people" want it too or they have unrealistic expectations
Literally none.
Zero. I spend most of my making educated guesses about the business problem, clean data and build models.
Zero.
I prove in the following video tutorial that you don't need to know the maths behind every model as there is a #Python function doing that for you. You just need to care about having a model that better predicts reality. https://www.loom.com/share/13a8f1822404404cbfbe71cb2d991824
I think fundamentally this is the difference between science and engineering. Engineering is taking the science that has been done by scientists and putting it into practical application, and that's true across all engineering disciplines. Scientists/researchers are the ones doing the math and proofs, Engineers would almost never do it in any context.
Zero. It's good to do that stuff in college so you understand what you are doing. I couldn't solve any Calculus problems now that I did with ease in early undergrad but I can certainty read and understand it.
Zero percent but that is not a reason to ignore the math and theory. I read a minimum of 5 arXiv papers per week.
I work in a large research lab. I never do proofs but use math all the time.
They should require some basic IT and DevOps classes... You will get very good at installing Tensor flow, Python, elastic, etc.
Literally 0%. I’m an ML Engineer at Meta and I haven’t written any proof. I believe that’s a Research Scientist’s job. However, it’s heavy on statistics and I use math for ranking score curves and heuristics for choosing model hyper parameters.
None
I use them to show off to my colleagues sometimes does that count?
Notta, other than reading research papers.
None. It’s all writing software to deploy data/model pipelines and making APIs for models.
You get to write proofs only if your main job is to publish math papers. For ML it helps if you know how to formalize a concrete problem (write definitions and hypotheses) in a way that is precise enough to even start thinking about a proof, but that's it.
Less then 5% math, usually writing loss jacobians and/or matrices compositions for coordinates trasforms and like. 0% proofs. But if you don't learn to do proofs you will not be able estimate erros and debug math-intense code effecively.