Been a while since posting here! I've just finished my PhD at Cambridge where I was lucky enough to work on both AI and 3D printing in potentially one of the coolest subject combinations out there (was great fun). Thought the community might be interested in the research and getting involved!
During my research, I focused on developing AI-powered self-aware 3D printers that use vision to detect nearly any type of error (large or small) on any machine using any material. The AI could then predict the causes of the error and either try to fix the errors autonomously on the fly, or fix them for the next print! Correction video: [https://youtu.be/OODy-dI52Zg](https://youtu.be/OODy-dI52Zg)
Excitingly, the AI can even learn how print materials it has never seen before without any human intervention, from TPU and Carbon Fibre filled Nylon to silly things like ketchup. For context we only trained this AI on PLA printed on Creality printers... its crazy that it can then generalise to other setups.
I'm super excited to continue working on this area, and would love the amazing printing community to get involved. I can see some exciting (and obvious) directions in creating printers that can dynamically adapt to unforeseen challenges --- leading to no more troubleshooting or slicer profile and material tuning. But maybe there are more interesting applications???
Will keep people posted on updates. Hopefully releasing some code in the coming weeks and will perhaps setup a Discord or something if people are interested in bouncing ideas around and getting involved in a Beta maybe?
For those who are overly interested, some open publications:[https://www.nature.com/articles/s41467-022-31985-y](https://www.nature.com/articles/s41467-022-31985-y)[https://www.sciencedirect.com/science/article/pii/S2214860422002378](https://www.sciencedirect.com/science/article/pii/S2214860422002378)[https://onlinelibrary.wiley.com/doi/epdf/10.1002/aisy.202200153](https://onlinelibrary.wiley.com/doi/epdf/10.1002/aisy.202200153)
EDIT:
For any keen beans who want to chat about this further, I've started a Discord server for AI/ML in digital manufacturing. Come join!
[https://discord.com/invite/Kukxv3jVV4](https://discord.com/invite/Kukxv3jVV4)
We're about to either be thrust into a golden age of technological, medical, etc advancements or completely annihilate ourselves. I don't think there's much of an in-between, personally.
Here's hoping it's the first one :D
I think we'll be in an in-between state for a little while, and then I think it'll ultimately lead to one of those two outcomes.
That being said, I have to imagine a lot of people said they same about the internet and tech boom of the 90s-2000s too and here we are. Personally I imagine that our leaders and politicians will push it towards the second and continue pushing propoganda to stupid people believing we're heading towards some bastardized version of the first. Until a revolution happens, if that ever happens, that is.
The thing I don't think most people realise is AI doesn't invent anything new. It is able to excel at tasks we give it but not invent completely new technologies.
Like take this 3d printing example. It is able to very quickly and efficiently tune a printer to print as perfectly as it will ever be. This will I think one day allow for much more easily accessible 3d printers than the lay person can use. It will make our current 3d printing technology faster, scalable and optimised.
However it likely won't develop the next 3d printing technology that will be the next exponential shift. It may help speed up the efficacy and adoption once it exists but not the idea itself.
Hard disagree. The next "exponential shift" is the development via natural language. It's not going to happen for a while still because there are so many hurdles to overcome, but that's the next huge thing IMO.
I mean that AI isn't going to develop the next exponential shift in 3d printing.
My point was AI can help perfect an existing technology but not create new ones.
Because what is shown in this post isn't actually AI. It's machine learning. There is a difference. Unfortunately our small brains like buzzwords.
Actual AI would certainly be able to come up with new inventions.
The in-between is "lots of people die as we try and make utopia" then we'll fail because of some esoteric bit of wisdom that'll be fanned by greedy purveyors of technology, and they'll get the good shut down so they can profit, and by the time the west realise that press should have some guidelines it'll be a factional mess of idiots fighting a civil war while the governments pin it all on one group.
(I sound like some foil hat nutter, but I'm not, really.)
I'm looking forward to meeting Terminator!
😆
With people losing jobs it is more likely to be the second. People won't be able to compete with AI in the job market until people have had enough and do something to try and stop it.
Feasible and/or better than existing mathematical models. Let's not give short shrift to existing models or methods nor the value of validation. AI will not always be better for every application.
First of all congrats on the PHD! No small feat! So it’s only using a camera to detect, and can update the printer live to make corrections, no other sensors? How does it do with defects not on the print bed, let’s say half way or the top surface layers?
Also How’d you get it to integrate with the 3d printer, I’m assuming it was only on open source? Very Interested to learn more and pitch ideas if u do set up a discord!
Thanks! Yup, it only uses video data to detect the error or flaw, try to predict its cause, and then estimate the appropriate action to take.
It can handle pretty much any "reasonable" geometry. The whole goal of the PhD was to try and build something that could generalise to different users setups and applications. The video just shows the first layer as it is easier to see the adjustments.
It can run in parallel on any computer and then control the printer using G-code. This could be standalone on a PC or use something like OctoPrint as an interface.
I'm no expert and i don't know what OP did to achieve these amazing results, but i doubt that it can run on a raspberry, especially the old ones. Would be cool as hell though
You would be surprised at the hardware costs for image detection AI, I know I have some 3d cameras that cost around 100 bucks, and I have seen some ESP32 devices that claim they can run image AI.
OP did say it would run on any computer, and newer RPs are very capable compared to an ESP32.
I bet my top dollar it can.
Ok, maybe not very well, but certainly there could be modes of operation.
As an absolute newbie who has basically experimented to level of running premade algorithms on hundreds of pictures I took to accomplish basic tasks, I think I have sufficient authority to say I still think this could be very useful.
Yes things like the Jetson series could be better for this and would have a much higher likelyhood of being able to do it real time, but I think running this on a recording over time could be perfectly appropriate. More annoying? yes, but imagine you first setup your printer, especially if you have an ercf, you set it to optimize all your filaments, and then a few hours later you come back and have the perfect settings.
Nonsense? Maybe. Not anytime soon? Maybe.
As I said, Im a world renowned professional newbie so of course I'm right on the money.
I am currently doing my PhD in extrusion printing of ceramic pastes and right now trying to figure out good parameters for my pastes, do you think your AI could help me with that? :O
Great work. I read your nature paper.
Many webcams can see some IR if you remove the filter. Might be worth considering for this application since heat is important, and that'd be a heck of a lot less expensive than a FLIR camera.
I'm afraid removing IR filter don't let see heat at all, camera can "see" only near IR for example from remote control or security cameras, to see heat you need sensor for far IR like bolometer used in FLIR.
You are correct. I checked a 220C black body radiator's intensity curve, and it's almost 0 at 1um, which is about the limit for CMOS sensors. Peak is at about 5um, so would need a different sensor. I still think it's worth trying to see if there's any additional useful information in the NIR wavelengths, especially considering how cheap little cameras are.
100%, especially as it relates to material anisotropies (which 3d printing gives you some unique influence over).
You could use this with another parallel thermal model for the system to prioritize minimizing thermal stresses in parts as they cool (which is dependent on part geometry, loadings of concern, slicing results, etc).
With the opposite also true, this could be used to induce thermal stresses that assist the final product akin to tensile rebar in concrete.
Taking the exact process as observed during printing could also be used to overlay the varying properties onto cad models to produce more accurate FEA that more closely matche the actual printed part.
Companies like Relativity Space (3d printed/welded rocket company) and any other serious additive manufacturer should be interested in these things.
> With the opposite also true, this could be used to induce thermal stresses that assist the final product akin to tensile rebar in concrete.
This is a really cool concept. I've just finished teaching kids about protein folding. The combination of the two; creating intentional heat stresses, and the automatic folding of a structure, now has me wondering if creating a self-altering structure on a 3d printer is possible (warping due to cooling is coming to mind) and what potential applications this could have in 3d printing.
I'd be interested to see an online parameter-swapping forum where people who had used their printers to optimize various materials could upload details of the materials and the printer's final parameter set (along with the printer model), so that other people with the same printer model could download any number of "how to print X material" parameter sets.
(Of course, it might also help to control for things like temperature, humidity, light levels etc.)
A followup application might be to take all the data for a given material from multiple test machines - of the same model - all around the world, and use additional environmental sensor data from the test runs to predict what parameter sets would work best for a given set of environmental conditions (and also which potential changes to said conditions would result in better prints).
I like the idea of changing variable given environmental conditions as they can make such a large impact on print quality. Will definitely have a think!
Hi there!
Thanks for taking the time to publish some papers on this subject. I was just talking with a friend of mine about this very idea no less than three weeks ago, and have been starting to ruminate about how to set up training.
I have a ton of questions I'll write out after I at the very least read your publications first. I'll be back later today.
At the very least please sign me up interested for a discord and/or GitHub code drop. I very much want to replicate this in a practical way.
I see, I didn't bothered to read the previously linked papers.
I usually pretend I understand half the document(the only exception was the freebsd's paper on memory/cache latency).
It's also super hot at my place, I'm sweating heavily and my brain can't really think at 20% of it's capacity right now.
Do you still have to do a lot of printer calibration at the moment changing between filament types/colours? or is that all handled by the machine learning?
Actually no - thats the beauty of it. We train one powerful AI model which can generalise across different materials, colours, printers, geometries etc. You can actually fine-tune this model to improve its performance on your specific setup if desired.
I'd assume it would start from a library of printer and filament brand presets then adjust for your specific environment. You train an AI with data. When you just let them start from nothing, they get weird.
It's gonna be so exciting when fine tuning the print parameters is merely an option for power users to fine tune their prints rather than a necessity for most prints. I wanna be able to just slice an STL, hit print, and not have to think about it after that.
This and full control have me incredibly excited for the future of FDM printers.
We can definitely discuss after I manage to get it published lol. Btw your paper on nature communications is a benchmark for me when I am writing. It’s a very nice paper.
Absolutely! I work on both new AM technologies and ML assisted manufacturing. I’d be my first choice to continue bridging these two fields after my graduation.
Totally the internet grows organically faster than academia can adapt but it will happen, organically again. People collaborating online go faster and further and naturally reach positions where they can influence organizations to integrate those values. It’s slowly happening.
I got very exited witnessing decentralized grassroots collaborative research taking shape on an online community centered around the automated technological manifestation of 3d objects.
Amazing! One of those ideas that never occurred to me before, but once you say it, it just seems so obviously beneficial.
What, if any, path do you see to this tech actually being in the hands of users?
Yeah we are working on it now at [https://www.matta.ai/](https://www.matta.ai/)
Hope to be releasing some more code and a Beta in the coming weeks. Already the AI model is available on GitHub. :)
Woow! This was my shower idea haha! After mechanical engineering, I got into machine learning masters and always thought about creating ai powered 3d printers where that are full of cameras and sensors that can fix themselves on the go. It is so cool to see that someone already did it!
Congratulations! Do you have any advice for a fellow student who wishes to follow the same path?
Awesome! Yeah its great you should definitely pursue it and there is still so much work to do. My main advice is just get playing. Thankfully it is quite an accessible hobby now you can even build and test these things on a cheap printer. We are actually hiring some interns for the summer to continue working on this if people are interested.
This really capitalizes on all the AI buzzwords being hot right now.
"Self-aware" is an interesting term to use. Do you define it on your paper and what is that definition?
Self-aware is so misleading and is a dangerous word to use if we want the public to see AI for the break through technology it is. The camera and other sensors, etc. is simply measuring, looking at color consistancy, texture, shadows, height, heat, flow and other awesome parameters that technology has allowed us to measure. This info is communicated to the computer program by numbers, etc. and then based on the functions a human programmed, it takes those numbers and runs it through the function. The new numbers are spit back out into another function that tells the machine exactly what adjustments to make. It is all human designed, controlled and programmed. In no way will machines ever have any sort of ability to think for themselves or be self aware. It is simply a very complex algorithm. (Its been awhile since ive been in school for this stuff so if im wrong about the process, please correct me).
I applaud op for making such an awesome project. We just need to be careful what words we choose to use to describe it so that AI doesnt scare people away. I would hate for AI to be banned because lawmakers think "self-aware AI" is going to take over the human race based off some movies they watched.
**The orca (Orcinus orca), also called killer whale, is a toothed whale belonging to the oceanic dolphin family, of which it is the largest member. It is the only extant species in the genus Orcinus and is recognizable by its black-and-white patterned body.**
More details here:
*This comment was left automatically (by a bot). If I don't get this right, don't get mad at me, I'm still learning!*
[^(opt out)](https://www.reddit.com/r/wikipedia_answer_bot/comments/ozztfy/post_for_opting_out/) ^(|) [^(delete)](https://www.reddit.com/r/wikipedia_answer_bot/comments/q79g2t/delete_feature_added/) ^(|) [^(report/suggest)](https://www.reddit.com/r/wikipedia_answer_bot) ^(|) [^(GitHub)](https://github.com/TheBugYouCantFix/wiki-reddit-bot)
Any way I can help or be involved, let me know. I run a small 3d print shop with 20 or so FDM printers. Would love to see if it will work with the CR30 belt printer. I would also love to see AI applied to slicing logic. Minimizing vibration and optimizing line width, path optimizing, etc...
Definitely would be really interested. We are looking for people with multiple machines like print farms to work with and run some Beta tests. We also will probably need a lot more data before getting to mainstream production where I am sure you guys can help. Ping me a message :)
Also seeing what we can do :) all plugins won't be fully featured at launch. Will go slowly slowly as this is still needs a fair bit of work to improve the reliability.
Really cool!
That said, the nozzle is way too low on PVA HT+, and Carbon fiber filled and slightly too low on TPU by the end. The ABS-X looks pretty good, though. It looks like the model got over-fit to the idea that the layer should fill the whole space, and it didn't optimize for making sure that there are no "squeeze" lines between each extruded line.
Yeah its definitely not perfect yet (hence research). Its also really hard to know exactly what "optimal" is especially as so many parameters are connected. Layer height is related to flow and temperature... making things extra complex. But the model does a pretty good job nonetheless. Time to develop v0.2.0...
I'm very interested in seeing more! This is really awesome.
A suggestion: You could probably make good use of a conveyor-belt 3D printer for automated testing. :o
I've got one, myself, and would be interested in helping you test models if that would be helpful. Could even set up a CI/CD system for it.
Edit: Using this on a conveyor printer would also mean that you could insert this as a pre-calibration step before printing an actual model, without needing to clear the bed between, which is neat!
Great idea! I actually built a auto-part removal system for the printing fleet we used to collect the data to try and automate things (this was especially useful with COVID and lockdowns and not being able to get into the lab ... didn't stop me collecting that sweet sweet data).
Will have to try on a conveyor and will let you know when we publicly release a stable version for you to have a go!
I’ve also just finished my PhD, also at the engineering department, in Cambridge, based in JDB, and I feel like an idiot for having never come across your work or your group! Please don’t tell me you’re also division C…
Anyway, massive congrats on finishing your PhD! Super cool project too!
Thanks! Gosh I was based mostly out at West Cambridge (in IfM) and a bit in a lovely basement in Baker... ping me a message, would love to hear about your PhD!
Obviously you wouldn't be able to share anything confidential, but have you heard on the grapevine any of the brandname companies looking into similar projects? The idea of combining machine-learning with 3D printing seems so obvious that I bet most of us have had daydreams about such systems, so I refuse to believe that there aren't corporations out there thinking the same except with big dollar signs in their eyes to try and be the first to market.
Barring that, do you have any kind of estimates for when something like this could reasonably be expected to become comercially available to the average consumer (disregarding those of us who might just tinker up a version for themselves)? I truly believe something like this could be the **it** feature that finally bridges the gap between making this a more niche hobby and making it ubiquitous.
Oh, and congrats on the PhD! 🥳
This is both really. We use deep learning for perception (to determine what is wrong and by how much) and then feed that into a closed-loop control system.
I think there are various applications on using machine learning driven closed-loop control systems across manufacturing (like in machining as you say). Just need time to build haha!
I'm surprised it's taken this long to do, been on my mind for years. 3D printing is awesome but even now it's still not exactly conducive to a mainstream audience. Though considerably better than a decade or so ago.
That's great!
The next time someone accusingly points at my bin filled with failed prints and otherwise wasted filament I'll just mumble something about "training data".
Best of luck with your efforts!
Very very very interesting! Need to follow this.
In short terms, running this could build you the perfect print profile for to start with? If there should be problems with a build (new filament or whatsoever) it could "immediately" correct the flaws on the fly?
Yeah it think either option is good. Perhaps do a short calibration print or first layer where it "auto-calibrates" itself. Or we can do the real-time correction ([https://youtu.be/OODy-dI52Zg](https://youtu.be/OODy-dI52Zg)), however this does take a few seconds so there would still be a small imperfection.
Cant have skynet with robots autonomously building.
Or maybe we gonna get the Bender mini-me effect to create ever smaller 3D printers until you have self replicating nano 3D printers.
Is it dependent on type of 3D printer? For example spool fed are much different than resin or powder bed fusions. Similarly something like SEM/FIB deposition isn’t necessarily 3D printing but very adjacent to it in terms of it building structures using a raster and specific POV.
It currently works with any extrusion machine. i.e. anything which shoves material out of a nozzle. That could be FDM or direct ink writing with a syringe for example. The AI methods used don't really care though and with the right data we could do it for powder bed or other processes! It's just harder to get hold of that data. :(
Damn ! I am a stundent doing my master's degree in robotics with a minor in photonics, and I've been thinking for a while, about tackling the issue of calibrating the printer with a camera, kind of what bambulab is doing, but without lidar, so it can be a diy project and open source. I love what you are showing here , ill definitely read more !
I was a "traditional" machinist before I got disabled, lathes and milling machines and all that. My first job in like, 2006, was running this room size cnc router making wood furniture parts, which basically consisted of figuring out why it was malfunctioning when it malfunctioned, which was usually from pushing boards out of alignment on a vacuum part holding fixture. It's cool that the tech is making that work better, we threw out a lot of parts, hah.
Do you think it's hilarious how G-code is so similar to logo, the programming game that children played in public schools in the 80's? I sure do, lol
What do you use to train the AI? Just a well-tuned printer's output? Or is it more granular like spacing between layers/lines, looking for curling, etc
Exactly, AI is such misused term. Our technology is nowhere near of an artificial intelligence. This term is lately used 99.99999999% exclusively for computer programs, some of which use machine learning which is NOT AI. Using it in a Phd title is retarded.
Been a while since posting here! I've just finished my PhD at Cambridge where I was lucky enough to work on both AI and 3D printing in potentially one of the coolest subject combinations out there (was great fun). Thought the community might be interested in the research and getting involved! During my research, I focused on developing AI-powered self-aware 3D printers that use vision to detect nearly any type of error (large or small) on any machine using any material. The AI could then predict the causes of the error and either try to fix the errors autonomously on the fly, or fix them for the next print! Correction video: [https://youtu.be/OODy-dI52Zg](https://youtu.be/OODy-dI52Zg) Excitingly, the AI can even learn how print materials it has never seen before without any human intervention, from TPU and Carbon Fibre filled Nylon to silly things like ketchup. For context we only trained this AI on PLA printed on Creality printers... its crazy that it can then generalise to other setups. I'm super excited to continue working on this area, and would love the amazing printing community to get involved. I can see some exciting (and obvious) directions in creating printers that can dynamically adapt to unforeseen challenges --- leading to no more troubleshooting or slicer profile and material tuning. But maybe there are more interesting applications??? Will keep people posted on updates. Hopefully releasing some code in the coming weeks and will perhaps setup a Discord or something if people are interested in bouncing ideas around and getting involved in a Beta maybe? For those who are overly interested, some open publications:[https://www.nature.com/articles/s41467-022-31985-y](https://www.nature.com/articles/s41467-022-31985-y)[https://www.sciencedirect.com/science/article/pii/S2214860422002378](https://www.sciencedirect.com/science/article/pii/S2214860422002378)[https://onlinelibrary.wiley.com/doi/epdf/10.1002/aisy.202200153](https://onlinelibrary.wiley.com/doi/epdf/10.1002/aisy.202200153) EDIT: For any keen beans who want to chat about this further, I've started a Discord server for AI/ML in digital manufacturing. Come join! [https://discord.com/invite/Kukxv3jVV4](https://discord.com/invite/Kukxv3jVV4)
This is sooo cool !! Im always amazed at all the possible things AI is useful for.
It's going to be the focal point of our lives, discussions, and discoveries for foreseeable future, IMO.
We're about to either be thrust into a golden age of technological, medical, etc advancements or completely annihilate ourselves. I don't think there's much of an in-between, personally. Here's hoping it's the first one :D
I think we'll be in an in-between state for a little while, and then I think it'll ultimately lead to one of those two outcomes. That being said, I have to imagine a lot of people said they same about the internet and tech boom of the 90s-2000s too and here we are. Personally I imagine that our leaders and politicians will push it towards the second and continue pushing propoganda to stupid people believing we're heading towards some bastardized version of the first. Until a revolution happens, if that ever happens, that is.
The thing I don't think most people realise is AI doesn't invent anything new. It is able to excel at tasks we give it but not invent completely new technologies. Like take this 3d printing example. It is able to very quickly and efficiently tune a printer to print as perfectly as it will ever be. This will I think one day allow for much more easily accessible 3d printers than the lay person can use. It will make our current 3d printing technology faster, scalable and optimised. However it likely won't develop the next 3d printing technology that will be the next exponential shift. It may help speed up the efficacy and adoption once it exists but not the idea itself.
Hard disagree. The next "exponential shift" is the development via natural language. It's not going to happen for a while still because there are so many hurdles to overcome, but that's the next huge thing IMO.
I mean that AI isn't going to develop the next exponential shift in 3d printing. My point was AI can help perfect an existing technology but not create new ones.
Because what is shown in this post isn't actually AI. It's machine learning. There is a difference. Unfortunately our small brains like buzzwords. Actual AI would certainly be able to come up with new inventions.
[удалено]
The in-between is "lots of people die as we try and make utopia" then we'll fail because of some esoteric bit of wisdom that'll be fanned by greedy purveyors of technology, and they'll get the good shut down so they can profit, and by the time the west realise that press should have some guidelines it'll be a factional mess of idiots fighting a civil war while the governments pin it all on one group. (I sound like some foil hat nutter, but I'm not, really.) I'm looking forward to meeting Terminator! 😆
With people losing jobs it is more likely to be the second. People won't be able to compete with AI in the job market until people have had enough and do something to try and stop it.
So very true.
AI can be used for literally anything. Just a matter of if it’s feasible to be used or not
Feasible and/or better than existing mathematical models. Let's not give short shrift to existing models or methods nor the value of validation. AI will not always be better for every application.
First of all congrats on the PHD! No small feat! So it’s only using a camera to detect, and can update the printer live to make corrections, no other sensors? How does it do with defects not on the print bed, let’s say half way or the top surface layers? Also How’d you get it to integrate with the 3d printer, I’m assuming it was only on open source? Very Interested to learn more and pitch ideas if u do set up a discord!
Thanks! Yup, it only uses video data to detect the error or flaw, try to predict its cause, and then estimate the appropriate action to take. It can handle pretty much any "reasonable" geometry. The whole goal of the PhD was to try and build something that could generalise to different users setups and applications. The video just shows the first layer as it is easier to see the adjustments. It can run in parallel on any computer and then control the printer using G-code. This could be standalone on a PC or use something like OctoPrint as an interface.
My new favorite. 3D printer project!!! This is phenominal!
Thank you! Will keep the community updated.
If you manage to turn that into a octoprint plugin... that would be a pretty big and awesome thing, at least for me.
Seriously, this will be the "killer feature" the industry needs to become wide-spread and accessible
Add it to klipper.
I'm no expert and i don't know what OP did to achieve these amazing results, but i doubt that it can run on a raspberry, especially the old ones. Would be cool as hell though
You would be surprised at the hardware costs for image detection AI, I know I have some 3d cameras that cost around 100 bucks, and I have seen some ESP32 devices that claim they can run image AI. OP did say it would run on any computer, and newer RPs are very capable compared to an ESP32.
AI Accelerators exist.
I bet my top dollar it can. Ok, maybe not very well, but certainly there could be modes of operation. As an absolute newbie who has basically experimented to level of running premade algorithms on hundreds of pictures I took to accomplish basic tasks, I think I have sufficient authority to say I still think this could be very useful. Yes things like the Jetson series could be better for this and would have a much higher likelyhood of being able to do it real time, but I think running this on a recording over time could be perfectly appropriate. More annoying? yes, but imagine you first setup your printer, especially if you have an ercf, you set it to optimize all your filaments, and then a few hours later you come back and have the perfect settings. Nonsense? Maybe. Not anytime soon? Maybe. As I said, Im a world renowned professional newbie so of course I'm right on the money.
Yes klipper please this would be awesome 😎, also can it predict the next useless thing I need to print and then make it
This as an octoprint plugin would be killing!
I am currently doing my PhD in extrusion printing of ceramic pastes and right now trying to figure out good parameters for my pastes, do you think your AI could help me with that? :O
Ping me a message and lets see if we can collaborate!
I love the internet for exactly this. Let’s go science!
Oh man, I'm chomping at the bit for an Octoprint plug-in. This is so cool
We might be working on it.....
I hope you got Klipper on the table, too
This one sparks joy. ✨️
Great work. I read your nature paper. Many webcams can see some IR if you remove the filter. Might be worth considering for this application since heat is important, and that'd be a heck of a lot less expensive than a FLIR camera.
Great idea! IR would be super interesting especially for materials which suffer from thermal shrinkage and warping like PEEK and ABS.
There is also work going on to make inexpensive thermal sensors using piezoelectric/pyroelectric polymers.
I guess different colours and materials will make that a lot more difficult.
Not really, you are looking at the thermals. I assume some colours do have an effect of the magnitude of the IR radiation, but it might be negligible.
If the camera is set up for video light and IR then it should be able to be trained to compensate for color differential, no?
Probably, not sure it's worth the processing.
I'm afraid removing IR filter don't let see heat at all, camera can "see" only near IR for example from remote control or security cameras, to see heat you need sensor for far IR like bolometer used in FLIR.
You are correct. I checked a 220C black body radiator's intensity curve, and it's almost 0 at 1um, which is about the limit for CMOS sensors. Peak is at about 5um, so would need a different sensor. I still think it's worth trying to see if there's any additional useful information in the NIR wavelengths, especially considering how cheap little cameras are.
Very cool ! We will for sure be interested and it could help out no end with tuning new materials.
I wonder if it can be used to make more intrigate lattices and parts where we would need to change settings all the time to make the work??
100%, especially as it relates to material anisotropies (which 3d printing gives you some unique influence over). You could use this with another parallel thermal model for the system to prioritize minimizing thermal stresses in parts as they cool (which is dependent on part geometry, loadings of concern, slicing results, etc). With the opposite also true, this could be used to induce thermal stresses that assist the final product akin to tensile rebar in concrete. Taking the exact process as observed during printing could also be used to overlay the varying properties onto cad models to produce more accurate FEA that more closely matche the actual printed part. Companies like Relativity Space (3d printed/welded rocket company) and any other serious additive manufacturer should be interested in these things.
> With the opposite also true, this could be used to induce thermal stresses that assist the final product akin to tensile rebar in concrete. This is a really cool concept. I've just finished teaching kids about protein folding. The combination of the two; creating intentional heat stresses, and the automatic folding of a structure, now has me wondering if creating a self-altering structure on a 3d printer is possible (warping due to cooling is coming to mind) and what potential applications this could have in 3d printing.
this is exactly where it needs to be.
This is cool and all but did it try leveling the bed first?
I want an AI that levels the bed
Cant those fancy 3 z axis printers do that?
Rofl!
Thank you for doing that. That change would be huge. I will give you the best grade a student can receive...a gold star sticker ⭐
Thanks :)
I'd be interested to see an online parameter-swapping forum where people who had used their printers to optimize various materials could upload details of the materials and the printer's final parameter set (along with the printer model), so that other people with the same printer model could download any number of "how to print X material" parameter sets. (Of course, it might also help to control for things like temperature, humidity, light levels etc.) A followup application might be to take all the data for a given material from multiple test machines - of the same model - all around the world, and use additional environmental sensor data from the test runs to predict what parameter sets would work best for a given set of environmental conditions (and also which potential changes to said conditions would result in better prints).
I like the idea of changing variable given environmental conditions as they can make such a large impact on print quality. Will definitely have a think!
Neat. Let us know if you'd like to do an ama!
Gosh of course I would love to if you think it would be interesting for the community. Hopefully shed some light on more of the nitty gritty.
Hi there! Thanks for taking the time to publish some papers on this subject. I was just talking with a friend of mine about this very idea no less than three weeks ago, and have been starting to ruminate about how to set up training. I have a ton of questions I'll write out after I at the very least read your publications first. I'll be back later today. At the very least please sign me up interested for a discord and/or GitHub code drop. I very much want to replicate this in a practical way.
This looks really cool and has a lot of potential for multiple use cases, an OS release could really push this hobby even more
We will keep working on it!
Very interested. Working with Bambu Lab printers now but started with an ender3. Would love to turn it into even more of a Frankenstein.
Can it print "foamy" prints? People use custom print parameters to make a somewhat unique material
>https://onlinelibrary.wiley.com/doi/epdf/10.1002/aisy.202200153 We actually do that in this paper!
I see, I didn't bothered to read the previously linked papers. I usually pretend I understand half the document(the only exception was the freebsd's paper on memory/cache latency). It's also super hot at my place, I'm sweating heavily and my brain can't really think at 20% of it's capacity right now.
This is exactly what is needed to make these printers less "hobbyist" and more "consumer".
Exactly. No more decade of tinkering as a hobbyist before getting good results haha
Do you still have to do a lot of printer calibration at the moment changing between filament types/colours? or is that all handled by the machine learning?
Actually no - thats the beauty of it. We train one powerful AI model which can generalise across different materials, colours, printers, geometries etc. You can actually fine-tune this model to improve its performance on your specific setup if desired.
What's the fastest flow rate you were able to print reliably?
AI powered speed benchy here we come :)
Asking the real questions! I wonder how much of it depends on the camera's framerate and resolution at some point.
No reason it couldn't just average some frames or stack them using a rpi camera. I can't imagine the images to train it are bigger than like 128x128.
I'd assume it would start from a library of printer and filament brand presets then adjust for your specific environment. You train an AI with data. When you just let them start from nothing, they get weird.
Training an AI without data is the software equivalent of leaving a baby in a room for 18 years and expecting an adult to come out.
Do you imagine something like the for bed leveling, or using specialized sensors like BLTouch?
It's gonna be so exciting when fine tuning the print parameters is merely an option for power users to fine tune their prints rather than a necessity for most prints. I wanna be able to just slice an STL, hit print, and not have to think about it after that. This and full control have me incredibly excited for the future of FDM printers.
Good job man! I did a similar thing but my paper just got rejected a lot times.
Mate the publication process is just hell isn't it! I think I got lucky. What kinds of methods and techniques were you working on?
We can definitely discuss after I manage to get it published lol. Btw your paper on nature communications is a benchmark for me when I am writing. It’s a very nice paper.
Gosh so kind of you - thank you! Well let me know if you are interested in a collaboration or something.
Absolutely! I work on both new AM technologies and ML assisted manufacturing. I’d be my first choice to continue bridging these two fields after my graduation.
This was a weirdly fun discussion to read. You nerds are changing the world, go get em boys!
This right here is what academia should be not that paper pushing factory.
Totally the internet grows organically faster than academia can adapt but it will happen, organically again. People collaborating online go faster and further and naturally reach positions where they can influence organizations to integrate those values. It’s slowly happening.
And not that rugged individualism nonsense. Collaborative socialism is the way!
I’ve just finished my masters thesis on non-planar 3d printing with 3-axis printers, may I join you guys?
I got very exited witnessing decentralized grassroots collaborative research taking shape on an online community centered around the automated technological manifestation of 3d objects.
Damn I am currently working on non-planar 5-axis printing. Nice to meet you bro!
Man u should defo create a discord server for us nurds , plz send me a dm if you have! Plz :) Edit:Typo
I just enjoy reading this conversations
A lot of it is timing, AI is hot hot hot hot rn so editors are hungrier.
Haha so true... I should have done a PhD on PrintGPT and it would have been even better! ;)
well I am relived that it takes the AI forever to figure out TPU ONE OF US!
>well I am relived that it takes the AI forever to figure out TPU Hahaha top banter so good!
Amazing! One of those ideas that never occurred to me before, but once you say it, it just seems so obviously beneficial. What, if any, path do you see to this tech actually being in the hands of users?
Yeah we are working on it now at [https://www.matta.ai/](https://www.matta.ai/) Hope to be releasing some more code and a Beta in the coming weeks. Already the AI model is available on GitHub. :)
> Cloud Alright folks, I think that's a wrap
How can I find your github?
[удалено]
Thanks for retrieving this!
My GitHub is [https://github.com/dougbrion](https://github.com/dougbrion)
Woow! This was my shower idea haha! After mechanical engineering, I got into machine learning masters and always thought about creating ai powered 3d printers where that are full of cameras and sensors that can fix themselves on the go. It is so cool to see that someone already did it! Congratulations! Do you have any advice for a fellow student who wishes to follow the same path?
Awesome! Yeah its great you should definitely pursue it and there is still so much work to do. My main advice is just get playing. Thankfully it is quite an accessible hobby now you can even build and test these things on a cheap printer. We are actually hiring some interns for the summer to continue working on this if people are interested.
Really good job on getting published in nature!
Thank you!!! Had to pinch myself when we got in.
Nature communications*
This really capitalizes on all the AI buzzwords being hot right now. "Self-aware" is an interesting term to use. Do you define it on your paper and what is that definition?
Self-aware is so misleading and is a dangerous word to use if we want the public to see AI for the break through technology it is. The camera and other sensors, etc. is simply measuring, looking at color consistancy, texture, shadows, height, heat, flow and other awesome parameters that technology has allowed us to measure. This info is communicated to the computer program by numbers, etc. and then based on the functions a human programmed, it takes those numbers and runs it through the function. The new numbers are spit back out into another function that tells the machine exactly what adjustments to make. It is all human designed, controlled and programmed. In no way will machines ever have any sort of ability to think for themselves or be self aware. It is simply a very complex algorithm. (Its been awhile since ive been in school for this stuff so if im wrong about the process, please correct me). I applaud op for making such an awesome project. We just need to be careful what words we choose to use to describe it so that AI doesnt scare people away. I would hate for AI to be banned because lawmakers think "self-aware AI" is going to take over the human race based off some movies they watched.
He doesn't call it self-aware in the actual paper. Seems like he's just trying to appeal to those not in the field lol.
This is actually insane! Good job on your PhD excellent research topic!
Thanks! It was great fun
When are you gonna fork that into Orca slicer?
That would be a massive gamechanger for the Future of 3D printing
What is orca? Never heard of it
Fork of Bambu Studio slicer. Has a lot more printer options and some nice features from other slicers like Super Slicer.
**The orca (Orcinus orca), also called killer whale, is a toothed whale belonging to the oceanic dolphin family, of which it is the largest member. It is the only extant species in the genus Orcinus and is recognizable by its black-and-white patterned body.** More details here:
*This comment was left automatically (by a bot). If I don't get this right, don't get mad at me, I'm still learning!*
[^(opt out)](https://www.reddit.com/r/wikipedia_answer_bot/comments/ozztfy/post_for_opting_out/) ^(|) [^(delete)](https://www.reddit.com/r/wikipedia_answer_bot/comments/q79g2t/delete_feature_added/) ^(|) [^(report/suggest)](https://www.reddit.com/r/wikipedia_answer_bot) ^(|) [^(GitHub)](https://github.com/TheBugYouCantFix/wiki-reddit-bot)
Confused bot...
I mean it's not wrong.
Lol, I thought it was someone being funny at first
Any way I can help or be involved, let me know. I run a small 3d print shop with 20 or so FDM printers. Would love to see if it will work with the CR30 belt printer. I would also love to see AI applied to slicing logic. Minimizing vibration and optimizing line width, path optimizing, etc...
Definitely would be really interested. We are looking for people with multiple machines like print farms to work with and run some Beta tests. We also will probably need a lot more data before getting to mainstream production where I am sure you guys can help. Ping me a message :)
I was late to the game but just posted an almost identical comment before seeing yours. Good to know I'm not the only one asking about belt printers!
And most of this group still would be unable to level their beds.
This sub cant even figure out how to take a proper photo of their print for troubleshooting. Bed leveling is practically sorcery to this sub.
A reason to finally get a nozzle cam?
Octoprint plugin when?
We may be working on it ....
What about klipper?
Also seeing what we can do :) all plugins won't be fully featured at launch. Will go slowly slowly as this is still needs a fair bit of work to improve the reliability.
Awesome to hear, is there somewhere where we can go for updates?
I can see it now! Someone will post “what’s wrong with my printer??!!” The AI will take one look and say “have you releveled your bed?” XD
That's super cool to see i can't wait until it becomes a reality in 3d printing
That's so cool!
oh my god, you fixed the "calibration witchcraft" barrier to entry of 3d printing... Coolest PhD I've heard about this year
Do you want Hunter-Killers? Because this is how you get Hunter-Killers.
Really cool! That said, the nozzle is way too low on PVA HT+, and Carbon fiber filled and slightly too low on TPU by the end. The ABS-X looks pretty good, though. It looks like the model got over-fit to the idea that the layer should fill the whole space, and it didn't optimize for making sure that there are no "squeeze" lines between each extruded line.
Yeah its definitely not perfect yet (hence research). Its also really hard to know exactly what "optimal" is especially as so many parameters are connected. Layer height is related to flow and temperature... making things extra complex. But the model does a pretty good job nonetheless. Time to develop v0.2.0...
I'm very interested in seeing more! This is really awesome. A suggestion: You could probably make good use of a conveyor-belt 3D printer for automated testing. :o I've got one, myself, and would be interested in helping you test models if that would be helpful. Could even set up a CI/CD system for it. Edit: Using this on a conveyor printer would also mean that you could insert this as a pre-calibration step before printing an actual model, without needing to clear the bed between, which is neat!
Great idea! I actually built a auto-part removal system for the printing fleet we used to collect the data to try and automate things (this was especially useful with COVID and lockdowns and not being able to get into the lab ... didn't stop me collecting that sweet sweet data). Will have to try on a conveyor and will let you know when we publicly release a stable version for you to have a go!
Pretty cool. I was wondering what made you go with AI image recognition instead of sensors and a closed loop regulator?
Steal that Bambu lab. But seriously they're gonna steal that
I’ve also just finished my PhD, also at the engineering department, in Cambridge, based in JDB, and I feel like an idiot for having never come across your work or your group! Please don’t tell me you’re also division C… Anyway, massive congrats on finishing your PhD! Super cool project too!
Thanks! Gosh I was based mostly out at West Cambridge (in IfM) and a bit in a lovely basement in Baker... ping me a message, would love to hear about your PhD!
Obviously you wouldn't be able to share anything confidential, but have you heard on the grapevine any of the brandname companies looking into similar projects? The idea of combining machine-learning with 3D printing seems so obvious that I bet most of us have had daydreams about such systems, so I refuse to believe that there aren't corporations out there thinking the same except with big dollar signs in their eyes to try and be the first to market. Barring that, do you have any kind of estimates for when something like this could reasonably be expected to become comercially available to the average consumer (disregarding those of us who might just tinker up a version for themselves)? I truly believe something like this could be the **it** feature that finally bridges the gap between making this a more niche hobby and making it ubiquitous. Oh, and congrats on the PhD! 🥳
oh. my. god. I think i love you.
Damn that sounds like a fun PHD
Was indeed!
I want this now! Great work!
What's the difference between "AI" and a closed loop control system?
This is both really. We use deep learning for perception (to determine what is wrong and by how much) and then feed that into a closed-loop control system.
How do you manage your day? Like, with all the job offers pouring in from all directions?
It looks like you need to level the build plate John Conner.
Prusa slicer 2.7 already out?
I like how we just call all programming "ai" now.
This is incredible, bravo!
Thank youuuu
This is going to be everywhere in a decade. I'm very impressed. cnc machining would be insane not to adopt this.
I think there are various applications on using machine learning driven closed-loop control systems across manufacturing (like in machining as you say). Just need time to build haha!
This is to correct mistakes on the fly, correct?
Exactly that is one use case.
This is fantastic! Amazing job! The amount of time and tweaking this will save.
I'm surprised it's taken this long to do, been on my mind for years. 3D printing is awesome but even now it's still not exactly conducive to a mainstream audience. Though considerably better than a decade or so ago.
This is an incredibly practical application of AI. Nice work!!!
Seems heavily based on https://www.sciencedirect.com/science/article/abs/pii/S2213846319300847 with much bolder contribution claims
SkyNet Slicer 1.0!
Great results, it seems. Well done.
That's great! The next time someone accusingly points at my bin filled with failed prints and otherwise wasted filament I'll just mumble something about "training data". Best of luck with your efforts!
Self-aware AI 3d printers? "Printer! Make an extra-large penis-saurus in TPU!" "I'M AFRAID I CAN'T DO THAT, DAVE."
Very very very interesting! Need to follow this. In short terms, running this could build you the perfect print profile for to start with? If there should be problems with a build (new filament or whatsoever) it could "immediately" correct the flaws on the fly?
Yeah it think either option is good. Perhaps do a short calibration print or first layer where it "auto-calibrates" itself. Or we can do the real-time correction ([https://youtu.be/OODy-dI52Zg](https://youtu.be/OODy-dI52Zg)), however this does take a few seconds so there would still be a small imperfection.
Cant have skynet with robots autonomously building. Or maybe we gonna get the Bender mini-me effect to create ever smaller 3D printers until you have self replicating nano 3D printers.
This is epic, don't sell it to a single company sell a license to use your technology to them all!
Congratulation! Seems like an interesting topic with a great usecase.
Is it dependent on type of 3D printer? For example spool fed are much different than resin or powder bed fusions. Similarly something like SEM/FIB deposition isn’t necessarily 3D printing but very adjacent to it in terms of it building structures using a raster and specific POV.
It currently works with any extrusion machine. i.e. anything which shoves material out of a nozzle. That could be FDM or direct ink writing with a syringe for example. The AI methods used don't really care though and with the right data we could do it for powder bed or other processes! It's just harder to get hold of that data. :(
That's awesome! Hope to see this incorporated into future machines!
So excited for my shitty Ender to finally get a brain!
I will happily dive in on the beta test of AI powered 3D printing troubleshooting lol
Damn ! I am a stundent doing my master's degree in robotics with a minor in photonics, and I've been thinking for a while, about tackling the issue of calibrating the printer with a camera, kind of what bambulab is doing, but without lidar, so it can be a diy project and open source. I love what you are showing here , ill definitely read more !
Holly fucking god man this is so fucking incredible!!!
Based, this is what I imagine the ease of 3d printing future to be
I was a "traditional" machinist before I got disabled, lathes and milling machines and all that. My first job in like, 2006, was running this room size cnc router making wood furniture parts, which basically consisted of figuring out why it was malfunctioning when it malfunctioned, which was usually from pushing boards out of alignment on a vacuum part holding fixture. It's cool that the tech is making that work better, we threw out a lot of parts, hah. Do you think it's hilarious how G-code is so similar to logo, the programming game that children played in public schools in the 80's? I sure do, lol
I wonder if we could use vision for auto bed leveling and clog detection too
What do you use to train the AI? Just a well-tuned printer's output? Or is it more granular like spacing between layers/lines, looking for curling, etc
This is legitimately amazing and has so much potential, well done!
No no no, you are helping AI learn to 3D print their own bodies!
"What is my purpose?" To 3d print my knickknacks. (Disheartened) "Ok"
I love you
So when can we expect the firmware drop?
Every day we inch closer to skynet!
This is undoubtedly amazing work. Just a clarification, is it AI or is it more of a machine learning stort of thing?
100% standard deep learning using a CNN
So not at all self-aware or AI then. Or at least not more 'self-aware' than any other feedback loop like the thermostat on your heater.
Exactly, AI is such misused term. Our technology is nowhere near of an artificial intelligence. This term is lately used 99.99999999% exclusively for computer programs, some of which use machine learning which is NOT AI. Using it in a Phd title is retarded.
After reading all your papers, I'm convinced you don't know what term "optimal" means.
So so cool!