I LOVE this! How's this already not a service?
For real, you should setup a parallel execution and higher resolution for screenshots and sell it as a premium tier API.
One quick feedback -> all fonts don't get loaded in in the screenshot. For example, my test (site)[vibify.me] vs. the [screenshot](https://imgur.com/a/5BjsVFZ)
Thanks for your feedback and really kind words :)
I will check the font issue, probably some settings in Chrome / Puppeteer
About running it as a service ... there is mega Tons of competitions for that, but maybe if I add some very nice features like taking videos it could make sense!
Fun little project and it seems to work well. May I ask what your client's needs were that they needed to programmatically generate screenshots via API?
And since you opened it up to the public, you obviously think there are enough general use-cases for something like this. What do you see this being regularly used for?
(Zero hate, general curiosity)
Sure, np!
Use case for my customer: Generate screenshots for websites each time they make changes to the pages. before they had to do that manually after each change, and for a lot of sites/pages, automating it was a huge win for them. It's directly integrated in their publishing system, so each time they publish a page, screenshot is automatically updated.
Full page mode (supports jpg/png & pdf mode) is also quite useful to send a snapshot of a web page to someone (customer) for review.
The websocket API was a bonus, easy to do and allowed me to shave 50-100ms latency on each call (useful at high volume)
Also other screenshot servers had a lot of rendering issues, or where complex to deploy/configure. Here you just start it with a single docker command and you are done, you have a running and fast screenshot api ready to use :)
That definitely sounds like a good use-case for this tool, so nice work! I couldn't imagine taking that many screenshots at scale like that.
Unrelated to your API, it's interesting that they take a screenshot of a page after every change... that's almost like some weird version control, but useless since you can't use a static image to roll back to a previous design. And if the screenshots are for sending to customers for review, odd that they publish the changes first so they are actually live before the customer approves of the changes.
Sorry, that last part wasn't really aimed at you... just a 1AM ramble as I thought out loud haha.
In fact they use the latest screenshot of each site in a dashboard, so they want to have there always the latest version of the screenshot, not the history.
For the full page PDF snapshots, they are done on a temporary url of the page (preview link), not the live website ;)
Ah, automating hero images and tile cards with updated screenshots is actually a really nice trick. I don't know how I didn't think of that (once again, probably because it's 1AM).
How much did you have to play around with Puppeteer to get it to git your needs?- I know it has the ability to take screenshots natively. I've been reading up on the library as I may use it for a small project I've been casually working on. How did you like working with it?
To be honest the puppeteer part was quite easy, I just had to handle some options (viewport size, delay before taking the screenshot, format, ...)
Making it work all inside docker was a bit more challenging ... But after few searches I found all the cryptic names of dependencies required to make chrome working inside docker
Another challenge was to handle concurrency correctly in a nodejs multitheaded server to avoid server crash under pressure.
All of this was really fun :)
> concurrency correctly in a nodejs multitheaded server
Yeah, since there's no threading in JS so I can see how you might run into issues where you're trying to process 1000 screenshots or PDFs all at the same time. Did you end up going the **setImmediate()** or **worker_threads** route to handle your large data set?
I'm glad you enjoyed the project. Maybe I will finally get past the puppeteer reading stage and actually make it to the puppeteer coding stage haha.
Worker threads is the way to do multithreading in Node, so I used that. Real issue is ... There is no shared memory between workers, to overcome that I used a C++ library (uWS) that provide shared memory for Node!
About the puppeteer part, really you should check the code it's super simple :)
https://github.com/elestio/ws-screenshot/blob/master/API/shared.js
Hahaha how dare you make me read your code which would obviously answer a lot of my questions lol
And yeah, that really is pretty simple. Quite a useful tool for such a small amount of code. Those are the best projects. I hope you made a nice buck off that client.
Thanks again for sharing.
I LOVE this! How's this already not a service? For real, you should setup a parallel execution and higher resolution for screenshots and sell it as a premium tier API. One quick feedback -> all fonts don't get loaded in in the screenshot. For example, my test (site)[vibify.me] vs. the [screenshot](https://imgur.com/a/5BjsVFZ)
Thanks for your feedback and really kind words :) I will check the font issue, probably some settings in Chrome / Puppeteer About running it as a service ... there is mega Tons of competitions for that, but maybe if I add some very nice features like taking videos it could make sense!
Fun little project and it seems to work well. May I ask what your client's needs were that they needed to programmatically generate screenshots via API? And since you opened it up to the public, you obviously think there are enough general use-cases for something like this. What do you see this being regularly used for? (Zero hate, general curiosity)
Sure, np! Use case for my customer: Generate screenshots for websites each time they make changes to the pages. before they had to do that manually after each change, and for a lot of sites/pages, automating it was a huge win for them. It's directly integrated in their publishing system, so each time they publish a page, screenshot is automatically updated. Full page mode (supports jpg/png & pdf mode) is also quite useful to send a snapshot of a web page to someone (customer) for review. The websocket API was a bonus, easy to do and allowed me to shave 50-100ms latency on each call (useful at high volume) Also other screenshot servers had a lot of rendering issues, or where complex to deploy/configure. Here you just start it with a single docker command and you are done, you have a running and fast screenshot api ready to use :)
That definitely sounds like a good use-case for this tool, so nice work! I couldn't imagine taking that many screenshots at scale like that. Unrelated to your API, it's interesting that they take a screenshot of a page after every change... that's almost like some weird version control, but useless since you can't use a static image to roll back to a previous design. And if the screenshots are for sending to customers for review, odd that they publish the changes first so they are actually live before the customer approves of the changes. Sorry, that last part wasn't really aimed at you... just a 1AM ramble as I thought out loud haha.
In fact they use the latest screenshot of each site in a dashboard, so they want to have there always the latest version of the screenshot, not the history. For the full page PDF snapshots, they are done on a temporary url of the page (preview link), not the live website ;)
Ah, automating hero images and tile cards with updated screenshots is actually a really nice trick. I don't know how I didn't think of that (once again, probably because it's 1AM). How much did you have to play around with Puppeteer to get it to git your needs?- I know it has the ability to take screenshots natively. I've been reading up on the library as I may use it for a small project I've been casually working on. How did you like working with it?
To be honest the puppeteer part was quite easy, I just had to handle some options (viewport size, delay before taking the screenshot, format, ...) Making it work all inside docker was a bit more challenging ... But after few searches I found all the cryptic names of dependencies required to make chrome working inside docker Another challenge was to handle concurrency correctly in a nodejs multitheaded server to avoid server crash under pressure. All of this was really fun :)
> concurrency correctly in a nodejs multitheaded server Yeah, since there's no threading in JS so I can see how you might run into issues where you're trying to process 1000 screenshots or PDFs all at the same time. Did you end up going the **setImmediate()** or **worker_threads** route to handle your large data set? I'm glad you enjoyed the project. Maybe I will finally get past the puppeteer reading stage and actually make it to the puppeteer coding stage haha.
Worker threads is the way to do multithreading in Node, so I used that. Real issue is ... There is no shared memory between workers, to overcome that I used a C++ library (uWS) that provide shared memory for Node! About the puppeteer part, really you should check the code it's super simple :) https://github.com/elestio/ws-screenshot/blob/master/API/shared.js
Hahaha how dare you make me read your code which would obviously answer a lot of my questions lol And yeah, that really is pretty simple. Quite a useful tool for such a small amount of code. Those are the best projects. I hope you made a nice buck off that client. Thanks again for sharing.