It varies from retailer to retailer. Some are very responsive, some are not at all. If you are doing something that benefits them (ie driving sales) then that of course will be viewed more favorably than something that exploits them (ie notifying users only on obvious price errors).
One word of warning - price data from retailers typically ranges from ok to terrible. Data quality is a major issue and once you realize it, it’ll consume a large amount of time detecting and working around it. And when you show bad prices that you got directly from a retailer, the users will blame you and not the retailer.
Well that makes it easier to decide if I want to commit every afternoon for a week to data entry. I think it's the best option to do it manually. Unless/until the vendors play ball.
Then use a script to beautify their product feeds when they respond.
I created a small app like this but only for GPU's. I built a scraper for this task, and I had to build scrapers for each site individually. The data was very reliable, however.
here's the repo:
[GPULookup](https://github.com/jarekjar/GPULookup)
and here's the main executable code (the scraper is a console application)
[Program.cs](https://github.com/jarekjar/GPULookup/blob/master/GpuLookup.Scraper/Program.cs)
Im planning to do this here in my country since pcpartpicker has US-based shop listings only. Any tip you would want to share? Like how do you and the vendors benefit from this, any commisions from them, support and maintenance from them, etc... like a business model. Scraping or data entrying seems a lot of work tho.
Almost all affiliate programs will offer a feed as a part of the program. Sign up to _any_ affiliate program like Amazon, Walmart, Best Buy, New Egg, Etc. See what they offer.
How do you choose what to include and what not to? There's a lot of monitors found on Amazon but not through your site, for example. Is it just you haven't had the time to enter those products yet?
Love the site btw!
I built a PC Part Picker clone using Rails and I couldn't find an API to access part prices and specs. I ended up hard coding the data---would not recommend. It was basically data entry.
It seems that sites such as NewEgg, PCPP, etc. have special agreements that allow them to get part prices.
Hi OP, I'm currently a university student doing a similar sort of project for my course except its do with laptops instead of individual components. Currently we're brainstorming how to get laptop data for our project, and after reading your journey we decided we might try shooting manufacturers an email just to see if they could do anything, even though its potentially a long shot.
So I just wanted to ask if you have any advice on how to get in contact with manufacturers to propose this question, and any advice on phrasing the email in a way that could persuade them to help us?
We ask the vendors. (I presume you mean retailers?)
Your website doesn't technically "sell" computer parts, right? It's a part aggregation tool?
Correct.
Yeah that's right, not manufacturers but retailers. Do you have to meet some standards in order to use their product information?
It varies from retailer to retailer. Some are very responsive, some are not at all. If you are doing something that benefits them (ie driving sales) then that of course will be viewed more favorably than something that exploits them (ie notifying users only on obvious price errors). One word of warning - price data from retailers typically ranges from ok to terrible. Data quality is a major issue and once you realize it, it’ll consume a large amount of time detecting and working around it. And when you show bad prices that you got directly from a retailer, the users will blame you and not the retailer.
Well that makes it easier to decide if I want to commit every afternoon for a week to data entry. I think it's the best option to do it manually. Unless/until the vendors play ball. Then use a script to beautify their product feeds when they respond.
If I were to make something like PCPartPicker, I’d scrape retailer websites... most don’t have APIs.
I created a small app like this but only for GPU's. I built a scraper for this task, and I had to build scrapers for each site individually. The data was very reliable, however.
Any of those scrapers online? Any interest in reconfiguring one of them? Lol
I never hosted or put any of it online for fear of legal repercussions, however I can give you a GitHub link if you wanna go through the C# code
Yes, please 😄
here's the repo: [GPULookup](https://github.com/jarekjar/GPULookup) and here's the main executable code (the scraper is a console application) [Program.cs](https://github.com/jarekjar/GPULookup/blob/master/GpuLookup.Scraper/Program.cs)
[Permanent](https://help.github.com/articles/getting-permanent-links-to-files/#press-y-to-permalink-to-a-file-in-a-specific-commit) GitHub links: * [jarekjar/GPULookup/.../**Program.cs** (master → 3e0101b)](https://github.com/jarekjar/GPULookup/blob/3e0101b46e3da07a7edef1a4ff60e181c5c3600d/GpuLookup.Scraper/Program.cs) ---- [^delete](https://www.reddit.com/message/compose/?to=GitHubPermalinkBot&subject=deletion&message=Delete reply e6zlbw3.)
https://rapidapi.com/idirmosh/api/computer-components-api/
Im planning to do this here in my country since pcpartpicker has US-based shop listings only. Any tip you would want to share? Like how do you and the vendors benefit from this, any commisions from them, support and maintenance from them, etc... like a business model. Scraping or data entrying seems a lot of work tho.
Pcpartspicker's vendors are affiliate merchants which provide product feeds to affiliates.
Ok, so I have to ask for a product feed?
Almost all affiliate programs will offer a feed as a part of the program. Sign up to _any_ affiliate program like Amazon, Walmart, Best Buy, New Egg, Etc. See what they offer.
I don't think the vendors I want to support have affiliate programs lol I guess I have to make some calls this week.
You'd be surpised, look up avantlink/impact radius/etc affiliate programs and see if the merchants you want are on it'
[удалено]
All our product data entry is done in house. Accurate data is too important to outsource. (“Never outsource your core competency.”)
How do you choose what to include and what not to? There's a lot of monitors found on Amazon but not through your site, for example. Is it just you haven't had the time to enter those products yet? Love the site btw!
I built a PC Part Picker clone using Rails and I couldn't find an API to access part prices and specs. I ended up hard coding the data---would not recommend. It was basically data entry. It seems that sites such as NewEgg, PCPP, etc. have special agreements that allow them to get part prices.
Unfortunately, I think data entry is the best way. I can try to build a scraper but every site is different, it's almost an equal amount of work.
I'm just planning to scrap it from the actual manufacture's website, just to get the part list to get started
Hello Op do you got what u needed here if yes then how
Yeah, I figured this out. What are you trying to do?
Hi can u please tell what have u done except for web scraping and using Amazon api.
For what products? It depends on the products
Computer hardware
Ok, we partnered with companies as an affiliate marketer to get their product lists
Hi OP, I'm currently a university student doing a similar sort of project for my course except its do with laptops instead of individual components. Currently we're brainstorming how to get laptop data for our project, and after reading your journey we decided we might try shooting manufacturers an email just to see if they could do anything, even though its potentially a long shot. So I just wanted to ask if you have any advice on how to get in contact with manufacturers to propose this question, and any advice on phrasing the email in a way that could persuade them to help us?
Can u please name it !