I sorry I didn’t make my position clear. I have absolutely no problem with scraping, I do not consider scraping black hat at all. I only meant to say that violating the TOS of a website is not exactly a white hat.
It seems Google has no problems with scraping, they build websites based on scraped content as well as ranking websites with scraped content in their search index. It’s not the act of scraping that violates their terms, it is the excessive automated queries that have the potential to shut down the service when abused.
Google also follows industry standards for ethical scraping practices, they only scrape and publish small snippets and always provide attribution to the source.
Furthermore, Google seems to have no problem with scraping from their websites as long as you are not abusing their service with excessive automated queries and provide proper attribution.
It’s the excessive automated queries to services that expressly bar such activities that are the common abusive technique that makes most of what SB does cross what I consider to be a very bright line of what is ethical. Otherwise, why would you need to use so many proxies and why are those proxies so consistently blocked after use by SB?
You seemed to be rationalizing this abuse by saying that you follow white hat SEO methods with data you acquired through arguably unethical practices. I’m not saying everything you do is black hat, I’m just saying that much of what SB does crosses some sort of ethical boundary.
To use an analogy, it’s a little like saying I use the money from bank robberies to feed the poor, therefore when I rob a bank it is completely ethical.
Finally, let me say that I do believe that there are some things that you can use SB for that are completely ethical, just that nearly all of it’s most useful tools are crossing lines of ethics and therefore not considered white hat in my opinion.
