Skip to main content

Anonymous wget

More
20 years 1 month ago #5424 by biund
Anonymous wget was created by biund
I would like to rip some content from a site using wget, by gathering a bunch of generated pages containing the data that interests me. Each page is of just a few KB.

I am thinking of running a perl script doing basically the following.
[code:1]
for $i ( 1 .. 5000 ) {
system("wget -O $i.htm http://thesite.com/content/?id=$i");
sleep 5; # just a few secs not to kill the server
}[/code:1]

I would like to proceed by running the script, but for various reasons I'd like to do it anonymously i.e. making the target site unaware of the origin of the requests.

Any ideas about that might be arranged?
More
20 years 1 month ago #5427 by nske
Replied by nske on topic Re: Anonymous wget
you can set the $http_proxy enviromental var to an anonymous proxy's address and use wget with --proxy=on parameter.
More
20 years 1 month ago #5429 by biund
Replied by biund on topic Re: Anonymous wget
Thanks.

Any hints on where I can find a truly anonymous proxy server appreciated.
More
20 years 1 month ago #5430 by nske
Replied by nske on topic Re: Anonymous wget
there are many in www.stayinvisible.com as well as a service to check whether they are indeed anonymous ;)
More
20 years 1 month ago #5441 by biund
Replied by biund on topic Re: Anonymous wget
Thanks a lot, nske. Amazing :shock:

Exactly what I was looking for. stayinvisible.com seems just right - others I've seen not as good.
Time to create page: 0.132 seconds