- Posts: 3
- Thank you received: 0
Anonymous wget
20 years 1 month ago #5424
by biund
Anonymous wget was created by biund
I would like to rip some content from a site using wget, by gathering a bunch of generated pages containing the data that interests me. Each page is of just a few KB.
I am thinking of running a perl script doing basically the following.
[code:1]
for $i ( 1 .. 5000 ) {
system("wget -O $i.htm http://thesite.com/content/?id=$i");
sleep 5; # just a few secs not to kill the server
}[/code:1]
I would like to proceed by running the script, but for various reasons I'd like to do it anonymously i.e. making the target site unaware of the origin of the requests.
Any ideas about that might be arranged?
I am thinking of running a perl script doing basically the following.
[code:1]
for $i ( 1 .. 5000 ) {
system("wget -O $i.htm http://thesite.com/content/?id=$i");
sleep 5; # just a few secs not to kill the server
}[/code:1]
I would like to proceed by running the script, but for various reasons I'd like to do it anonymously i.e. making the target site unaware of the origin of the requests.
Any ideas about that might be arranged?
20 years 1 month ago #5427
by nske
Replied by nske on topic Re: Anonymous wget
you can set the $http_proxy enviromental var to an anonymous proxy's address and use wget with --proxy=on parameter.
20 years 1 month ago #5429
by biund
Replied by biund on topic Re: Anonymous wget
Thanks.
Any hints on where I can find a truly anonymous proxy server appreciated.
Any hints on where I can find a truly anonymous proxy server appreciated.
20 years 1 month ago #5430
by nske
Replied by nske on topic Re: Anonymous wget
there are many in
www.stayinvisible.com
as well as a service to check whether they are indeed anonymous ;)
Time to create page: 0.132 seconds