Automated HTTP GET (with cookies) in Curl

• Problem: I have a list of url stored  in a file /tmp/myurls and needs to do a HTTP GET (with cookies) for each url wiithin the file.
• Solution: Here is how I use curl in a perl script to do a HTTP GET per ul.  At first I create a cookie from the first url, then hit each one with the cookies. Of course there will be more error checking for creating the cookies such as the first url is not working.
• Impression: Curl serves pretty well with -c to store cookies in a file, -b to use cookies from a file in this case. Python can be used to do the similar thing and I can keep the cookies in memory instead of reading it every time. However Python seems unknown/unpopular to the sys admins here, so it is not even an option. Writing C with libcurl is another option as well.

./hits.pl /tmp/myurls myhost.mydomain.com /tmp/cookies

hits.pl

=================

#!/usr/bin/perl -w

$host=$ARGV[1];
$cookiesFile=$ARGV[2];
open( INPUT, “<$ARGV[0]” );
$init=1;
while( $line = <INPUT> ){
$url=’http://&#8217;.$host.$line;
print $url,”n”;
# only need to get the cookies for the first time
if ($init) {
system “curl -c  $cookiesFile -s -o /dev/null “$url””;
$init=0;
}
system “curl -b  $cookiesFile -s -o /dev/null “$url””;
}
close INPUT;