Archive for September 27th, 2008
setting the user agent in php curl
In : PHP, Posted by admin on Sep.09, 2008
I am trying to write a spider to scraoe Google result. Google returns different content for differnet user agent. Here is how to setup the user agent in curl.
$url=’http://www.google.com/search?hl=en&q=’.$keyword.’&start=10′;
$ch = curl_init($url);
$useragent=”Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.1) Gecko/20061204 Firefox/2.0.0.1″;
curl_setopt($ch, CURLOPT_HEADER, 1);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_USERAGENT, $useragent);