This post was most recently updated on December 11th, 2018
Download a URL’s Content Using PHP cURL
Downloading content at a specific URL is common practice on the internet, especially due to increased usage of web services and APIs offered by Amazon, Alexa, Digg, etc. PHP’s cURL library, which often comes with default shared hosting configurations, allows web developers to complete this task.
The Code
copy/* gets the data from a URL */function get_data($url){ $ch = curl_init(); $timeout = 5; curl_setopt($ch,CURLOPT_URL,$url); curl_setopt($ch,CURLOPT_RETURNTRANSFER,1); curl_setopt($ch,CURLOPT_CONNECTTIMEOUT,$timeout); $data = curl_exec($ch); curl_close($ch); return $data;}
The Usage
copy$returned_content = get_data('http://davidwalsh.name');
Alternatively, you can use the file_get_contents function remotely, but many hosts don’t allow this.
Set the User Agent With PHP cURL
A few months back, I shared with you how to download the contents of a URL and execute a HTTP POST transmission using PHP cURL. For security purposes, some hosts require that a common user agent be present in the POST. If an unacceptable user agent is given, the POST is ignored. Luckily, cURL allows us to “spoof” the server using any user agent we choose:
copycurl_setopt($ch,CURLOPT_USERAGENT,'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.13) Gecko/20080311 Firefox/2.0.0.13');
A little sneaky but it could get you out of a jam at some point!