Use CURL with PHP to Handle HTTP Requests and Create Useful Scripts

In an not so old tutorial, i’ve described how one can use urllib in Python to fetch url data and write cunning little scripts that automate online works. Sometimes, especially when you need to server some content to other users via a website, you will need to write such programs in php. Learning how to use curl is a pretty nice idea overall, since there are wrappers for the famous libcurl library at almost every language you can imagine. Therefore, you will sooner or later be using that.

Thankfully, curl fits very nicely with PHP. The good idea about libcurl is that you just create a curl handle object and then you can edit any option related to curl at will. You want to post data ? Can be done by just setting a flag and specifying that data. Next you just want to get a post. Just remove the previously set flag. Want to add some cookies to the whole routine ? Once again, it’s a matter of one command. Let’s now see together some real life examples of using curl with PHP:

Use CURL to login to Facebook

I always prefer to use real life examples when explaining how to write some code. In this one, you will be presented with a function that automatically logs you in facebook. Whenever you log to such websites automatically, at 99% percent of situations, you will have to use cookies. Luckily, doing so is pretty easy with curl. You just call curl_setopt using the cookiejar and cookiefile parameters, as long as the cookie name. Take a look at the code and i will explain it thoroughly below :

 $login_email = 'name@mail.com';
 $login_pass = 'password';

 $ch = curl_init();
 curl_setopt($ch, CURLOPT_URL, 'http://www.facebook.com/login.php');
 curl_setopt($ch, CURLOPT_POSTFIELDS,'email='.urlencode($login_email).'&pass='.urlencode($login_pass).'&login=Login');
 curl_setopt($ch, CURLOPT_POST, 1);
 curl_setopt($ch, CURLOPT_HEADER, 0);
 curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
 curl_setopt($ch, CURLOPT_COOKIEJAR, "/tmp/my_cookies.txt");
 curl_setopt($ch, CURLOPT_COOKIEFILE, "/tmp/my_cookies.txt");
 curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
 curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.3) Gecko/20070309 Firefox/2.0.0.3");
 curl_setopt($ch, CURLOPT_REFERER, "http://www.facebook.com");
 $page = curl_exec($ch);

As you see, we first initialize curl and get the connection object. Then, we use the important curl_setopt function to specify some important parameters. The first parameter is, of course, the url that we want to connect to. Then, follow the parameters that we want to post to the login form, in the form of “parameter = value&parameter2=value2&parameter3=value3″, as a simple string. We specify that we want to post these values to the form when we set the flag CURLOPT_POST to 1 (meaning true). CURLOPT_HEADER can be set to 0 since we don’t really need the headers of the request. The FOLLOWLOCATION parameter is just used in case of page redirects. If the page we request redirects to another page, we get that page automatically. Therefore, it’s a good addition, especially when we look at login functions. Specifying the user agent is just a way to masquarade your script, so that the remote server thinks you’re Mozilla Firefox. This is useful, if you’re a bit on the grey or dark side of hats Finally, we specify the referer so that facebook thinks we first visited the main page. Right after, we execute the http request. Notice the RETURNTRANSFER = 1 parameter. When set, curl_exec, instead of just returning a simple true or false value, it returns the whole page it retrieved. This one is really really useful.

Please bear in mind that you may have to create the cookie file yourself before calling curl_exec. Not doing so might trigger a failure on running this code at the first time. After that, in the second execution, the file will have been created and no problem will occur again. To be sincere, i don’t really know why this happens, but i’ve encountered it, so please watch out for it.

Keep the Requests Going

Now that we have logged in, we can keep on with our requests, getting more pages or doing other nasty stuff. Just set the parameters accordingly. Want to retrieve a facebook page where you do not need to post anything, like the home one ? Just reset the CURLOPT_POST parameter, specify the cookie that holds our login information and execute a new curl_exec that will fetch you that page.

curl_setopt($ch, CURLOPT_URL,"$some_url");
 curl_setopt($ch, CURLOPT_COOKIEFILE, "/tmp/my_cookies.txt");
 curl_setopt($ch, CURLOPT_POST, 0);
 $page = curl_exec($ch);

I hope that you will have fun with curl and that you use it for automating important stuff. Use it sensibly

If you enjoyed this post, make sure you subscribe to my RSS feed!

Random Posts

• Code a Twitter Auto Post Script for Wordpress Blogs in Python
• 5. C++ Looping Structures
• Create And Reverse Simple Substitution Ciphers For Fun (and Profit ?)
• 15 Great and FREE Online Resources That Every Programmer Should Know About
• An Introduction to Unix Commands
• Windows API Programming Introduction
• 5 Reasons Why You Should Learn Python Programming
• Introduction to Reverse Engineering and Software Cracking
• Top 10 PHP Techniques That Will Save You Time and Effort
• How to Create And Use Makefiles