Jump to content

Proxy support on multi curl


calebf

Recommended Posts

I have an array of proxies and I need one to be chosen randomly and applied to each curl handle. This script runs successfully when proxy support is removed, but as soon as I try to add a proxy it refuses to connect. My proxies are good. I have private proxies that validate by confirming that the IP address of the requester is on a whitelist of IPs I configured. I've also tried working public proxies with no validation, with the same result. The error it returns is that it "cannot connect to google.com:3238". Why it's trying to connect to that port is beyond me, the proxies I use, their port is 3238, but it should be connecting to IP:PORT then Google.com:80. Not sure why it's mixing them up.  Here's my code:

 

function multi($urls,$proxies,$agents,$referer) {

// create the multi curl handle
$mh = curl_multi_init();
$handles = array();
$i = 0;
foreach ($urls as $url) {
	// create a new single curl handle
	$ch = curl_init();

	// setting several options like url, timeout, returntransfer
	// simulate multithreading by calling the wait.php script and sleeping for $rand seconds
	curl_setopt($ch, CURLOPT_URL, $url);
	curl_setopt($ch, CURLOPT_HEADER, 1);
	curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
	curl_setopt($ch, CURLOPT_PROXY, $proxies[rand(0,count($proxies))]);
	curl_setopt($ch, CURLOPT_HTTPPROXYTUNNEL, 1);
	curl_setopt($ch, CURLOPT_COOKIEJAR, 'cookies.txt');
	curl_setopt($ch, CURLOPT_COOKIEFILE, 'cookies.txt');
	curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 30);
	curl_setopt($ch, CURLOPT_REFERER, $referer);
	curl_setopt($ch, CURLOPT_USERAGENT, $agent);

	// add this handle to the multi handle
	curl_multi_add_handle($mh,$ch);

	// put the handles in an array to loop this later on
	$handles[] = $ch;
	$i++;
}

// execute the multi handle
$running=null;
do {
	curl_multi_exec($mh,$running);
	// added a usleep for 0.25 seconds to reduce load
	usleep (250000);
} while ($running > 0);

echo 'Curl error: ' . curl_error($ch)."<br />".print_r(curl_getinfo($ch));

// get the content of the urls (if there is any)
for($i=0;$i<count($handles);$i++) {
	// get the content of the handle
	$output[] = curl_multi_getcontent($handles[$i]);

	// remove the handle from the multi handle
	curl_multi_remove_handle($mh,$handles[$i]);
}
// close the multi curl handle to free system resources
curl_multi_close($mh);

// echo the output to the screen
return $output;
}

 

I only post on forums as a last resort. Typically a few Google searches solves the problem, but I've spent too much time researching this one with no progress. I'm assuming it's simply a parameter that needs to be added to the CURL handles, but I've tried every proxy related config option out there and still no dice.

Link to comment
Share on other sites

This thread is more than a year old. Please don't revive it unless you have something important to add.

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...

Important Information

We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue.