world of tanks

[Taiphoz]

Hay guys..

 

Im not the best php coder for a start, I am trying to uses getfilecontent or some other php function to grab our clans roster from the official world of tanks website.

 

The problem is that they seem to populate the roster after the page is loaded via jQuery and java.

 

Is there any way with php for me to get access to this roster or is the fact their doing it like that a permanent block.

 

the roster is here if anyone wants to have a crack at it http://game.worldoftanks.eu/clans/500000156/

[gizmola]

You might want to explore this:  The actual url being called in their ajax is:  http://game.worldoftanks.eu/clans/500000156/members/?type=table&offset=0&limit=100&order_by=name&search=&echo=1&id=clan_members_index

 

This returns this json which is the actual data you seek:

 

{"request_data":{"items":[{"member_since":"19.04.2011","role":"Recruit","account_id":500516908,"name":"Ashfur"},{"member_since":"20.04.2011","role":"Recruit","account_id":500302252,"name":"Babalou"},{"member_since":"24.04.2011","role":"Recruit","account_id":500511100,"name":"Brison"},{"member_since":"05.05.2011","role":"Recruit","account_id":500623661,"name":"Brutstorm"},{"member_since":"19.04.2011","role":"Recruit","account_id":500479102,"name":"Burningwaflz"},{"member_since":"17.04.2011","role":"Recruit","account_id":500153964,"name":"captainbob"},{"member_since":"17.03.2011","role":"Recruit","account_id":500027201,"name":"Crookshanks"},{"member_since":"08.05.2011","role":"Recruit","account_id":500407454,"name":"Dalen"},{"member_since":"13.04.2011","role":"Recruit","account_id":500494914,"name":"Darazun"},{"member_since":"08.05.2011","role":"Recruit","account_id":500473624,"name":"dourdog"},{"member_since":"15.03.2011","role":"Commander","account_id":500022488,"name":"Dvalin"},{"member_since":"16.03.2011","role":"Recruit","account_id":500058767,"name":"DyslexAcon"},{"member_since":"27.04.2011","role":"Recruit","account_id":500569624,"name":"frakkedup"},{"member_since":"16.04.2011","role":"Recruit","account_id":500397254,"name":"guardsmanlee"},{"member_since":"17.04.2011","role":"Recruit","account_id":500210482,"name":"He11Bringer"},{"member_since":"22.04.2011","role":"Recruit","account_id":500517696,"name":"horror"},{"member_since":"20.04.2011","role":"Recruit","account_id":500061931,"name":"Janoowawawa"},{"member_since":"22.04.2011","role":"Recruit","account_id":500030603,"name":"jazeera"},{"member_since":"09.05.2011","role":"Recruit","account_id":500348886,"name":"Kinglewis"},{"member_since":"15.04.2011","role":"Recruit","account_id":500071210,"name":"M3ssi"},{"member_since":"01.04.2011","role":"Recruit","account_id":500024655,"name":"MISGUIDED"},{"member_since":"16.04.2011","role":"Recruit","account_id":500162520,"name":"mixiarse"},{"member_since":"18.04.2011","role":"Recruit","account_id":500526022,"name":"MortisLock"},{"member_since":"06.05.2011","role":"Recruit","account_id":500497558,"name":"ninjagordy"},{"member_since":"20.04.2011","role":"Recruit","account_id":500298061,"name":"pastykiller"},{"member_since":"21.04.2011","role":"Recruit","account_id":500086666,"name":"purazz"},{"member_since":"17.04.2011","role":"Recruit","account_id":500510102,"name":"RB26DETT_CZ"},{"member_since":"16.03.2011","role":"Recruit","account_id":500027254,"name":"remmster"},{"member_since":"16.04.2011","role":"Recruit","account_id":500399768,"name":"Tomasu41"},{"member_since":"08.05.2011","role":"Recruit","account_id":500635276,"name":"Whoepsie"}],"total_count":30,"filtered_count":30,"offset":0,"echo":1},"result":"success"}

[Taiphoz]

hay m8, as I said my php isnt amazing just enough to bodge stuff together, can you explain what you did to get that info from the page ? when I check that url i get a 404.

[Taiphoz]

this is what I tried after a little hunt on google about json.

 

    $curlSession = curl_init();
    curl_setopt($curlSession, CURLOPT_URL, 'http://game.worldoftanks.eu/clans/500000156/members/?type=table&offset=0&limit=100&order_by=name&search=&echo=1&id=clan_members_index');
    curl_setopt($curlSession, CURLOPT_BINARYTRANSFER, false);
    curl_setopt($curlSession, CURLOPT_RETURNTRANSFER, true);

    $jsonData = json_decode(curl_exec($curlSession));
    curl_close($curlSession);
    var_dump($jsonData);

[gizmola]

Taiphoz:  I got it from using firebug to look at the net connections.  Probably in order to get the site to respond to your request you'll have to experiment with setting some headers.  The curl library extension curl is often used for this type of thing.  Here's a couple of headers in the request: 

 

Content-Type: application/x-www-form-urlencoded

X-Requested-With: XMLHttpRequest

Referer: http://game.worldoftanks.eu/clans/500000156/

 

They also might require that you have acquired a session cookie by visiting the refered page. 

 

[Taiphoz]

Ah that did the trick. thanks for the help m8 without the tip about setting headers I would probably have never gotten this to work.

[animator]

Can you help me with this ? I would like to create something similiar but i dont know how to do it. I used the curl commands but all I get is 404 - Not Found...

 

I can get with firebug everything also with the XHR but how can I do it with PHP ?

[ignace]

Set the X-Requested-With header on your cURL object, anything else without the header will get a 404.

 

Edit: nevermind I just noticed they use a X-CSRFToken which means that unless you know the token you can't get the information.

[animator]

I know the token from firebug:

 

X-CSRFToken: 2b4f94f8d03e0b00ea81e99fe90d53f1

 

But when i use this in header it doesnt work. Still 404.

 

This is what i tried with no success:

 

<?php
$url='http://uc.worldoftanks.eu/clans/500000156/members/?type=table&offset=0&limit=100&order_by=name&search=&echo=1&id=clan_members_index';
print_r(get_data($url)); 

function get_data($url)
{
$ch = curl_init();
$timeout = 5;
curl_setopt($ch,CURLOPT_URL,$url);
curl_setopt($ch,CURLOPT_RETURNTRANSFER,1);
curl_setopt($ch,CURLOPT_CONNECTTIMEOUT,$timeout);
curl_setopt($ch, CURLOPT_HTTPHEADER, array(
"Host: uc.worldoftanks.eu",
"User-Agent: Mozilla/5.0 (Windows NT 6.1; rv:2.0.1) Gecko/20100101 Firefox/4.0.1" ,
"Accept: application/json, text/javascript, */*; q=0.01" ,
"Accept-Language: en-us,en;q=0.5" ,
"Accept-Encoding: gzip, deflate",
"Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7",
"Keep-Alive: 115"     ,
"Connection: keep-alive",
"X-Requested-With: XMLHttpRequest" ,
"X-CSRFToken: 2b4f94f8d03e0b00ea81e99fe90d53f1" ,
"Referer: http://uc.worldoftanks.eu/uc/clans/500000842/",
"Cookie: __utma=47689720.699525485.1297685819.1297685819.1303296267.2; __utmz=47689720.1297685819.1.1.utmcsr=game.worldoftanks.com|utmccn=(referral)|utmcmd=referral|utmcct=/account/login/; csw_popup=true; csrftoken=2b4f94f8d03e0b00ea81e99fe90d53f1; __utma=21812543.945991289.1307194162.1307194162.1307194162.1; __utmb=21812543.13.10.1307194162; __utmc=21812543; __utmz=21812543.1307194162.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none)"

));
$data = curl_exec($ch);
curl_close($ch);
return $data;
}
?>

[ignace]

You should allow cURL to create and send cookies as I noticed

 

csrftoken=2b4f94f8d03e0b00ea81e99fe90d53f1

 

In your cookie header. I think CURLOPT_COOKIEJAR does that.

[animator]

Could you please give me an example how to do it ? I tried the cookiejar but still 404. Maybe i am doing something wrong. I am not so skilled in PHP with cURL.