Lexicon Posted January 6, 2012 Share Posted January 6, 2012 Im trying to make a php program that will grab all the user's tweets they have ever done.. I'm using curl to request the twitter page and scrape the html. It looks as though the maximum amount of tweets I can get is 20, which is the amount on the initial first page. There is some script that loads another 20 tweets once I scroll to the bottom of the page... Does anyone know how to make the page automatically load a certain amount so I can scrape more than the first 20 at once. Quote Link to comment Share on other sites More sharing options...
wolfcry Posted January 6, 2012 Share Posted January 6, 2012 Can you post some of your code so we can take a look? Does it have some sort of counter that loops (which I'm assuming it does) until reaching a maximum limit? Quote Link to comment Share on other sites More sharing options...
Lexicon Posted January 6, 2012 Author Share Posted January 6, 2012 I basically use my curl "wrapper" to send a web request to load the twitter page and get all the html. If check someone's profile on twitter, you will see an intitial page. But once you scroll down u can pretty much scroll down forever and it just keeps loading more and more content. Quote Link to comment Share on other sites More sharing options...
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.