angel1987 Posted February 28, 2012 Share Posted February 28, 2012 I tried searching on google but couldn't find any relevant information, please redirect me to relevant source or help me with the code. I want to pass a domain name in text field which will be scanned and then the script will display entire site map. Not external links or links on a page. Sorry it is not easy for me to explain. Eg. if i pass abc.com Script will display abc.com/12/adn.php abc.com/asd/asd/ etc Whatever their url format is. All the links on that domain. Quote Link to comment Share on other sites More sharing options...
batwimp Posted February 28, 2012 Share Posted February 28, 2012 Your going to need to use Screen Scraping (google "PHP screen scraping") and some regular expressions (which should be included in a screen scraping tutorial). There are a lot of tutorials out there on screen scraping. Quote Link to comment Share on other sites More sharing options...
angel1987 Posted February 29, 2012 Author Share Posted February 29, 2012 But how will screen scraping give all the links associated with that domain name on that website? I searched on google but every article talks about fetching a part of the content on a web page from the website. I think i need to do something like a xml site map generator where it crawls website and displays all the links. Any tutorial for that? Quote Link to comment Share on other sites More sharing options...
batwimp Posted February 29, 2012 Share Posted February 29, 2012 Sorry. I thought you wanted to grab the links from the pages themselves. If you want to do a site search for links, there is probably a way to do a customized search from google, but I don't know how, exactly. Anybody else here know? Quote Link to comment Share on other sites More sharing options...
angel1987 Posted March 2, 2012 Author Share Posted March 2, 2012 I basically wanted to scan an entire website to see if it has any adult content or not. So i wanted to use PHP to scan for links and pages. Quote Link to comment Share on other sites More sharing options...
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.