PHP Page loading forever

[Ninjakreborn]

I have a PHP Page.  It is doing a lot of site crawling, throughout a lot of loops.

Basically it's getting a massive array together and it's huge.  However, it's so big that PHP times out beyond

belief. I mean there is no error or nothing. Google throws a major error about page not found, IE shows an internet connection error. It really

times out big time. Even though it's not really "A LOT". Any advice on what to do in that situation. I can't show the site/code because of NDA,

but was curious if someone had ever done that much work on a PHP page before to have it do that?

[BlueSkyIS]

don't make such a massive array? flush() content to the browser as it becomes available? hard to guess without anything but "it takes too long"

[Ninjakreborn]

I am not trying to show to browser. I am writing a crawler.  It is going to be used to get data from a site.  Eventually it's going to be doing three prime operations (one for each site).

Right now, it's only getting data from one site. I have gotten all the code done to get the data from the site.  I needed it in an array, so I can loop through it and put it into an array.

 

SO, that's the weird part. It's entirely not working. I mean it loads for a LONG time, then goes to a "Error" page in Chrome, and a "Internet Connectivity" error page in IE.  Strange.  I will try the flush content, and see how that goes.  I need to do something to allow it to finish loading.

[Ninjakreborn]

I think I am going about this the wrong way.  Instead, I am going to break it up into a few different options. I will do the core stuff in a cron, and do the rest dynamically on the fly. 

 

Thanks again for the feedback about Flush.