text file data to arrays?

[muppet77]

i have a text file that has spaces between each column.

 

eg

 

12 12 13 22 34.5

10 13 18 88 32.5

12 33 17 23 22.3

 

(the actual data is weather data that accumulates a new line every 15 mins and is seen at www.maidenerleghweather.com/clientraw.txt )

 

each column represents a weather variable.

 

i would like to use php to find the maximum values in each column and minimum values etc etc.

 

how can i get the text data into arrays, so that i can then easily perform some simple stats on it?

 

HOWEVER each line would NOT be an array.

 

the first value in each line makes up array number one, second makes up two etc

 

any ideas for a newbie beginner would be welcomed.

[ignace]

$fopen = fopen('http://www.maidenerleghweather.com/clientraw.txt');
while($line = fgets($fopen, 16)) {
    list($nameit, $nameit2, $nameit3, $nameit4, $nameit5) = sscanf($line, '%d %d %d %d %f');
    // process
}

[AbraCadaver]

With a fixed number of columns you could do this:

 

$rows = file('http://www.maidenerleghweather.com/clientraw.txt');

foreach($rows as $row) {
    list($col[1][], $col[2][], $col[3][], $col[4][], $col[5][]) = explode(' ', $row);
}

[AbraCadaver]

Oh, and then probably sort(), reset() and end() on each column to get min and max.

[muppet77]

perfect!

 

how would i then interrogate the data and find out some stats??

 

print max($col[3]);

print count($col[3]);

print min($col[3]);

 

???

[muppet77]

.........and is there a way i can find a value in array[0] - ie the time, if i know a value in array[3]?

 

eg if i find the max temperature in one column, can i look up the date and time that it occurred at?

 

also,

 

is it possible to lookup values with conditional if functions?

 

so for example if i had the month in a column, could i look up the maximum temp array[3] when the month = november?

 

thanks for your help guys, i hope to be this good someday!

[ignace]

perfect!

 

how would i then interrogate the data and find out some stats??

 

print max($col[3]);

print count($col[3]);

print min($col[3]);

 

???

 

Yes

 

eg if i find the max temperature in one column, can i look up the date and time that it occurred at?

 

You can't retrieve data that isn't send to you by the weather station.

[muppet77]

perfect!

 

how would i then interrogate the data and find out some stats??

 

print max($col[3]);

print count($col[3]);

print min($col[3]);

 

???

 

Yes

 

eg if i find the max temperature in one column, can i look up the date and time that it occurred at?

 

You can't retrieve data that isn't send to you by the weather station.

 

 

 

no...sorry.... in http://www.maidenerleghweather.com/clientraw.txt

 

the first column is date and the second column is time.

 

this means that array[0] is date and [1] is time, right?

 

i have the script running at

 

http://www.maidenerleghweather.com/1atest.php

 

which currently displays the max temp recorded (7.4)

 

can i lookup the date and time from col 1 and 2?

[AbraCadaver]

It's going to be much easier to just dump this into a database, MySQL or SQLite.

[muppet77]

any pointers mate?

 

[muppet77]

well this is what i have so far and i have worked out the function i need to grab the time and date of the record.

 

<?php
$rows = file('http://www.maidenerleghweather.com/clientraw.txt');foreach($rows as $row) {    list($col[0][], $col[1][], $col[2][], $col[3][], $col[4][]) = explode(' ', $row);}

$count2 = count($col[2]);
$dat = max($col[0]);
$tim = end($col[1]);

$tempmax = max($col[2]);
$tempmaxpos = array_search($tempmax,$col[2]);
$tempmaxdate = $col[0][$tempmaxpos];
$tempmaxtime = $col[1][$tempmaxpos];

$tempmin = min($col[2]);
$tempminpos = array_search($tempmin,$col[2]);
$tempmindate = $col[0][$tempminpos];
$tempmintime = $col[1][$tempminpos];


print "
max temp $tempmax @ $tempmaxtime on $tempmaxdate <br />
min temp $tempmin @ $tempmintime on $tempmindate <br />
last update sent @ $tim on $dat<br />
$count2 records";
?> 

 

 

all i need now is to work out how to do conditional searches.

 

eg max temp when date = november for example.

[AbraCadaver]

any pointers mate?

 

The problem is that your data is not consistent.  Many lines have 31 or 32 columns and several have 29 or 30.

[muppet77]

yes that's as i am fine tuning the scraping - i am just 'paving the way' ahead to see if it is do-able and hence worthwhile.

 

do you think i need a database - the php stuff seems ok for me?

 

 

[AbraCadaver]

yes that's as i am fine tuning the scraping - i am just 'paving the way' ahead to see if it is do-able and hence worthwhile.

 

do you think i need a database - the php stuff seems ok for me?

 

First, see if you have database extensions loaded and I'll give you an example.  Use phpinfo() or just try the functions and see if you get a fatal error:

 

mysql_connect();
//and
sqlite_open();

[muppet77]

have  a look at this

 

http://www.maidenerleghweather.com/11aa.php

[AbraCadaver]

Cool.  Here is an example.  Use the sqlite.org site for help with functions etc...  This will create a new table every time:

 

// name these whatever you want after you have decided how many columns there will be
$columns = array('column1', 'column2', 'column3', 'column4', 'column5', 'column6', 'column7', 'column8',
'column9', 'column10', 'column11', 'column12', 'column13', 'column14', 'column15', 'column16',
'column17', 'column18', 'column19', 'column20', 'column21', 'column22', 'column23', 'column24',
'column25', 'column26', 'column27', 'column28', 'column29', 'column30', 'column31', 'column32');

// open or create db, webserver must have write perms to directory where script resides, or try /tmp
$con = sqlite_open('client.db');
// delete the table
$res = sqlite_query($con, "DROP TABLE stats");
// create the table with the defined columns
$res = sqlite_query($con, "CREATE TABLE stats ( " . implode(',', $columns) . " )");

// read file into array of lines (rows)
$lines = file('http://www.maidenerleghweather.com/clientraw.txt', FILE_IGNORE_NEW_LINES);

foreach($lines as $i => $line) {
  // split line into columns
  $data = explode(' ', $line);
  // remove date (first column) from array
  $thedate = array_shift($data);
  // replace / with - so strtotime will work and add time (first column) to date
  $datetime = str_replace('/', '-', $thedate) . " " . $data[0];
  // put datetime in a more standard format and assign to first column (replacing time)
  $data[0] = date('Y-m-d G:i:s', strtotime($datetime));
  // make sure data column count is what is expected
  if(count($data) == count($columns)) {
    // insert data into db
    $res = sqlite_query($con, "INSERT INTO stats ( " . implode(',', $columns) . " ) VALUES ( " . "'" . implode("','", $data) . "' )");
  } else {
    echo "Line " . $i++ . ": " . count($data) . " columns, " . count($columns) . " expected!<br />\n";
  }
}

// then just use standard SQL to get what you want, max of some column where month is November
$res = sqlite_query($con, "SELECT MAX(column2) AS high FROM stats WHERE strftime('%m', column1) = 11");
$rows = sqlite_fetch_all($res, SQLITE_ASSOC);

 

 

[muppet77]

so i just put this in a page and save it as php?

 

sorry - a complete novice.

[AbraCadaver]

so i just put this in a page and save it as php?

 

sorry - a complete novice.

 

Yep, try it.

[muppet77]

how do i copy and paste it without getting lines disjointed and wrong?

 

tried it and it fails?

[muppet77]

ah.getting warmer.

 

i now get

 

http://www.maidenerleghweather.com/recordtest.php

 

 

[AbraCadaver]

ah.getting warmer.

 

i now get

 

http://www.maidenerleghweather.com/recordtest.php

 

OK, last example an then you have to hit some tutorials:

 

echo $rows[0]['high'];