Not detecting duplicates

3raser · October 1, 2011

I've made a small chicken names website, and I also have a chicken name list as a thread on a chicken related forum. People submit their names, and I manually add them to the list. With the names growing so much, I decided to make a small background system that lets me put in the current list names, then the new names submitted. It will then run the lists, compare them, and spit out/ignore any duplicates and generate the new list.

But each time I use it, it doesn't get rid of the duplicates. Can anyone tell me why?

if($_GET['list_updater'] || $_POST['list_updater'])
{
	if(!$_POST['current_list'] || !$_POST['new_names'])
	{
			?>
				<table>
					<form action="index.php" method="POST">
					<input type="hidden" name="list_updater" value="1">
						<tr>
							<td>
								<textarea name="current_list" cols="40" rows="20"></textarea>
							</td>
						</tr>

						<tr>
							<td>
								<textarea name="new_names" cols="40" rows="20"></textarea>
							</td>
						</tr>
						<tr>
							<td><input type="submit" value="Generate New List"></td>
						</tr>
					</form>
				</table>
			<?php
	}
	else
	{
		//start our counting numbers
		$x = 0;
		$s = 0;
		$a = 0;

		//make our arrays
		$check_to[0];
		$new[0];

		//get the values from our lists
		$old_list = explode("\n",$_POST['current_list']);
		$new_list = explode("\n",$_POST['new_names']);

		//get new list variables
		foreach($old_list as $value)
		{
			$check_to[$x] = $value;
			$x++;
		}

		//add all our new names
		foreach($new_list as $value)
		{
			for($s = 0; $s < count($new_list); $s++)
			{

			}
		}
		?>
			<textarea cols="45" rows="20">
				<?php
					for($t = 0; $t < count($new); $t++)
					{
						echo $new[$t];
					}
				?>
			</textarea>
		<?php
	}
}

Drummin · October 1, 2011

You must be posting a huge current list.. Why not just query the DB when user posts a new name and if a match is not found add it? Seems like a simpler approach. If you need to check names, just make a new db table for holding, list them with a link to get rid of obvious bad names. Then instead of "posting" this list just have a link or form button set a flag variable that says runcomparison="t" and query the tables adding or deleting for each name.

3raser · October 1, 2011

You must be posting a huge current list.. Why not just query the DB when user posts a new name and if a match is not found add it? Seems like a simpler approach. If you need to check names, just make a new db table for holding, list them with a link to get rid of obvious bad names. Then instead of "posting" this list just have a link or form button set a flag variable that says runcomparison="t" and query the tables adding or deleting for each name.

Because, there is another list I run somewhere else that is just a text list.

http://www.backyardchickens.com/forum/viewtopic.php?pid=6114934#p6114934

Now tell me how I could possibly have any power to install a database system on that. Not my website.

Drummin · October 1, 2011

I see your text areas, how are you entering new names? One at a time? Some type of list. Still I wouldn't post current names (assuming these ARE on your site), just compare posted names to DB.

3raser · October 1, 2011

I see your text areas, how are you entering new names? One at a time? Some type of list. Still I wouldn't post current names (assuming these ARE on your site), just compare posted names to DB.

As I have mentioned...I can't use a DB due to several reasons.

I have the first textbox to paste in the current list, then the second one to get the new ones. The code then removes all duplicates and adds the new names.

Drummin · October 1, 2011

Give this a go

<?php
if(isset($_GET['list_updater']) || isset($_POST['list_updater']) && !$_POST['current_list'] && !$_POST['new_names']){
?>
<form action="test45.php" method="POST">
<input type="hidden" name="list_updater" value="1">
<table>
<tr>
	<td><textarea name="current_list" cols="40" rows="20"></textarea></td>
</tr>
<tr>
	<td><textarea name="new_names" cols="40" rows="20"></textarea></td>	
</tr>
<tr>
	<td><input type="submit" value="Generate New List"></td>
</tr>
</table>
</form>
<?php }else{
if(isset($_POST['current_list']) && isset($_POST['new_names'])){
//make our arrays
$new_list=array();
//get the values from our lists
$old_list = explode("\n",$_POST['current_list']);
$new_list = explode("\n",$_POST['new_names']); 
$result = array_diff($new_list, $old_list);
$result2 = implode('', array_values($result));
array_push($old_list, $result2);
$new_list=implode("\n",$old_list);
?>
<textarea cols="45" rows="20">
<?php
print_r($new_list);
?>
</textarea>
<?php }} ?>

Buddski · October 1, 2011

Or you could use array_merge on the 2 fields, then use array_unique to remove duplicate values..

$list = explode("\n",$_POST['current_list']);
$new = explode("\n",$_POST['new_names']);
$new_list = array_unique(array_merge($list,$new));
echo '<textarea>' , join("\n",$new_list) , '</textarea>';

Drummin · October 1, 2011

Oh that's cool.

3raser · October 1, 2011

Or you could use array_merge on the 2 fields, then use array_unique to remove duplicate values..
$list = explode("\n",$_POST['current_list']);
$new = explode("\n",$_POST['new_names']);
$new_list = array_unique(array_merge($list,$new));
echo '<textarea>' , join("\n",$new_list) , '</textarea>';

Oh, my, god. Thank you.

3raser · October 1, 2011

Ok, my current code is:

else
	{

		//get the values from our lists
		$old_list = explode("\n",$_POST['current_list']);
		$new_list = explode("\n",$_POST['new_names']);
		$joined_list = array_unique(array_merge($old_list, $new_list));

		echo join("\n", $joined_list);
	}

Yet whenever I type in Apples, Oranges in the first box, then Oranges, Melons in the other box, it returns Oranges two times. Why is it not removing the duplicates still?

xyph · October 1, 2011

My guess is some values have a trailing "\r" on them.

Windows systems use \r\n rather than just \n. If the value is the last on the list, it won't have the trailing \r

<form action="<?php echo $_SERVER['SCRIPT_NAME']; ?>" method="post">
<textarea name="current" rows="10"><?php if( isset($_POST['current']) ) echo $_POST['current']; ?></textarea>
<textarea name="new" rows="10"><?php if( isset($_POST['new']) ) echo $_POST['new']; ?></textarea><br>
<input type="submit">
</form>
<?php 

if( $_POST ) {
$cur = str_replace( "\r", '', $_POST['current'] );
$new = str_replace( "\r", '', $_POST['new'] );
$cur = explode( "\n", $cur );
$new = explode( "\n", $new );
$merged = array_unique( array_merge($cur,$new) );
echo '<textarea rows="20">' . implode("\n",$merged) . '</textarea>';
}



?>

3raser · October 1, 2011

My guess is some values have a trailing "\r" on them.

Windows systems use \r\n rather than just \n. If the value is the last on the list, it won't have the trailing \r

<form action="<?php echo $_SERVER['SCRIPT_NAME']; ?>" method="post">
<textarea name="current" rows="10"><?php if( isset($_POST['current']) ) echo $_POST['current']; ?></textarea>
<textarea name="new" rows="10"><?php if( isset($_POST['new']) ) echo $_POST['new']; ?></textarea><br>
<input type="submit">
</form>
<?php 

if( $_POST ) {
$cur = str_replace( "\r", '', $_POST['current'] );
$new = str_replace( "\r", '', $_POST['new'] );
$cur = explode( "\n", $cur );
$new = explode( "\n", $new );
$merged = array_unique( array_merge($cur,$new) );
echo '<textarea rows="20">' . implode("\n",$merged) . '</textarea>';
}



?>

Thank you! It works!

xyph · October 1, 2011

Some one line fun

<form action="<?php echo $_SERVER['SCRIPT_NAME']; ?>" method="post">
<textarea name="current" rows="10"><?php if( isset($_POST['current']) ) echo $_POST['current']; ?></textarea>
<textarea name="new" rows="10"><?php if( isset($_POST['new']) ) echo $_POST['new']; ?></textarea><br>
<input type="submit">
</form>
<?php 

if( $_POST ) {
echo '<textarea rows="20">' .
	implode("\n",
		array_unique(
			array_merge(
				explode( "\n", str_replace( "\r", '', $_POST['current'] ) ),
				explode( "\n", str_replace( "\r", '', $_POST['new'] ) )
			)
		)
	) .
	'</textarea>';
}



?>

3raser · October 2, 2011

Doesn't work:

	else
	{

		//get the values from our lists
		$old_list = str_replace(" ", '_', $_POST['currnet_list']);
		$old_list = str_replace("\r", '', $_POST['current_list']);
		$new_list = str_replace("\r", '', $_POST['new_names']);
		$new_list = str_replace(" ", '_', $_POST['new_names']);
		$old_list = explode("\n",$old_list);
		$new_list = explode("\n",$new_list);
		$joined_list = array_unique(array_merge($old_list, $new_list));

		foreach($joined_list as $value)
		{
			echo $value."<br/>";
		}
	}

xyph · October 2, 2011

Typos are known to cause issues in pretty much every language.

3raser · October 2, 2011

Typos are known to cause issues in pretty much every language.

I fixed the "current_list" typo, but it still doesn't work.

I can't seem to figure out why it doesn't remove duplicates.

Drummin · October 2, 2011

Buddski's version works and so did mine assuming you are entering names in list form like this.

Achilles

Adel

Alfredo

Alistair

Amapola

Angus

Apollo

Ares

Arizona (Ari)

Athos

Aubrey

Barb (Barbecue)

Barbeque

Batman

Beauregard

Ben Dover

Bert

Big Ben

Did you try my copy?

Drummin · October 2, 2011

Actually as xyph pointed out, if you are typing in names and not pasting from a list of names you will probably have \r which should be stripped out.

$cur = str_replace( "\r", '', $_POST['current_list'] );
$new = str_replace( "\r", '', $_POST['new_names'] );
$list = explode("\n",$cur);
$new = explode("\n",$new);
$new_list = array_unique(array_merge($list,$new));
echo '<textarea cols="45" rows="20">' , join("\n",$new_list) , '</textarea>';

Pikachu2000 · October 2, 2011

Get rid of all extraneous leading and trailing whitespace at the same time by using array_map() with trim() on the merged array before array_unique() is called.

$list = explode("\n",$cur);
$new = explode("\n",$new);
$merged = array_merge($list,$new);
$merged = array_map('trim', $merged);
$new_list = array_unique($merged);

3raser · October 2, 2011

Get rid of all extraneous leading and trailing whitespace at the same time by using array_map() with trim() on the merged array before array_unique() is called.
$list = explode("\n",$cur);
$new = explode("\n",$new);
$merged = array_merge($list,$new);
$merged = array_map('trim', $merged);
$new_list = array_unique($merged);

I thought I was getting rid of any whitespace like so:

//get the values from our lists
		$old_list = str_replace(" ", '_', $_POST['current_list']);
		$old_list = str_replace("\r", '', $_POST['current_list']);
		$new_list = str_replace("\r", '', $_POST['new_names']);
		$new_list = str_replace(" ", '_', $_POST['new_names']);
		$old_list = explode("\n",$old_list);
		$new_list = explode("\n",$new_list);

Pikachu2000 · October 2, 2011

trim removes more than just those. And do you really want to remove spaces/tabs/etc. between words? I would think that would be neither needed nor desired.

xyph · October 2, 2011

trim() with array_map() would be the ideal solution.

Sign In

Not detecting duplicates

Recommended Posts

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Join the conversation

Important Information