Sanitize + htmlentities Problem with Accents and Needed Special Characters

[nightkarnation]

Hey Guys!

I would really like some feedback on the following:

 

I have a site in Portuguese. Php retrieves a lot of POST's with Special Characters and Portuguese Accents (which are expected).

With my sanatize function I am having some real problems with the 'htmlentities' for XSS Injection Prevention.

htmlentities is changing the accents to strange characters and messes up my database.

 

sanitize( &$_GET );
sanitize( &$_POST );
sanitize( &$_COOKIE );

function sanitize( &$some) 
{
$some = array_map('htmlentities', $some); //XSS Prevention

foreach( $some as $key => $value ) 
{
	$value = str_replace( '--', '', $value );
	$value = str_replace( '/*', '', $value );
	$value = str_replace( '"', '', $value );
	$value = str_replace( "'", '', $value );
	$value = ereg_replace( '[\( ]+0x', '', $value );
	if ($value != $some[$key]) 
	{
		$some[$key] = $value;
	}
}
}

 

The only solution I can think of is to take out the 'htmlentities' function, but I would really like to have this as a prevention against XSS, is there any way around this to have both things working?

 

Any ideas, suggestions?

 

Thanks in advance!

[simshaun]

First, you should be inserting the posted value into your database as is (accented characters, quotes and all) and format your content upon display.

To do this, be sure your db collation (and field collation) is set appropriately.  I recommend utf8_unicode_ci.  Otherwise, the character data may not store correctly. 

Then, ensure your PHP MySQL connection is using utf8 (see mysql_set_charset()). 

Finally, ensure your HTML encoding type is set to utf8.  One way is to put this meta tag in your head tag.

<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />

 

Second, I don't recommend using your sanitize() function at all.  It seems as if its primary purpose based on its functionality is for escaping the data for use in a SQL string.  You should either A. be using mysql_real_escape_string() instead, or B. use prepared statements or an abstraction layer that uses parametrized queries.

 

Third (and as a side-note): You should use preg_replace instead of ereg_replace (be sure to add the expression delimiters).

[nightkarnation]

Simshaun

 

Thanks a lot for your kind help!

Very useful information.

 

I will apply all this tomorrow and let you know how it went,

 

Cheers!

[btherl]

And remember to use htmlentities() on the data you fetch from the db, before inserting it into the html.  That's where the XSS prevention will happen.  So the process is:

 

Input => mysql_real_escape_string()/prepared statements => db => htmlentities() => insert into HTML output