[Duckington]

Hi,

Having quite a bit of trouble dealing with special characters being sent in an AJAX request at the moment.

This is an example of the XML body being sent:

<updateUnitComments><studentID>1234</studentID><qualID>123</qualID><unitID>123</unitID><comments>comments go here</comments></updateUnitComments>

(The request is sent with UTF-8 encoding btw)


However, if there is something like an ampersand or a less than or greater than symbol in the comments (which there could easily be, as it's from a textarea), it doesn't work.

Now I've been through loads of google links on the matter and tried pretty much everything I can find, including:

- escape()
- wrapping the comments in CDATA
- manually replacing the symbols with something else
- Converting & to &
- etc..

But even when converting an & to something like %26 or whatever it's code is, it still doesn't work, so I'm quite confused

[codeprada]

Data is always encoded when sent via a request. For example look at this URL.

http://www.site.com?user=John Doe

That's an invalid URL because of the space between John and Doe so therefore to pass the data correctly you must represent a space with another character or characters. Normally it's %20 or + used to represent a space. There are other characters that must be encoded also like & and =. Check out these two functions

• encodeURI
  
• decodeURI

This post has been edited by codeprada: 06 February 2012 - 09:02 AM

[Duckington]

Hi,

Tried that one as well, still doesn't like it.

For example, let's say I use that function to encode the data sent (the comment's I am sending in this example are: "HELLO & GOODBYE"):

var comments = $("#"+commentsDiv).val();
comments = encodeURI(comments);  

xmlhttp = setUpHTTPRequest();
var xmlBody = "<updateCriteriaComments><studentID>'.$this->studentID.'</studentID><qualID>'.$this->id.'</qualID><criteriaID>"+critID+"</criteriaID><valueID></valueID><comments>"+comments+"</comments></updateCriteriaComments>";

alert(xmlBody);           

If I alert the XML body there, it is correct, it alerts:

<updateCriteriaComments><studentID>37255</studentID><qualID>418</qualID><criteriaID>520</criteriaID><valueID></valueID><comments>HELLO%20&%20GOODBYE</comments></updateCriteriaComments>

However, once it is then actually sent to the PHP script:

if(xmlhttp)
                        {
                                xmlhttp.open("POST", "'.$CFG->wwwroot.'/mod/qualification/update_student_criteria.php", true);
                                xmlhttp.onreadystatechange = function(){
                                                                    cmt.cancel();
                                                                    if(xmlhttp.readyState === 4)
                                                                    {
                                                                        if(comments == " ")
                                                                        {
                                                                            updateCommentCell(cellID, xmlhttp.responseXML, true);
                                                                        }
                                                                        else
                                                                        {
                                                                            updateCommentCell(cellID, xmlhttp.responseXML);
                                                                        }

                                                                    }
                                                                }
                                xmlhttp.setRequestHeader("Content-Type","application/x-www-form-urlencoded");
                                xmlhttp.send("request=" + "<?xml version=\'1.0\' encoding=\'UTF-8\'?>" + xmlBody);
                        }

If I print out the $_POST['request'] variable, which contains the XML body just sent, it prints out:

<?xml version=\'1.0\' encoding=\'UTF-8\'?><updateCriteriaComments><studentID>37255</studentID><qualID>418</qualID><criteriaID>520</criteriaID><valueID></valueID><comments>HELLO 

So it's failing as soon as it gets to the &.

At this stage in the PHP script I have done nothing to the variable, I've not stripped anything from it, or run it through any other funtions, I am simply printing it our using print_r() so I'm somewhat at a loss as to where I'm going wrong.

Can you see where my problem is?

Thanks.

Update:

if I run it through escape(comments) instead, it gets as far as sending the data correctly to the PHP script. So I'll have to see if I can work out why it's then failing...

[codeprada]

That's because you must encode your data. Try this

xmlhttp.send("request=" + encodeURI("<?xml version='1.0' encoding='UTF-8'?>" + xmlBody));

This post has been edited by codeprada: 06 February 2012 - 09:34 AM

[Duckington]

I can get it to send to the PHP script now, although the encodeURI doesn't seem to make any difference to it, i got it to send by running it through escape().

However, now it's breaking in the PHP.

If i print out the XML the php script has received, it looks like:

<?xml version=\'1.0\' encoding=\'UTF-8\'?><updateCriteriaComments><studentID>37255</studentID><qualID>418</qualID><criteriaID>520</criteriaID><valueID></valueID><comments>HELLO & GOODBYE</comments></updateCriteriaComments>

So the comments are getting sent successfully, but the second I try and create a new SimpleXMLElement object using it, it still fails if there is a character like an & in it.

Error[2]: SimpleXMLElement::__construct() [simplexmlelement.--construct]: Entity: line 1: parser error : xmlParseEntityRef: no name

Error[2]: SimpleXMLElement::__construct() [simplexmlelement.--construct]: lID>418</qualID><criteriaID>520</criteriaID><valueID></valueID><comments>HELLO &

Error[2]: SimpleXMLElement::__construct() [simplexmlelement.--construct]:

See, I'm not sure if the XML printed out from the PHP script should in fact have the & in it properly, or if it should still be in the encoded format, such as %26.

Also: "Grrr..don't like XML".

[codeprada]

You must remove the backslashes in the version and encoding attributes. Note in my example they're not there.

[JackOfAllTrades]

For this:

<?xml version=\'1.0\' encoding=\'UTF-8\'?><updateCriteriaComments><studentID>37255</studentID><qualID>418</qualID><criteriaID>520</criteriaID><valueID></valueID><comments>HELLO & GOODBYE</comments></updateCriteriaComments>

to be valid XML it must be:

<?xml version='1.0' encoding='UTF-8'?><updateCriteriaComments><studentID>37255</studentID><qualID>418</qualID><criteriaID>520</criteriaID><valueID></valueID><comments>HELLO &amp; GOODBYE</comments></updateCriteriaComments>

Although I believe CDATA should work as well:

<?xml version='1.0' encoding='UTF-8'?><updateCriteriaComments><studentID>37255</studentID><qualID>418</qualID><criteriaID>520</criteriaID><valueID></valueID><comments><![CDATA[HELLO & GOODBYE]]></comments></updateCriteriaComments>

[Duckington]

Okay, let's start this from the beginning again, because I still can't work out where I'm going wrong.

This is the function which sends my AJAX request:

submit : function(commentsDiv){ /* Submit comment */
                        
                        var comments = $("#"+commentsDiv).val();
                        
                        /* Build XML Body */
                        xmlhttp = setUpHTTPRequest();
                        var xmlBody = "<updateCriteriaComments><studentID>{$this->studentID}</studentID><qualID>{$this->id}</qualID><criteriaID>"+critID+"</criteriaID><valueID></valueID><comments><![CDATA["+comments+"]]></comments></updateCriteriaComments>";
                        alert(xmlBody);
                        if(xmlhttp)
                        {
                                xmlhttp.open("POST", "{$CFG->wwwroot}/mod/qualification/update_student_criteria.php", true);
                                xmlhttp.onreadystatechange = function(){
                                                                    cmt.cancel();
                                                                    if(xmlhttp.readyState === 4)
                                                                    {
                                                                        if(comments == " ")
                                                                        {
                                                                            updateCommentCell(cellID, xmlhttp.responseXML, true);
                                                                        }
                                                                        else
                                                                        {
                                                                            updateCommentCell(cellID, xmlhttp.responseXML);
                                                                        }

                                                                    }
                                                                }
                                xmlhttp.setRequestHeader("Content-Type","application/x-www-form-urlencoded");
                                xmlhttp.send("request=" + encodeURI("<?xml version='1.0' encoding='UTF-8'?>" + xmlBody));
                        }
            

                    },

So, i have removed the backslashes from the xml version/encoding, and I am using CDATA on the comments section.

Now, when I submit this, it alerts the XML body before it is sent, so I can see what it is. So using the example comment of "c & d", it alerts this:

<updateCriteriaComments><studentID>37255</studentID><qualID>418</qualID><criteriaID>7212</criteriaID><valueID></valueID><comments><![CDATA[c & d]]></comments></updateCriteriaComments>

So it is wrapped in CDATA as you can see.

Now it's sent to the PHP script, and before I do anything there, I just print it out so I can again see what it has received. The printing out is done like this:

$request = $_POST['request'];	
$request = stripslashes($request);
printOut( print_r($request, true) );

printOut() literally just writes the string it's given to a log file.
I have to use stripslashes on the POST, since the software we use automatically runs every REQUEST through addslashes in it's core library, and I can't disable that/not allowed to.

So from the above request, what the PHP prints out as the XML body it's received is this:

<?xml version='1.0' encoding='UTF-8'?><updateCriteriaComments><studentID>37255</studentID><qualID>418</qualID><criteriaID>7212</criteriaID><valueID></valueID><comments><![CDATA[c 

So the CDATA hasn't made any difference there, as the & has still messed it up.


So okay, the next thing I try is to replace the & with an & to be sent.

So here is the alerted body before the request:

<updateCriteriaComments><studentID>37255</studentID><qualID>418</qualID><criteriaID>7212</criteriaID><valueID></valueID><comments><![CDATA[c &amp; d]]></comments></updateCriteriaComments>

And again, when we get to the PHP script, all it has found is:

<?xml version='1.0' encoding='UTF-8'?><updateCriteriaComments><studentID>37255</studentID><qualID>418</qualID><criteriaID>7212</criteriaID><valueID></valueID><comments><![CDATA[c 

Because there is still an "&" in "&" presumably.


Okay, so what if I try replacing the & with &, but removing the CDATA:

Alerted body:

<updateCriteriaComments><studentID>37255</studentID><qualID>418</qualID><criteriaID>7212</criteriaID><valueID></valueID><comments>c &amp; d</comments></updateCriteriaComments>

PHP printed body:

<?xml version='1.0' encoding='UTF-8'?><updateCriteriaComments><studentID>37255</studentID><qualID>418</qualID><criteriaID>7212</criteriaID><valueID></valueID><comments>c 

Okay, so at this point losing the will to live.

Sorry I'm being stupid, but as far as I can see I've done everything that's been suggested. I've tried wrapping it in CDATA, I've tried replacing & with & I've tried wrapping the whole request in encodeURI() and yet it's still not getting sent across properly.

Is it something to do with the encoding i'm using, or..?

:-(

[Duckington]

I'm sort of..almost there..I think.

If I use encodeURIComponent on the comments, rather than wrap it in CDATA then it gets sent correctly to the PHP.

Then in the PHP i have to use urldecode() to decode it again.

..

Then use rawurlencode() to encode it AGAIN to send it back to the javascript where I then have to use decodeURIComponent() to get it's proper value...





Yeah, I'm quitting my job I think.

[JackOfAllTrades]

Quote

Yeah, I'm quitting my job I think.

Totally feel ya here, dude! This sort of encoding crap -- like much of XML -- can certainly be extremely frustrating. I thought I'd left XML behind...now I find out soon I'm going to be back in SOAP-land! Woohoo! WSDL-hell!

[codeprada]

Apparently for URL parameters use encodeURIComponent instead of encodeURI or escape. Have a look here for a more detailed explanation.

Here's a quick example I was working on for reference.

<html>
<head>
    <script type="text/javascript">
               var xml = 'params=' + encodeURIComponent("<?xml version='1.0' encoding='UTF-8'?><updateCriteriaComments><studentID>37255</studentID><qualID>418</qualID><criteriaID>520</criteriaID><valueID></valueID><comments>HELLO & GOODBYE</comments></updateCriteriaComments>");
		
		var xmlhttp = (function(){
			if(window.XMLHttpRequest)
				return new XMLHttpRequest();
			else
				return new ActiveXObject("Microsoft.XMLHTTP");
		})()
		
		xmlhttp.onreadystatechange = function(){
			if(xmlhttp.readyState == 4 && xmlhttp.status == 200)
				alert(xmlhttp.responseText);
		}
		
		xmlhttp.open('POST', 'process.php', true);
		xmlhttp.setRequestHeader('Content-Type', 'application/x-www-form-urlencoded')
		xmlhttp.setRequestHeader('Content-Length', xml.length);
		xmlhttp.setRequestHeader('Connection', 'close');
		xmlhttp.send(xml);
    </script>
</head>
<body>
  
</body>
</html>

process.php

<?php
	if(isset($_POST['params']))
		echo $_POST['params'];
?>

You shouldn't have to decode your XML in your PHP script. It's just like a regular POST from a form. You don't decode all the $_POST or $_GET variables so you shouldn't have to with this.
encodeURI cuts the XML off at the & character while encodeURIComponent works just fine.

This post has been edited by codeprada: 07 February 2012 - 08:13 AM