-=(ASP-Support-Group)=- Character encoding problem with AJAX

Hello everyone,

My website is in windows-1251 encoding and everything in the db is
stored in windows-1251. Now I am trying to use AJAX and as I understand
AJAX sends the request in UTF-8. So I will need to convert UTF-8 to
windows-1251 once I get the request and then store in db. There is a
function in PHP iconv doing that, but what do we (ASP programmers) do
in this situation. I've been trying to find an equivalent of ICONV for
ASP all over the Internet - no luck, I wonder am I the only ASP
programmer using cyrillic and AJAX?
And no, I am not willing to redo the whole site in UTF-8 - first
because it's alot of work, and second will double my database in size
since each character will take not 1 but 2 bytes.

Any suggestions will be appreciated.
George

Messages in this topic (1) Reply (via web post) | Start a new topic

.

Well it's important to remember ajax is really a method of interaction, as
apposed to an actual language or object. What your really having an issue
with is xmlhttp, or xmlhttprequest. And as the name suggest, these are XML
based protocols and therefore you should really always use UTF-8. Ok,
enough patronising background, the fix..

Basically if your not using any strange character, then you should be able
to just dump the data into a string, and ASP will do it for you. But I
presume you've tried that and come across a problem, so you can use the
ADODB stream object to convert it. This does require you converting your
string into a byte array first, but it's a reliable method that uses only
inbuilt asp objects(There are a few external objects around that will do all
this a lot easier).

The decode function
Public Function ConvertUtf8BytesToString(ByRef data() As Byte) As String

Dim objStream As ADODB.Stream
Dim strTmp As String

' init stream
Set objStream = New ADODB.Stream
objStream.Charset = "utf-8"
objStream.Mode = adModeReadWrite
objStream.Type = adTypeBinary
objStream.Open

' write bytes into stream
objStream.Write data
objStream.Flush

' rewind stream and read text
objStream.Position = 0
objStream.Type = adTypeText
strTmp = objStream.ReadText

' close up and return
objStream.Close
ConvertUtf8BytesToString = strTmp

End Function
The base64 converterPrivate Function DecodeBase64(ByVal strData As String)
As Byte() Dim objXML As MSXML2.DOMDocument Dim objNode As
MSXML2.IXMLDOMElement ' help from MSXML Set objXML = New
MSXML2.DOMDocument Set objNode = objXML.createElement("b64")
objNode.dataType = "bin.base64"; objNode.Text = strData DecodeBase64 =
objNode.nodeTypedValue ' thanks, bye Set objNode = Nothing Set
objXML = NothingEnd FunctionYour actual conversion code.. Dim strB64 As
String
Dim data() As Byte
Dim strTmp As String

' define test data as base64 and decode to array of bytes
strB64 = "R3JlZXRpbmdzIGFuZCBTYWx1dGF0aW9uISAo4oKsKSBhbmQgc29";
strB64 = strB64 & "tZSBVcmR1OiDaqdix2KfahtuMINm+2Kfaqdiz2KrYp9mG24w="
data = DecodeBase64(strB64)

' convert from utf-8 to string
strTmp = ConvertUtf8BytesToString(data)

' convert back to bytes
data = ConvertStringToUtf8Bytes(strTmp)

TonyHome

Catapultam habeo. Nisi pecuniam omnem mihi dabis, ad caput tuum saxum immane
mittam
----- Original Message -----
From: "g5sev" < g5sev%40yahoo.com">g5sevyahoo.com>
To: < ASP-Support-Group%40yahoogroups.com">ASP-Support-Groupyahoogroups.com>
Sent: Saturday, March 01, 2008 7:30 PM
Subject: -=(ASP-Support-Group)=- Character encoding problem with AJAX

Hello everyone,

My website is in windows-1251 encoding and everything in the db is
stored in windows-1251. Now I am trying to use AJAX and as I understand
AJAX sends the request in UTF-8. So I will need to convert UTF-8 to
windows-1251 once I get the request and then store in db. There is a
function in PHP iconv doing that, but what do we (ASP programmers) do
in this situation. I've been trying to find an equivalent of ICONV for
ASP all over the Internet - no luck, I wonder am I the only ASP
programmer using cyrillic and AJAX?
And no, I am not willing to redo the whole site in UTF-8 - first
because it's alot of work, and second will double my database in size
since each character will take not 1 but 2 bytes.

Any suggestions will be appreciated.
George

Group homepage is here:
http://groups.yahoo.com/group/ASP-Support-Group

To unsubscribe from this group, send an email to:
ASP-Support-Group-unsubscribe%40yahoogroups.com">ASP-Support-Group-unsubscribeyahoogroups.com and cross your fingers
because Yahoo is 5h1te.

Yahoo! Groups Links

Messages in this topic (2) Reply (via web post) | Start a new topic

.

Well that got all messed up, please see the attached file for properly
formatted code..

TonyHome

Catapultam habeo. Nisi pecuniam omnem mihi dabis, ad caput tuum saxum immane
mittam
----- Original Message -----
From: "Tony Cheetham&quot; < Tony%40Enkidu.cx">TonyEnkidu.cx>
To: < ASP-Support-Group%40yahoogroups.com">ASP-Support-Groupyahoogroups.com>
Sent: Saturday, March 01, 2008 11:18 PM
Subject: Re: -=(ASP-Support-Group)=- Character encoding problem with AJAX

Well it's important to remember ajax is really a method of interaction, as
apposed to an actual language or object. What your really having an issue
with is xmlhttp, or xmlhttprequest. And as the name suggest, these are XML
based protocols and therefore you should really always use UTF-8. Ok,
enough patronising background, the fix..

Basically if your not using any strange character, then you should be able
to just dump the data into a string, and ASP will do it for you. But I
presume you've tried that and come across a problem, so you can use the
ADODB stream object to convert it. This does require you converting your
string into a byte array first, but it's a reliable method that uses only
inbuilt asp objects(There are a few external objects around that will do all
this a lot easier).

The decode function
Public Function ConvertUtf8BytesToString(ByRef data() As Byte) As String

Dim objStream As ADODB.Stream
Dim strTmp As String

' init stream
Set objStream = New ADODB.Stream
objStream.Charset = "utf-8"
objStream.Mode = adModeReadWrite
objStream.Type = adTypeBinary
objStream.Open

' write bytes into stream
objStream.Write data
objStream.Flush

' rewind stream and read text
objStream.Position = 0
objStream.Type = adTypeText
strTmp = objStream.ReadText

' close up and return
objStream.Close
ConvertUtf8BytesToString = strTmp

End Function
The base64 converterPrivate Function DecodeBase64(ByVal strData As String)
As Byte() Dim objXML As MSXML2.DOMDocument Dim objNode As
MSXML2.IXMLDOMElement ' help from MSXML Set objXML = New
MSXML2.DOMDocument Set objNode = objXML.createElement("b64&quot;)
objNode.dataType = "bin.base64"; objNode.Text = strData DecodeBase64 =
objNode.nodeTypedValue ' thanks, bye Set objNode = Nothing Set
objXML = NothingEnd FunctionYour actual conversion code.. Dim strB64 As
String
Dim data() As Byte
Dim strTmp As String

' define test data as base64 and decode to array of bytes
strB64 = "R3JlZXRpbmdzIGFuZCBTYWx1dGF0aW9uISAo4oKsKSBhbmQgc29";
strB64 = strB64 & "tZSBVcmR1OiDaqdix2KfahtuMINm+2Kfaqdiz2KrYp9mG24w=&quot;
data = DecodeBase64(strB64)

' convert from utf-8 to string
strTmp = ConvertUtf8BytesToString(data)

' convert back to bytes
data = ConvertStringToUtf8Bytes(strTmp)

TonyHome

Catapultam habeo. Nisi pecuniam omnem mihi dabis, ad caput tuum saxum immane
mittam
----- Original Message -----
From: "g5sev" < g5sev%40yahoo.com">g5sevyahoo.com>
To: < ASP-Support-Group%40yahoogroups.com">ASP-Support-Groupyahoogroups.com>
Sent: Saturday, March 01, 2008 7:30 PM
Subject: -=(ASP-Support-Group)=- Character encoding problem with AJAX

Hello everyone,

My website is in windows-1251 encoding and everything in the db is
stored in windows-1251. Now I am trying to use AJAX and as I understand
AJAX sends the request in UTF-8. So I will need to convert UTF-8 to
windows-1251 once I get the request and then store in db. There is a
function in PHP iconv doing that, but what do we (ASP programmers) do
in this situation. I've been trying to find an equivalent of ICONV for
ASP all over the Internet - no luck, I wonder am I the only ASP
programmer using cyrillic and AJAX?
And no, I am not willing to redo the whole site in UTF-8 - first
because it's alot of work, and second will double my database in size
since each character will take not 1 but 2 bytes.

Any suggestions will be appreciated.
George

Group homepage is here:
http://groups.yahoo.com/group/ASP-Support-Group

To unsubscribe from this group, send an email to:
ASP-Support-Group-unsubscribe%40yahoogroups.com">ASP-Support-Group-unsubscribeyahoogroups.com and cross your fingers
because Yahoo is 5h1te.

Yahoo! Groups Links

Group homepage is here:
http://groups.yahoo.com/group/ASP-Support-Group

To unsubscribe from this group, send an email to:
ASP-Support-Group-unsubscribe%40yahoogroups.com">ASP-Support-Group-unsubscribeyahoogroups.com and cross your fingers
because Yahoo is 5h1te.

Yahoo! Groups Links

Messages in this topic (3) Reply (via web post) | Start a new topic

.