Unicode JSON Serialization Bug?

Hello All-

I've been told that I may have encountered a bug - I'm
trying to
serialize a model that includes fields that could be unicode
data.
Here is the information I collected.  Let me know if there's
anything
else that would be helpful.

Specs of the local dev workstation - having the same issues
in prod
using Linux/FastCGI:

Windows XP Home Edition Version 2002 SP 2
Python 2.4.4
Latest Django from SVN: 0.97-pre-SVN-6024
SQLite3
Using internal django web server (for dev)

Here's the traceback:

Traceback (most recent call last):
File
"C:Python24libsite-packagesdjangocorehandlersbas
e.py" in
get_response
 77. response = callback(request, *callback_args,
**callback_kwargs)
File "C:Documents and SettingsGabe HarrimanMy
Documentsdevtextycmsviews.py" in syndicate
 178. comments = serializers.serialize("json",
texty.comment_set.all(),
fields=('name','comment','created_date',),
ensure_ascii=False)
File
"C:Python24libsite-packagesdjangocoreserializers
__init__.py"
in serialize
 67. s.serialize(queryset, **options)
File
"C:Python24libsite-packagesdjangocoreserializers
base.py"
in serialize
 50. self.end_serialization()
File
"C:Python24libsite-packagesdjangocoreserializers
json.py"
in end_serialization
 26. simplejson.dump(self.objects, self.stream,
cls=DjangoJSONEncoder, **self.options)
File
"C:Python24libsite-packagesdjangoutilssimplejson
__init__.py" in dump
 139. fp.write(chunk)

 UnicodeEncodeError at /cms/syndicate/d5f8eba5-d4a8-4590-
a3d9-4d9e34503c5b.js
 'ascii' codec can't encode characters in position 1-7:
ordinal not
in range(128)


Unicode error hint:
The string that could not be encoded/decoded was:
"ﺥﺾﻤﻼﺚﻴﻴ"

As you can see from above, the error starts here:

comments = serializers.serialize("json",
texty.comment_set.all(),
fields=('name','comment','created_date',),
ensure_ascii=False)

The data in the name field is
"ﺥﺾﻤﻼﺚﻴﻴ".  The rest is normal
ascii
data.

Let me know if this looks like a bug or not.  If not, I'll
keep
hacking away on my end.  Sorry if I left anything out - I'm
new to
this whole unicode thing.


--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the
Google Groups "Django users" group.
To post to this group, send email to django-usersgooglegroups.com
To unsubscribe from this group, send email to
django-users-unsubscribegooglegroups.com
For more options, visit this group at htt
p://groups.google.com/group/django-users?hl=en
-~----------~----~----~----~------~----~------~--~---

Does anyone have any idea about this one?  I'm kind of in a
bind on
this one.

Thanks!

Gabe

On Aug 30, 2:12 pm, mynameisgabe <mynameisg...gmail.com> wrote:
> Hello All-
>
> I've been told that I may have encountered abug- I'm
trying to
> serialize a model that includes fields that could
beunicodedata.
> Here is the information I collected.  Let me know if
there's anything
> else that would be helpful.
>
> Specs of the local dev workstation - having the same
issues in prod
> using Linux/FastCGI:
>
> Windows XP Home Edition Version 2002 SP 2
> Python 2.4.4
> Latest Django from SVN: 0.97-pre-SVN-6024
> SQLite3
> Using internal django web server (for dev)
>
> Here's the traceback:
>
> Traceback (most recent call last):
> File
"C:Python24libsite-packagesdjangocorehandlersbas
e.py" in
> get_response
>  77. response = callback(request, *callback_args,
**callback_kwargs)
> File "C:Documents and SettingsGabe HarrimanMy
> Documentsdevtextycmsviews.py" in syndicate
>  178. comments =
serializers.serialize("json",
> texty.comment_set.all(),
fields=('name','comment','created_date',),
> ensure_ascii=False)
> File
"C:Python24libsite-packagesdjangocoreserializers
> __init__.py"
> in serialize
>  67. s.serialize(queryset, **options)
> File
"C:Python24libsite-packagesdjangocoreserializers
base.py"
> in serialize
>  50. self.end_serialization()
> File
"C:Python24libsite-packagesdjangocoreserializers
json.py"
> in end_serialization
>  26. simplejson.dump(self.objects, self.stream,
> cls=DjangoJSONEncoder, **self.options)
> File
"C:Python24libsite-packagesdjangoutilssimplejson
> __init__.py" in dump
>  139. fp.write(chunk)
>
>  UnicodeEncodeError at
/cms/syndicate/d5f8eba5-d4a8-4590-
> a3d9-4d9e34503c5b.js
>  'ascii' codec can't encode characters in position 1-7:
ordinal not
> in range(128)
>
> Unicodeerror hint:
> The string that could not be encoded/decoded was:
"ﺥﺾﻤﻼﺚﻴﻴ"
>
> As you can see from above, the error starts here:
>
> comments = serializers.serialize("json",
texty.comment_set.all(),
> fields=('name','comment','created_date',),
ensure_ascii=False)
>
> The data in the name field is
"ﺥﺾﻤﻼﺚﻴﻴ".  The rest is normal
ascii
> data.
>
> Let me know if this looks like abugor not.  If not,
I'll keep
> hacking away on my end.  Sorry if I left anything out -
I'm new to
> this wholeunicodething.


--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the
Google Groups "Django users" group.
To post to this group, send email to django-usersgooglegroups.com
To unsubscribe from this group, send email to
django-users-unsubscribegooglegroups.com
For more options, visit this group at htt
p://groups.google.com/group/django-users?hl=en
-~----------~----~----~----~------~----~------~--~---

After doing more searching I ran across this post, claiming
that the
ensure_ascii argument was worthless and caused the bug.  I
removed it
from the argument list and all is well!!

http://groups.googl
e.com/group/django-users/browse_thread/thread/87b1478c02d743
e0/e4849fcf74147466?lnk=gst&q=unicode+json&rnum=9#e4
849fcf74147466

Is this bug documented?

G

On Aug 30, 2:12 pm, mynameisgabe <mynameisg...gmail.com> wrote:
> Hello All-
>
> I've been told that I may have encountered abug- I'm
trying to
> serialize a model that includes fields that could
beunicodedata.
> Here is the information I collected.  Let me know if
there's anything
> else that would be helpful.
>
> Specs of the local dev workstation - having the same
issues in prod
> using Linux/FastCGI:
>
> Windows XP Home Edition Version 2002 SP 2
> Python 2.4.4
> Latest Django from SVN: 0.97-pre-SVN-6024
> SQLite3
> Using internal django web server (for dev)
>
> Here's the traceback:
>
> Traceback (most recent call last):
> File
"C:Python24libsite-packagesdjangocorehandlersbas
e.py" in
> get_response
>  77. response = callback(request, *callback_args,
**callback_kwargs)
> File "C:Documents and SettingsGabe HarrimanMy
> Documentsdevtextycmsviews.py" in syndicate
>  178. comments =
serializers.serialize("json",
> texty.comment_set.all(),
fields=('name','comment','created_date',),
> ensure_ascii=False)
> File
"C:Python24libsite-packagesdjangocoreserializers
> __init__.py"
> in serialize
>  67. s.serialize(queryset, **options)
> File
"C:Python24libsite-packagesdjangocoreserializers
base.py"
> in serialize
>  50. self.end_serialization()
> File
"C:Python24libsite-packagesdjangocoreserializers
json.py"
> in end_serialization
>  26. simplejson.dump(self.objects, self.stream,
> cls=DjangoJSONEncoder, **self.options)
> File
"C:Python24libsite-packagesdjangoutilssimplejson
> __init__.py" in dump
>  139. fp.write(chunk)
>
>  UnicodeEncodeError at
/cms/syndicate/d5f8eba5-d4a8-4590-
> a3d9-4d9e34503c5b.js
>  'ascii' codec can't encode characters in position 1-7:
ordinal not
> in range(128)
>
> Unicodeerror hint:
> The string that could not be encoded/decoded was:
"ﺥﺾﻤﻼﺚﻴﻴ"
>
> As you can see from above, the error starts here:
>
> comments = serializers.serialize("json",
texty.comment_set.all(),
> fields=('name','comment','created_date',),
ensure_ascii=False)
>
> The data in the name field is
"ﺥﺾﻤﻼﺚﻴﻴ".  The rest is normal
ascii
> data.
>
> Let me know if this looks like abugor not.  If not,
I'll keep
> hacking away on my end.  Sorry if I left anything out -
I'm new to
> this wholeunicodething.


--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the
Google Groups "Django users" group.
To post to this group, send email to django-usersgooglegroups.com
To unsubscribe from this group, send email to
django-users-unsubscribegooglegroups.com
For more options, visit this group at htt
p://groups.google.com/group/django-users?hl=en
-~----------~----~----~----~------~----~------~--~---