Unicode strings through Database API

Hi,

I'm in need of a little education. This may be a python
issue rather
than a Django one, but is anybody can help I'd appreciate
it.

I've written an a little script which gets external data
and inserts it
into the database using the Django models. Unfortunately (or
fortunately, depending on your point of view) somebody has
put the word
'Résumé' into the data source that I'm using and those
accented
charactered are causing an exception when I call the save
method on my
object.

The field is declared as a CharField, and the data assigned
is a

Traceback (most recent call last):
  File "<console>", line 1, in ?
  File "/Users/pauls/Documents/Web
Development/myapp/myscript.py", line
37, in main
    e.save()
  File ".../django/db/models/base.py", line 199,
in save
    ','.join(placeholders)), db_values)
  File ".../django/db/backends/util.py", line
19, in execute
    self.db.queries.append({
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe9
in position
991: ordinal not in range(128)

I understand that it's having problems because the string
is identified
as an ASCII string and that character is beyond that, but
how would I
identify this string as (probably) ISO8859-1?

Paul


--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the
Google Groups "Django users" group.
To post to this group, send email to django-usersgooglegroups.com
To unsubscribe from this group, send email to
django-users-unsubscribegooglegroups.com
For more options, visit this group at http://gr
oups.google.com/group/django-users
-~----------~----~----~----~------~----~------~--~---

Ok, I've dug a little more and I realised I'd got the
wrong end of the
stick.

The string I'm assigning to the field is a unicode string.
Django is
trying to do a .encode('ascii') on it and failing.
Understandable
really

I assume I have to do a .encode('utf-8') on it before I
store it. Does
that tend to be the standard?


--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the
Google Groups "Django users" group.
To post to this group, send email to django-usersgooglegroups.com
To unsubscribe from this group, send email to
django-users-unsubscribegooglegroups.com
For more options, visit this group at http://gr
oups.google.com/group/django-users
-~----------~----~----~----~------~----~------~--~---

On 6/30/06, Paul <psargegmail.com> wrote:
> The string I'm assigning to the field is a unicode
string. Django is
> trying to do a .encode('ascii') on it and failing.
Understandable
> really

Are you sure it's the unicode type, and not just a byte
string?

> I assume I have to do a .encode('utf-8') on it before
I store it. Does
> that tend to be the standard?

It depends how you've set up your DB storage, I think. But
whatever
you do, keep your storage encoding consistent.  (You may
have some
pain if you already have non-ASCII stuff in the DB.)

I've seen some situations where multiple encodings were
stuck into the
same column, and you don't want to go there.

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the
Google Groups "Django users" group.
To post to this group, send email to django-usersgooglegroups.com
To unsubscribe from this group, send email to
django-users-unsubscribegooglegroups.com
For more options, visit this group at http://gr
oups.google.com/group/django-users
-~----------~----~----~----~------~----~------~--~---

Jeremy Dunck wrote:

> I've seen some situations where multiple encodings
were stuck into the
> same column, and you don't want to go there.

I can believe that :-/

That for the advice. Things seem to be working now I've
encoded into
utf-8. Even Django's admin interface seems to understand
utf-8
implicitly 


--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the
Google Groups "Django users" group.
To post to this group, send email to django-usersgooglegroups.com
To unsubscribe from this group, send email to
django-users-unsubscribegooglegroups.com
For more options, visit this group at http://gr
oups.google.com/group/django-users
-~----------~----~----~----~------~----~------~--~---