YUI file upload and form errors

Hi,
I'm having a problem with Django "HTML escaping"
JSON data sent in
response to an asynchronous form submission when the form
has an
<input type="file" ...> field. Forms that
don't have a file field
yield proper responses, and when Javascript is disabled on
the browser
normal form submissions work as well.

I'm using the Yahoo User Interface library, specifically the
Dialog &
Connection Manager components, to send and receive
asynchronous
messages to/from my view.

As an example, the JSON response seen by the javascript
might be:

   "<pre>{"valid": false,
"errors": {"options":
"&lt;ul class=
"errorlist\"&gt;&lt;li&gt;This field
is required.&lt;/li&gt;&lt;/
ul&gt;"}}</pre>"

when it should be:

   "{"valid": false, "errors":
{"options": "<ul class=
"errorlist\"><li>This field is
required.</li></ul>"}}"

You can see that the Django system encapsulates the entire
response in
<pre></pre> tags. Additionally, the underlying
error message HTML is
also escaped.

Does anyone know why this escaping might be happening? Can
you suggest
how I might avoid the escaping of the response?

Graham
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the
Google Groups "Django users" group.
To post to this group, send email to django-usersgooglegroups.com
To unsubscribe from this group, send email to
django-users-unsubscribegooglegroups.com
For more options, visit this group at htt
p://groups.google.com/group/django-users?hl=en
-~----------~----~----~----~------~----~------~--~---

On Wed, 2008-01-02 at 15:38 -0800, grahamu wrote:
> Hi,
> I'm having a problem with Django "HTML
escaping" JSON data sent in
> response to an asynchronous form submission when the
form has an
> <input type="file" ...> field. Forms
that don't have a file field
> yield proper responses, and when Javascript is disabled
on the browser
> normal form submissions work as well.
> 
> I'm using the Yahoo User Interface library,
specifically the Dialog &
> Connection Manager components, to send and receive
asynchronous
> messages to/from my view.
> 
> As an example, the JSON response seen by the javascript
might be:
> 
>    "<pre>{"valid": false,
"errors": {"options":
"&lt;ul class=
> "errorlist\"&gt;&lt;li&gt;This
field is required.&lt;/li&gt;&lt;/
> ul&gt;"}}</pre>"
> 
> when it should be:
> 
>    "{"valid": false,
"errors": {"options": "<ul
class=
> "errorlist\"><li>This field is
required.</li></ul>"}}"
> 
> You can see that the Django system encapsulates the
entire response in
> <pre></pre> tags. Additionally, the
underlying error message HTML is
> also escaped.
> 
> Does anyone know why this escaping might be happening?
Can you suggest
> how I might avoid the escaping of the response?

Both the "why" and the "how" are
documented in docs/templates_python.txt
in the source. The short answer is that any time a variable
is rendered
into a template auto-escaping is applied. If you don't want
this to
happen, you can mark the particular variable as safe from
further
escaping using either mark_safe() in your view (probably the
best
approach -- marking it safe as soon as you know that fact)
or in the
template with the "safe" filter ({{ some_var|safe
}}) or wrap an entire
section of the template within the {% autoescape off %} ...
{%
endautoescape %} template tag.

Regards,
Malcolm

-- 
A conclusion is the place where you got tired of thinking. 
http://www.pointy-s
tick.com/blog/


--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the
Google Groups "Django users" group.
To post to this group, send email to django-usersgooglegroups.com
To unsubscribe from this group, send email to
django-users-unsubscribegooglegroups.com
For more options, visit this group at htt
p://groups.google.com/group/django-users?hl=en
-~----------~----~----~----~------~----~------~--~---

On Jan 2, 4:43 pm, Malcolm Tredinnick <malc...pointy-stick.com>
wrote:
> On Wed, 2008-01-02 at 15:38 -0800, grahamu wrote:
> > Hi,
> > I'm having a problem with Django "HTML
escaping" JSON data sent in
> > response to an asynchronous form submission when
the form has an
> > <input type="file" ...> field.
Forms that don't have a file field
> > yield proper responses, and when Javascript is
disabled on the browser
> > normal form submissions work as well.
>
> > I'm using the Yahoo User Interface library,
specifically the Dialog &
> > Connection Manager components, to send and receive
asynchronous
> > messages to/from my view.
>
> > As an example, the JSON response seen by the
javascript might be:
>
> >    "<pre>{"valid": false,
"errors": {"options":
"&lt;ul class=
> >
"errorlist\"&gt;&lt;li&gt;This field
is required.&lt;/li&gt;&lt;/
> > ul&gt;"}}</pre>"
>
> > when it should be:
>
> >    "{"valid": false,
"errors": {"options": "<ul
class=
> > "errorlist\"><li>This field
is required.</li></ul>"}}"
>
> > You can see that the Django system encapsulates
the entire response in
> > <pre></pre> tags. Additionally, the
underlying error message HTML is
> > also escaped.
>
> > Does anyone know why this escaping might be
happening? Can you suggest
> > how I might avoid the escaping of the response?
>
> Both the "why" and the "how" are
documented in docs/templates_python.txt
> in the source. The short answer is that any time a
variable is rendered
> into a template auto-escaping is applied. If you don't
want this to
> happen, you can mark the particular variable as safe
from further
> escaping using either mark_safe() in your view
(probably the best
> approach -- marking it safe as soon as you know that
fact) or in the
> template with the "safe" filter ({{
some_var|safe }}) or wrap an entire
> section of the template within the {% autoescape off %}
... {%
> endautoescape %} template tag.
>
> Regards,
> Malcolm
>
> --
> A conclusion is the place where you got tired of
thinking.http://www.pointy-s
tick.com/blog/

Malcom,
Thanks for your speedy response. I don't believe this is a
template
issue as I'm returning a JSON response and not rendering to
a
template.

The view code logic:

   if not form.is_valid():
      return JSONFormErrors(form)
   else:
      # return some other data

and:

   def JSONFormErrors(form):
      errors = form.errors
      response_dict = {}
      response_dict.update({'valid': not errors})
      response_dict.update({'errors': errors})
      return JsonResponse(response_dict)

   class JsonResponse(HttpResponse):
      def __init__(self, data):
         HttpResponse.__init__(self, json_encode(data),
mimetype='application/javascript')

json_encode is a version of Wolfgang Kriesing's encoder
(http://
dpaste.com/hold/25654/).

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the
Google Groups "Django users" group.
To post to this group, send email to django-usersgooglegroups.com
To unsubscribe from this group, send email to
django-users-unsubscribegooglegroups.com
For more options, visit this group at htt
p://groups.google.com/group/django-users?hl=en
-~----------~----~----~----~------~----~------~--~---

> On Jan 2, 4:43 pm, Malcolm Tredinnick <malc...pointy-stick.com>
> wrote:
>
>
>
> > On Wed, 2008-01-02 at 15:38 -0800, grahamu wrote:
> > > Hi,
> > > I'm having a problem with Django "HTML
escaping" JSON data sent in
> > > response to an asynchronous form submission
when the form has an
> > > <input type="file" ...>
field. Forms that don't have a file field
> > > yield proper responses, and when Javascript
is disabled on the browser
> > > normal form submissions work as well.
>
> > > I'm using the Yahoo User Interface library,
specifically the Dialog &
> > > Connection Manager components, to send and
receive asynchronous
> > > messages to/from my view.
>
> > > As an example, the JSON response seen by the
javascript might be:
>
> > >    "<pre>{"valid":
false, "errors": {"options":
"&lt;ul class=
> > >
"errorlist\"&gt;&lt;li&gt;This field
is required.&lt;/li&gt;&lt;/
> > > ul&gt;"}}</pre>"
>
> > > when it should be:
>
> > >    "{"valid": false,
"errors": {"options": "<ul
class=
> > > "errorlist\"><li>This
field is required.</li></ul>"}}"
>
> > > You can see that the Django system
encapsulates the entire response in
> > > <pre></pre> tags. Additionally,
the underlying error message HTML is
> > > also escaped.
>
> > > Does anyone know why this escaping might be
happening? Can you suggest
> > > how I might avoid the escaping of the
response?
>
> > Both the "why" and the "how"
are documented in docs/templates_python.txt
> > in the source. The short answer is that any time a
variable is rendered
> > into a template auto-escaping is applied. If you
don't want this to
> > happen, you can mark the particular variable as
safe from further
> > escaping using either mark_safe() in your view
(probably the best
> > approach -- marking it safe as soon as you know
that fact) or in the
> > template with the "safe" filter ({{
some_var|safe }}) or wrap an entire
> > section of the template within the {% autoescape
off %} ... {%
> > endautoescape %} template tag.
>
> > Regards,
> > Malcolm
>
> > --
> > A conclusion is the place where you got tired of
thinking.http://www.pointy-s
tick.com/blog/
>
> Malcom,
> Thanks for your speedy response. I don't believe this
is a template
> issue as I'm returning a JSON response and not
rendering to a
> template.
>
> The view code logic:
>
>    if not form.is_valid():
>       return JSONFormErrors(form)
>    else:
>       # return some other data
>
> and:
>
>    def JSONFormErrors(form):
>       errors = form.errors
>       response_dict = {}
>       response_dict.update({'valid': not errors})
>       response_dict.update({'errors': errors})
>       return JsonResponse(response_dict)
>
>    class JsonResponse(HttpResponse):
>       def __init__(self, data):
>          HttpResponse.__init__(self,
json_encode(data),
> mimetype='application/javascript')
>
> json_encode is a version of Wolfgang Kriesing's encoder
(http://
> dpaste.com/hold/25654/).

Just to be clear, the encoding problem _does not_ occur when
the form
does not have an <input type="file"> field.
Form errors are returned
in the JSON string in perfect form, no HTML escaping
happens. And the
view logic (code path) is identical whether or not a file
input field
is present in the form.
Graham

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the
Google Groups "Django users" group.
To post to this group, send email to django-usersgooglegroups.com
To unsubscribe from this group, send email to
django-users-unsubscribegooglegroups.com
For more options, visit this group at htt
p://groups.google.com/group/django-users?hl=en
-~----------~----~----~----~------~----~------~--~---

On Jan 2, 7:13 pm, grahamu <gra...flyingcracker.com>
wrote:
> > On Jan 2, 4:43 pm, Malcolm Tredinnick
<malc...pointy-stick.com>
> > wrote:
>
> > > On Wed, 2008-01-02 at 15:38 -0800, grahamu
wrote:
> > > > Hi,
> > > > I'm having a problem with Django
"HTML escaping" JSON data sent in
> > > > response to an asynchronous form
submission when the form has an
> > > > <input type="file" ...>
field. Forms that don't have a file field
> > > > yield proper responses, and when
Javascript is disabled on the browser
> > > > normal form submissions work as well.
>
> > > > I'm using the Yahoo User Interface
library, specifically the Dialog &
> > > > Connection Manager components, to send
and receive asynchronous
> > > > messages to/from my view.
>
> > > > As an example, the JSON response seen by
the javascript might be:
>
> > > >   
"<pre>{"valid": false,
"errors": {"options":
"&lt;ul class=
> > > >
"errorlist\"&gt;&lt;li&gt;This field
is required.&lt;/li&gt;&lt;/
> > > > ul&gt;"}}</pre>"
>
> > > > when it should be:
>
> > > >    "{"valid": false,
"errors": {"options": "<ul
class=
> > > >
"errorlist\"><li>This field is
required.</li></ul>"}}"
>
> > > > You can see that the Django system
encapsulates the entire response in
> > > > <pre></pre> tags.
Additionally, the underlying error message HTML is
> > > > also escaped.
>
> > > > Does anyone know why this escaping might
be happening? Can you suggest
> > > > how I might avoid the escaping of the
response?
>
> > > Both the "why" and the
"how" are documented in docs/templates_python.txt
> > > in the source. The short answer is that any
time a variable is rendered
> > > into a template auto-escaping is applied. If
you don't want this to
> > > happen, you can mark the particular variable
as safe from further
> > > escaping using either mark_safe() in your
view (probably the best
> > > approach -- marking it safe as soon as you
know that fact) or in the
> > > template with the "safe" filter ({{
some_var|safe }}) or wrap an entire
> > > section of the template within the {%
autoescape off %} ... {%
> > > endautoescape %} template tag.
>
> > > Regards,
> > > Malcolm
>
> > > --
> > > A conclusion is the place where you got tired
of thinking.http://www.pointy-s
tick.com/blog/
>
> > Malcom,
> > Thanks for your speedy response. I don't believe
this is a template
> > issue as I'm returning a JSON response and not
rendering to a
> > template.
>
> > The view code logic:
>
> >    if not form.is_valid():
> >       return JSONFormErrors(form)
> >    else:
> >       # return some other data
>
> > and:
>
> >    def JSONFormErrors(form):
> >       errors = form.errors
> >       response_dict = {}
> >       response_dict.update({'valid': not errors})
> >       response_dict.update({'errors': errors})
> >       return JsonResponse(response_dict)
>
> >    class JsonResponse(HttpResponse):
> >       def __init__(self, data):
> >          HttpResponse.__init__(self,
json_encode(data),
> > mimetype='application/javascript')
>
> > json_encode is a version of Wolfgang Kriesing's
encoder (http://
> > dpaste.com/hold/25654/).
>
> Just to be clear, the encoding problem _does not_ occur
when the form
> does not have an <input type="file">
field. Form errors are returned
> in the JSON string in perfect form, no HTML escaping
happens. And the
> view logic (code path) is identical whether or not a
file input field
> is present in the form.
> Graham

I tried marking all error strings as safe...

def JSONFormErrors(form):
    from django.utils.safestring import mark_safe
    errors = form.errors

    # mark each value (string) in the error dictionary as
safe
    for key in errors:
        mark_safe(errors[key])

    response_dict = {}
    response_dict.update({'valid': not errors})
    response_dict.update({'errors': errors})
    return = JsonResponse(response_dict)

...but that had no effect. Debugging this is tough, I can't
seem to
find the code which is escaping the output. If I can find
that perhaps
I can determine why it is getting escaped.

Any ideas?
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the
Google Groups "Django users" group.
To post to this group, send email to django-usersgooglegroups.com
To unsubscribe from this group, send email to
django-users-unsubscribegooglegroups.com
For more options, visit this group at htt
p://groups.google.com/group/django-users?hl=en
-~----------~----~----~----~------~----~------~--~---

On Wed, 2008-01-02 at 15:38 -0800, grahamu wrote:
[...]
> As an example, the JSON response seen by the javascript
might be:
> 
>    "<pre>{"valid": false,
"errors": {"options":
"&lt;ul class=
> "errorlist\"&gt;&lt;li&gt;This
field is required.&lt;/li&gt;&lt;/
> ul&gt;"}}</pre>"
> 
> when it should be:
> 
>    "{"valid": false,
"errors": {"options": "<ul
class=
> "errorlist\"><li>This field is
required.</li></ul>"}}"
> 
> You can see that the Django system encapsulates the
entire response in
> <pre></pre> tags. Additionally, the
underlying error message HTML is
> also escaped.

You might want to double-check that you're debugging the
correct
problem. There is no code in Django that is adding the
"pre" tags here.
The only occurrence of the string "<pre" in the
source code is in a
docstring for the "debug" template tag. So working
out at which step
that extra content is being added will no doubt help you
zero in on the
problem code.

Regards,
Malcolm

-- 
Two wrongs are only the beginning. 
http://www.pointy-s
tick.com/blog/


--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the
Google Groups "Django users" group.
To post to this group, send email to django-usersgooglegroups.com
To unsubscribe from this group, send email to
django-users-unsubscribegooglegroups.com
For more options, visit this group at htt
p://groups.google.com/group/django-users?hl=en
-~----------~----~----~----~------~----~------~--~---