List Info

Thread: Problems with file names in UTF-8 on Windows




Problems with file names in UTF-8 on Windows
user name
2006-08-16 17:06:15
 > Here comes my revised/extended patch.
 > [...]
 > Roland
 >
Sorry, but it will not work:

1. function MultiByteToWideChar() in
__xmlIOWin32UTF8ToWChar() converts
    native string to unicode even if not all byte sequences
are valid;
    so, you need use flag MB_ERR_INVALID_CHARS to ensure,
    that whole string looks like utf-8;
   (but it not guarantee, that source string is really
utf-8)
2. functions _wstat() / _wfopen() are implemented in
msvcrt.dll on 
Windows 9.x,
    but they use unicode variants  of  Win32 API  functions,
which also 
exists in
    kernel32.dll, but simply returns error "not
implemented";
    IMHO, the only way to solve this problem is to
    detect platform using GetVersionEx()
3. one time detection of function presence is good idea;
    but can you reassure me, that using of static variables
will not
    led to problems in multithreaded environment?
    may be, it is more safe to detect platform during
    library initialization in main thread?

PS.
Probably, I am excessively fastidious. Sorry once again.
But I don't like that some functions in libxml become
platform dependent and use heuristic tricks.

Really include in documentation note about using
file names on different platforms?

Is finer decision possible?

_______________________________________________
xml mailing list, project page  http://xmlsoft.org/
xmlgnome.org
http://mai
l.gnome.org/mailman/listinfo/xml
Problems with file names in UTF-8 on Windows
user name
2006-08-21 08:18:26

Hi Daniel and Emelyanov,

At present I am damned overloaded. I will come back to your messages ASAP...

Roland
Problems with file names in UTF-8 on Windows
user name
2006-08-21 08:30:06
On Mon, Aug 21, 2006 at 10:18:26AM +0200, Roland Schwingel
wrote:
> Hi Daniel and Emelyanov,
> 
> At present I am damned overloaded. I will come back to
your messages 
> ASAP...

  Thanks, hopefully within a couple of weeks because pushing
a new libxml2
version out would make sense around the end of the month.

Daniel

-- 
Red Hat Virtualization group http://redhat.com/v
irtualization/
Daniel Veillard      | virtualization library  http://libvirt.org/
veillardredhat.com  | libxml GNOME XML XSLT toolkit  http://xmlsoft.org/
http://veillard.com/ |
Rpmfind RPM search engine  http://rpmfind.net/
_______________________________________________
xml mailing list, project page  http://xmlsoft.org/
xmlgnome.org
http://mai
l.gnome.org/mailman/listinfo/xml
Problems with file names in UTF-8 on Windows
user name
2006-08-21 08:18:26

Hi Daniel and Emelyanov,

At present I am damned overloaded. I will come back to your messages ASAP...

Roland
Problems with file names in UTF-8 on Windows
user name
2006-08-21 08:30:06
On Mon, Aug 21, 2006 at 10:18:26AM +0200, Roland Schwingel
wrote:
> Hi Daniel and Emelyanov,
> 
> At present I am damned overloaded. I will come back to
your messages 
> ASAP...

  Thanks, hopefully within a couple of weeks because pushing
a new libxml2
version out would make sense around the end of the month.

Daniel

-- 
Red Hat Virtualization group http://redhat.com/v
irtualization/
Daniel Veillard      | virtualization library  http://libvirt.org/
veillardredhat.com  | libxml GNOME XML XSLT toolkit  http://xmlsoft.org/
http://veillard.com/ |
Rpmfind RPM search engine  http://rpmfind.net/
_______________________________________________
xml mailing list, project page  http://xmlsoft.org/
xmlgnome.org
http://mai
l.gnome.org/mailman/listinfo/xml
Problems with file names in UTF-8 on Windows
user name
2006-08-30 12:09:11

Hi...

Daniel Veillard <veillardredhat.com> wrote on 21.08.2006 10:30:06:
> On Mon, Aug 21, 2006 at 10:18:26AM +0200, Roland Schwingel wrote:
&gt; > At present I am damned overloaded. I will come back to your messages
> > ASAP...
&gt;
>   Thanks, hopefully within a couple of weeks because pushing a new libxml2
&gt; version out would make sense around the end of the month.

Here is the reworked patch...
What has changed. It operates now in "paranoid" mode...
1) Handling of msvcrt moved (as requested by Daniel) to an own static function
   which is additionally secured against threadingproblems with a mutex.
2) There is an additional check against the Operating system version to
   force win9x to use ascii mode.
3) Added the MB_ERR_INVALID_CHARS to MultiByteToWideChar() calls,
   even IMHO not needed, because if it would produce incorrect wchars
   a subsequent call operating on that result would fail anyway, forcing
   windows path to fallback to ascii mode.

The patch is again made against vanilla libxml2 2.6.26

Hope I could help with this now,

Roland

Problems with file names in UTF-8 on Windows
user name
2006-08-30 13:13:27
On Wed, Aug 30, 2006 at 02:09:11PM +0200, Roland Schwingel
wrote:
> Hi...
> 
> Daniel Veillard <veillardredhat.com> wrote on
21.08.2006 10:30:06:
> > On Mon, Aug 21, 2006 at 10:18:26AM +0200, Roland
Schwingel wrote:
> > > At present I am damned overloaded. I will
come back to your messages 
> > > ASAP...
> > 
> >   Thanks, hopefully within a couple of weeks
because pushing a new 
> libxml2
> > version out would make sense around the end of the
month.
> 
> Here is the reworked patch...
> What has changed. It operates now in
"paranoid" mode...
> 1) Handling of msvcrt moved (as requested by Daniel) to
an own static 
> function
>    which is additionally secured against
threadingproblems with a mutex.
> 2) There is an additional check against the Operating
system version to
>    force win9x to use ascii mode.
> 3) Added the MB_ERR_INVALID_CHARS to
MultiByteToWideChar() calls, 
>    even IMHO not needed, because if it would produce
incorrect wchars
>    a subsequent call operating on that result would
fail anyway, forcing
>    windows path to fallback to ascii mode.
> 
> The patch is again made against vanilla libxml2 2.6.26

  Okay, this seems to answer all points raised in the past,
and is
changing windows only code (except for the duplicate
function removal),
so I applied it and commited it to CVS.

    thanks a lot !

Daniel

-- 
Red Hat Virtualization group http://redhat.com/v
irtualization/
Daniel Veillard      | virtualization library  http://libvirt.org/
veillardredhat.com  | libxml GNOME XML XSLT toolkit  http://xmlsoft.org/
http://veillard.com/ |
Rpmfind RPM search engine  http://rpmfind.net/
_______________________________________________
xml mailing list, project page  http://xmlsoft.org/
xmlgnome.org
http://mai
l.gnome.org/mailman/listinfo/xml
Problems with file names in UTF-8 on Windows
user name
2006-08-31 12:59:02

Hi...

xml-bouncesgnome.org wrote on 30.08.2006 15:13:27:
>
>   Okay, this seems to answer all points raised in the past, and is
&gt; changing windows only code (except for the duplicate function removal),
> so I applied it and commited it to CVS.
>
>     thanks a lot !
>
> Daniel

Not to forget, this should also fix bug #346367.

BTW: libxml2 appears also to run fine on Windows Vista Pre-RC1

Roland
Problems with file names in UTF-8 on Windows
user name
2006-08-31 13:07:49
On Thu, Aug 31, 2006 at 02:59:02PM +0200, Roland Schwingel
wrote:
> Hi...
> 
> xml-bouncesgnome.org wrote on 30.08.2006 15:13:27:
> > 
> >   Okay, this seems to answer all points raised in
the past, and is
> > changing windows only code (except for the
duplicate function removal),
> > so I applied it and commited it to CVS.
> > 
> >     thanks a lot !
> > 
> > Daniel
> 
> Not to forget, this should also fix bug #346367.

  Okay, done.

> BTW: libxml2 appears also to run fine on Windows Vista
Pre-RC1 

  I assume it's great 

Daniel

-- 
Red Hat Virtualization group http://redhat.com/v
irtualization/
Daniel Veillard      | virtualization library  http://libvirt.org/
veillardredhat.com  | libxml GNOME XML XSLT toolkit  http://xmlsoft.org/
http://veillard.com/ |
Rpmfind RPM search engine  http://rpmfind.net/
_______________________________________________
xml mailing list, project page  http://xmlsoft.org/
xmlgnome.org
http://mai
l.gnome.org/mailman/listinfo/xml
Problems with file names in UTF-8 on Windows
user name
2006-08-31 14:28:30

Hi...

Daniel Veillard <veillardredhat.com> wrote on 31.08.2006 15:07:49:
> On Thu, Aug 31, 2006 at 02:59:02PM +0200, Roland Schwingel wrote:
&gt; > BTW: libxml2 appears also to run fine on Windows Vista Pre-RC1
>
>   I assume it's great
Do you mean libxml2 or Vista?


libxml2: definitely YES!
Vista: Well... It is still beta and has it's bug, and I ask my self
       was all that really needed? Do I really need this stuff?
       Does the world need it (except the PC industry)?
       But thats an off topic discussion.

BTW: I just tested the limited set of commands I use in libxml2 (reading,parsing, getting data)
     no validation and so on, but I don't expect any problems up to now.

Roland
[1-10] [11]

about | contact  Other archives ( Real Estate discussion Medical topics )