Special file "nul" in Windows and os.stat

Hi, people!

I'm following the issue 1311: http://bugs.python.o
rg/issue1311

There (and always talking in windows), the OP says the in
Py2.4
os.path.exists("nul") returned True and now in 2.5
returns False. Note
that "nul" is an special file, something like
/dev/null.

We made some tests, and we have inconsisten behaviour in
previous
Python versions. For example, in Py2.3.5 in my machine I get
a False,
as in Py2.5. But other person in the bug, gets True in 2.3.3
and
False in 2.5.

Even the OP has differents result for the same Python 2.4 in
different machines.

Right now (but don't know exactly since when), Python relies
in
kernel32.dll functions to make the stat on the file (if stat
raises an
error, os.path.exists says that the file does not exist). Of
course,
if I call to this function separately, I have the same
behaviour.

So, the question is what we should do?:

1. Rely on the kernel32 function and behaves like it says?

2. Return a fixed response for this special file
"nul"?

Personally, I prefer the first one, but it changed the
semantic of
os.path.exists("nul") (but this semantic is not
clear, as we get
different behaviour in different Python versions and
windows
versions).

Thank you very much!

Regards,

-- 
.    Facundo

Blog: http://www.tanique
til.com.ar/plog/
PyAr: http://www.python.org/ar/
_______________________________________________
Python-Dev mailing list
Python-Devpython.org
ht
tp://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/p
ython-dev/nessto%40sharedlog.com

On Oct 24, 2007, at 4:23 PM, Facundo Batista wrote:
> There (and always talking in windows), the OP says the
in Py2.4
> os.path.exists("nul") returned True and now
in 2.5 returns False. Note
> that "nul" is an special file, something like
/dev/null.

It's special, but in a different way.  /dev/null really
exists in the  
Unix filesystem; "nul" is more magical than that.

What's more, it has peers: "prn", "com1"
and others like that.

I don't know what the right way to handle these is (I'm no
Windows  
guru, or even regular user), but it's important to realize
that the  
pain of the specialness isn't limited.  


   -Fred

-- 
Fred Drake   <fdrake at acm.org>



_______________________________________________
Python-Dev mailing list
Python-Devpython.org
ht
tp://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/p
ython-dev/nessto%40sharedlog.com

> So, the question is what we should do?:
> 
> 1. Rely on the kernel32 function and behaves like it
says?
> 
> 2. Return a fixed response for this special file
"nul"?
> 
> Personally, I prefer the first one, but it changed the
semantic of
> os.path.exists("nul") (but this semantic is
not clear, as we get
> different behaviour in different Python versions and
windows
> versions).

Note that the same issue would exist for 'aux', 'con' and
'prn' too -
'comXX' 'lptXX' 'clock$' also seem to get special treatment.
 I agree it is
unfortunate that the behaviour has changed, but these
special names are
broken enough on Windows that (1) seems the sanest thing to
do.

Mark

_______________________________________________
Python-Dev mailing list
Python-Devpython.org
ht
tp://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/p
ython-dev/nessto%40sharedlog.com

Fred Drake wrote:
> On Oct 24, 2007, at 4:23 PM, Facundo Batista wrote:
>> There (and always talking in windows), the OP says
the in Py2.4
>> os.path.exists("nul") returned True and
now in 2.5 returns False. Note
>> that "nul" is an special file, something
like /dev/null.
> 
> It's special, but in a different way.  /dev/null really
exists in the  
> Unix filesystem; "nul" is more magical than
that.
> 
> What's more, it has peers: "prn",
"com1" and others like that.

It's even worse than that, because file extensions are
ignored in this 
magical-ness:

C:Documents and SettingsUser>type nul

C:Documents and SettingsUser>type nul.lst

C:Documents and SettingsUser>type foo.lst
The system cannot find the file specified.

> I don't know what the right way to handle these is (I'm
no Windows  
> guru, or even regular user), but it's important to
realize that the  
> pain of the specialness isn't limited.  

http://www.microso
ft.com/technet/prodtechnol/Windows2000Pro/reskit/part3/proch
17.mspx?mfr=true 
gives the list as CON, AUX, COM1, COM2, COM3, COM4, LPT1,
LPT2, LPT3, 
PRN, NUL; but I can't imagine testing against that list
would be the 
best idea.  For example, 
http://www
.microsoft.com/technet/solutionaccelerators/cits/interopmigr
ation/unix/unixbld/unixbld4.mspx 
adds CLOCK$, among others (although I don't find CLOCK$ to
be special, 
it's rumored to be an NT only thing, and I'm running XP). 
So I think 
implementing Facundo's option 2 (test for "nul")
will not work in the 
general case for finding "special files" (don't
forget to throw in mixed 
case names).  I hate to think of trying to match Windows'
behavior if 
there are multiple dots in the name.

I think I'd leave the current behavior of calling the kernel
function, 
even though it varies based on Windows version (if I'm
reading the issue 
correctly).

Eric.


_______________________________________________
Python-Dev mailing list
Python-Devpython.org
ht
tp://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/p
ython-dev/nessto%40sharedlog.com

Fred Drake wrote:
> It's special, but in a different way.  /dev/null really
exists in the  
> Unix filesystem; "nul" is more magical than
that.
> 
> What's more, it has peers: "prn",
"com1" and others like that.
> 

For the record, the fixed names 'aux', 'con', 'nul', and
'prn' along 
with the set of 'com[0-9]' and 'lpt[0-9]' names that are
reserved. And 
for that matter, any of those with an extension is reserved
as well. 
These files always exist as far as I am concerned (where
existence is 
defined by your ability to open() them).

def is_special_on_win32(name):
     import os.path, re
     name = os.path.basename(name)
     return
(re.match('(nul|prn|aux|con|com[0-9]|lpt[0-9])(..*)?$',
                      name) is not None)

-- 
Scott Dial
scottscottdial.com
scodialcs.indiana.edu
_______________________________________________
Python-Dev mailing list
Python-Devpython.org
ht
tp://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/p
ython-dev/nessto%40sharedlog.com

> So, the question is what we should do?:

Before this question can be answered, I think we need to
fully
understand what precisely is happening in 2.4, and what
precisely
is happening in 2.5.

AFAICT, it is *not* the case that Python 2.4 (indirectly)
has
hard-coded the names CON, PRN, NUL etc. in the C library.
Instead,
Python 2.4 *also* relies on kernel32 functions to determine
that
these files "exist".

My question now is what specific kernel32 functions Python
2.4
calls to determine that NUL is a file; before that question
is sufficiently answered, I don't think any action should
be
taken.

Regards,
Martin
_______________________________________________
Python-Dev mailing list
Python-Devpython.org
ht
tp://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/p
ython-dev/nessto%40sharedlog.com

On Oct 24, 2007 11:05 PM, "Martin v. Löwis"
<martinv.loewis.de> wrote:
> > So, the question is what we should do?:
>
> Before this question can be answered, I think we need
to fully
> understand what precisely is happening in 2.4, and what
precisely
> is happening in 2.5.
>
> AFAICT, it is *not* the case that Python 2.4
(indirectly) has
> hard-coded the names CON, PRN, NUL etc. in the C
library. Instead,
> Python 2.4 *also* relies on kernel32 functions to
determine that
> these files "exist".
>
> My question now is what specific kernel32 functions
Python 2.4
> calls to determine that NUL is a file; before that
question
> is sufficiently answered, I don't think any action
should be
> taken.
>


os.path.exist() in win32 just calls os.stat() and decides it
doesn't
exist if an error is returned. os.stat() uses the vcrt
stat()in 2.4,
but 2.5 implements it directly in terms of win32 api to deal
with
limitations in the vcrt implementation.

The hand-rolled stat uses GetFileAttributesEx, which returns
0 for the
special filenames, with an error code of "The parameter
is incorrect"
(87), which is why os.path.exists() claims it doesn't
exist.

Interestingly, plain old GetFileAttributes() works, and
returns
FILE_ATTRIBUTE_ARCHIVE for them.
_______________________________________________
Python-Dev mailing list
Python-Devpython.org
ht
tp://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/p
ython-dev/nessto%40sharedlog.com

On Oct 30, 2007 4:10 PM, "Martin v. Löwis"
<martinv.loewis.de> wrote:
> >> My question now is what specific kernel32
functions Python 2.4
> >> calls to determine that NUL is a file; before
that question
> >> is sufficiently answered, I don't think any
action should be
> >> taken.
> >>
> >
> > os.path.exist() in win32 just calls os.stat() and
decides it doesn't
> > exist if an error is returned. os.stat() uses the
vcrt stat()in 2.4,
> > but 2.5 implements it directly in terms of win32
api to deal with
> > limitations in the vcrt implementation.
>
> That doesn't really answer the question, though - you
merely state
> that Python 2.4 calls the CRT, but then my question is
still what
> kernel32 functions are called to have stat on NUL
succeed.
>

I'm not 100% (it calls it through a function pointer and I'm
not sure
I tracked it down correctly), but I think it calls it
through the C
stat() function. In other words, it doesn't use any kernel32
functions
directly, it calls the stat() that's exported from the
MSVCRT.

> > Interestingly, plain old GetFileAttributes()
works, and returns
> > FILE_ATTRIBUTE_ARCHIVE for them.
>
> What about the other attributes (like modification
time, size, etc)?
>

GetFileAttributes() doesn't return those, just the FAT
filesystem
attributes. GetFileSize and GetFileTime fail.

> Regards,
> Martin
>
_______________________________________________
Python-Dev mailing list
Python-Devpython.org
ht
tp://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/p
ython-dev/nessto%40sharedlog.com

2007/11/3, "Martin v. Löwis" <martinv.loewis.de>:

> > GetFileAttributes() doesn't return those, just the
FAT filesystem
> > attributes. GetFileSize and GetFileTime fail.
>
> Ok, so how does msvcrt stat() manage to fill these
fields if those
> functions fail?

Beyond the question to this specific question, I do not like
the
inconsistency of windows with itself during time and
versions.

As Mask Hammond said, I think that we should rely on what
windows is
saying to us as strict as possible. If windows change its
behaviour,
ok, I do not think that we need to "patch" these
behaviour holes.

What do you think? Is a mistake to adhere to windows
behaviour?

Regards,

-- 
.    Facundo

Blog: http://www.tanique
til.com.ar/plog/
PyAr: http://www.python.org/ar/
_______________________________________________
Python-Dev mailing list
Python-Devpython.org
ht
tp://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/p
ython-dev/nessto%40sharedlog.com

> As Mask Hammond said, I think that we should rely on
what windows is
> saying to us as strict as possible. If windows change
its behaviour,
> ok, I do not think that we need to "patch"
these behaviour holes.
> 
> What do you think? Is a mistake to adhere to windows
behaviour?

We certainly should rely on the Windows behavior. The next
question then
is: What exactly *is* "the Windows behavior".
Windows is not just
inconsistent across versions, but apparently so even within
a single
version.

IIUC, GetFileAttributes and FindFirstFile both claim that
NUL exists,
whereas GetFileAttributesEx claims that it doesn't exist,
all in a
single version, and all is Windows API.

Please understand that Python 2.4 *also* adheres to Windows
behavior.

Regards,
Martin
_______________________________________________
Python-Dev mailing list
Python-Devpython.org
ht
tp://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/p
ython-dev/nessto%40sharedlog.com