Created: (LUCENE-727) MMapDirectory can't create new index on Windows

MMapDirectory can't create new index on Windows
-----------------------------------------------

                 Key: LUCENE-727
                 URL: http:
//issues.apache.org/jira/browse/LUCENE-727
             Project: Lucene - Java
          Issue Type: Bug
          Components: Index
    Affects Versions: 2.0.0
         Environment: Windows XP, JDK 1.4
            Reporter: benson margulies


When I set the system property to request the use of the
mmap directory, and start building a large index, the
process dies with an IOException trying to delete a file.
Apparently, Lucene isn't closing down the memory map before
deleting the file.


-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the
administrators: http://issues.apache.org/jira/secure/Administrators.jspa

-
For more information on JIRA, see: http://www.atl
assian.com/software/jira

        

------------------------------------------------------------
---------
To unsubscribe, e-mail: java-dev-unsubscribelucene.apache.org
For additional commands, e-mail: java-dev-helplucene.apache.org

    [ http://issues.apache.org/jira/brows
e/LUCENE-727?page=comments#action_12453726 ] 
            
Hoss Man commented on LUCENE-727:
---------------------------------

Can you inlcude a full stack trace of the exception so we
can see what code path causes this problem?

it would be even better if you could inlcude a JUnit test
demonstrating the problem.

> MMapDirectory can't create new index on Windows
> -----------------------------------------------
>
>                 Key: LUCENE-727
>                 URL: http:
//issues.apache.org/jira/browse/LUCENE-727
>             Project: Lucene - Java
>          Issue Type: Bug
>          Components: Index
>    Affects Versions: 2.0.0
>         Environment: Windows XP, JDK 1.4
>            Reporter: benson margulies
>
> When I set the system property to request the use of
the mmap directory, and start building a large index, the
process dies with an IOException trying to delete a file.
Apparently, Lucene isn't closing down the memory map before
deleting the file.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the
administrators: http://issues.apache.org/jira/secure/Administrators.jspa

-
For more information on JIRA, see: http://www.atl
assian.com/software/jira

        

------------------------------------------------------------
---------
To unsubscribe, e-mail: java-dev-unsubscribelucene.apache.org
For additional commands, e-mail: java-dev-helplucene.apache.org

This is a known JDK issue with memory mapping a file.

The references may not be GC'd at the time the request to
delete is  
performed, and thus you will get an IOException.

See http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=471
5154


On Nov 27, 2006, at 4:26 PM, Hoss Man (JIRA) wrote:

>     [ http
://issues.apache.org/jira/browse/LUCENE-727? 
> page=comments#action_12453726 ]
>
> Hoss Man commented on LUCENE-727:
> ---------------------------------
>
> Can you inlcude a full stack trace of the exception so
we can see  
> what code path causes this problem?
>
> it would be even better if you could inlcude a JUnit
test  
> demonstrating the problem.
>
>> MMapDirectory can't create new index on Windows
>> -----------------------------------------------
>>
>>                 Key: LUCENE-727
>>                 URL: http:
//issues.apache.org/jira/browse/LUCENE-727
>>             Project: Lucene - Java
>>          Issue Type: Bug
>>          Components: Index
>>    Affects Versions: 2.0.0
>>         Environment: Windows XP, JDK 1.4
>>            Reporter: benson margulies
>>
>> When I set the system property to request the use
of the mmap  
>> directory, and start building a large index, the
process dies with  
>> an IOException trying to delete a file. Apparently,
Lucene isn't  
>> closing down the memory map before deleting the
file.
>
> -- 
> This message is automatically generated by JIRA.
> -
> If you think it was sent incorrectly contact one of the
 
> administrators: http://issues.a
pache.org/jira/secure/ 
> Administrators.jspa
> -
> For more information on JIRA, see: http://www.atlassian.com/ 
> software/jira
>
>
>
>
------------------------------------------------------------
---------
> To unsubscribe, e-mail: java-dev-unsubscribelucene.apache.org
> For additional commands, e-mail: java-dev-helplucene.apache.org
>

     [ http://issues.apache.org/jira/browse/LUCENE-727?page=all
 ]

benson margulies updated LUCENE-727:
------------------------------------

    Attachment: lucenewindowsmmaptest.tgz

Here you go. 

> MMapDirectory can't create new index on Windows
> -----------------------------------------------
>
>                 Key: LUCENE-727
>                 URL: http:
//issues.apache.org/jira/browse/LUCENE-727
>             Project: Lucene - Java
>          Issue Type: Bug
>          Components: Index
>    Affects Versions: 2.0.0
>         Environment: Windows XP, JDK 1.4
>            Reporter: benson margulies
>         Attachments: lucenewindowsmmaptest.tgz
>
>
> When I set the system property to request the use of
the mmap directory, and start building a large index, the
process dies with an IOException trying to delete a file.
Apparently, Lucene isn't closing down the memory map before
deleting the file.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the
administrators: http://issues.apache.org/jira/secure/Administrators.jspa

-
For more information on JIRA, see: http://www.atl
assian.com/software/jira

        

------------------------------------------------------------
---------
To unsubscribe, e-mail: java-dev-unsubscribelucene.apache.org
For additional commands, e-mail: java-dev-helplucene.apache.org

    [ http://issues.apache.org/jira/brows
e/LUCENE-727?page=comments#action_12454073 ] 
            
Hoss Man commented on LUCENE-727:
---------------------------------

Per robert engels on java-dev...

> This is a known JDK issue with memory mapping a file.
> 
> The references may not be GC'd at the time the request
to delete is
> performed, and thus you will get an IOException.
> 
> See http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=471
5154

...so it may not be possible to fix this.

> MMapDirectory can't create new index on Windows
> -----------------------------------------------
>
>                 Key: LUCENE-727
>                 URL: http:
//issues.apache.org/jira/browse/LUCENE-727
>             Project: Lucene - Java
>          Issue Type: Bug
>          Components: Index
>    Affects Versions: 2.0.0
>         Environment: Windows XP, JDK 1.4
>            Reporter: benson margulies
>         Attachments: lucenewindowsmmaptest.tgz
>
>
> When I set the system property to request the use of
the mmap directory, and start building a large index, the
process dies with an IOException trying to delete a file.
Apparently, Lucene isn't closing down the memory map before
deleting the file.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the
administrators: http://issues.apache.org/jira/secure/Administrators.jspa

-
For more information on JIRA, see: http://www.atl
assian.com/software/jira

        

------------------------------------------------------------
---------
To unsubscribe, e-mail: java-dev-unsubscribelucene.apache.org
For additional commands, e-mail: java-dev-helplucene.apache.org

     [ http://issues.apache.org/jira/browse/LUCENE-727?page=all
 ]

Michael McCandless reassigned LUCENE-727:
-----------------------------------------

    Assignee: Michael McCandless

> MMapDirectory can't create new index on Windows
> -----------------------------------------------
>
>                 Key: LUCENE-727
>                 URL: http:
//issues.apache.org/jira/browse/LUCENE-727
>             Project: Lucene - Java
>          Issue Type: Bug
>          Components: Index
>    Affects Versions: 2.0.0
>         Environment: Windows XP, JDK 1.4
>            Reporter: benson margulies
>         Assigned To: Michael McCandless
>         Attachments: lucenewindowsmmaptest.tgz
>
>
> When I set the system property to request the use of
the mmap directory, and start building a large index, the
process dies with an IOException trying to delete a file.
Apparently, Lucene isn't closing down the memory map before
deleting the file.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the
administrators: http://issues.apache.org/jira/secure/Administrators.jspa

-
For more information on JIRA, see: http://www.atl
assian.com/software/jira

        

------------------------------------------------------------
---------
To unsubscribe, e-mail: java-dev-unsubscribelucene.apache.org
For additional commands, e-mail: java-dev-helplucene.apache.org

    [ http://issues.apache.org/jira/brows
e/LUCENE-727?page=comments#action_12454081 ] 
            
Michael McCandless commented on LUCENE-727:
-------------------------------------------


Thank you Benson for the unit test and opening this issue!

Indeed, when I run your test, I see the exception in
Lucene-2.0.0, but, not on the trunk.

It looks like lockless commits (LUCENE-701 ) has fixed this
because the files in the index are now "write
once".  Specifically, we no longer try to rename any
files (nor do we even use the "deletable" file
anymore).  The general philosopy is to rely on as little
from the filesystem as we possibly can, so as to maximize
portability...

This is the exception on Lucene 2.0.0:

    [junit] Testcase:
testMmapIndex(org.apache.lucene.index.TestWindowsMMap):	Caus
ed an ERROR
    [junit] Cannot delete
F:mikesrclucene-2.0.0testLuceneMmapdeletable
    [junit] java.io.IOException: Cannot delete
F:mikesrclucene-2.0.0testLuceneMmapdeletable
    [junit] 	at
org.apache.lucene.store.FSDirectory.renameFile(FSDirectory.j
ava:268)
    [junit] 	at
org.apache.lucene.index.IndexWriter.writeDeleteableFiles(Ind
exWriter.java:827)
    [junit] 	at
org.apache.lucene.index.IndexWriter.deleteFiles(IndexWriter.
java:778)
    [junit] 	at
org.apache.lucene.index.IndexWriter.mergeSegments(IndexWrite
r.java:747)
    [junit] 	at
org.apache.lucene.index.IndexWriter.mergeSegments(IndexWrite
r.java:686)
    [junit] 	at
org.apache.lucene.index.IndexWriter.maybeMergeSegments(Index
Writer.java:674)
    [junit] 	at
org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.
java:479)
    [junit] 	at
org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.
java:462)
    [junit] 	at
org.apache.lucene.index.TestWindowsMMap.testMmapIndex(TestWi
ndowsMMap.java:61)

Since this is fixed on trunk, I'll resolve it as fixed.

But first I'll commit the new unit test (thanks Benson)!

> MMapDirectory can't create new index on Windows
> -----------------------------------------------
>
>                 Key: LUCENE-727
>                 URL: http:
//issues.apache.org/jira/browse/LUCENE-727
>             Project: Lucene - Java
>          Issue Type: Bug
>          Components: Index
>    Affects Versions: 2.0.0
>         Environment: Windows XP, JDK 1.4
>            Reporter: benson margulies
>         Assigned To: Michael McCandless
>         Attachments: lucenewindowsmmaptest.tgz
>
>
> When I set the system property to request the use of
the mmap directory, and start building a large index, the
process dies with an IOException trying to delete a file.
Apparently, Lucene isn't closing down the memory map before
deleting the file.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the
administrators: http://issues.apache.org/jira/secure/Administrators.jspa

-
For more information on JIRA, see: http://www.atl
assian.com/software/jira

        

------------------------------------------------------------
---------
To unsubscribe, e-mail: java-dev-unsubscribelucene.apache.org
For additional commands, e-mail: java-dev-helplucene.apache.org

     [ http://issues.apache.org/jira/browse/LUCENE-727?page=all
 ]

Michael McCandless resolved LUCENE-727.
---------------------------------------

    Resolution: Fixed

> MMapDirectory can't create new index on Windows
> -----------------------------------------------
>
>                 Key: LUCENE-727
>                 URL: http:
//issues.apache.org/jira/browse/LUCENE-727
>             Project: Lucene - Java
>          Issue Type: Bug
>          Components: Index
>    Affects Versions: 2.0.0
>         Environment: Windows XP, JDK 1.4
>            Reporter: benson margulies
>         Assigned To: Michael McCandless
>         Attachments: lucenewindowsmmaptest.tgz
>
>
> When I set the system property to request the use of
the mmap directory, and start building a large index, the
process dies with an IOException trying to delete a file.
Apparently, Lucene isn't closing down the memory map before
deleting the file.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the
administrators: http://issues.apache.org/jira/secure/Administrators.jspa

-
For more information on JIRA, see: http://www.atl
assian.com/software/jira

        

------------------------------------------------------------
---------
To unsubscribe, e-mail: java-dev-unsubscribelucene.apache.org
For additional commands, e-mail: java-dev-helplucene.apache.org