Re: Using UNIX:UNIX-MMAP to read large text files under CMUCL?

	[CC'ed to cmucl-imp -- see issue on CVS tree]

* "Denis Papathanasiou"
<e4044eb40704050934m6f8536ceu29e406a1c33e8216XXX>
:

| I'm familiar with how to use mmap() in C for this (reading
large text
| files), but I'm not sure how to go about it under CMUCL.
|
| I found this example in comp.lang.lisp, but since it just
returns the
| pointer (the output of the mmap call as an integer), how
do I convert
| that pointer into some kind of lisp object/structure that
I can use to
| read the results, either with (read-sequence) or
(read-line)?
<snip>

I don';t think you can call lisp functions directly on the
storage
returned by unix:mmap. Maybe you can use OPEN with :MAPPED
T, which
should give you a simple-stream to a mmaped backed file on
which you
can use lisp functions.

| Also, when using CMUCL's mmap call, am I forced to open
the file with
| (unix:unix-open) or can I use (with-open-file) as well?
|
| When I tried the latter, the call to (unix:unix-mmap)
returned a null value:


| CL-USER> (with-open-file (file-stream
"/path/to/very_large_file.txt"
| CL-USER> :direction :input)
           (let ((mmap-ptr (unix:unix-mmap nil (file-length
| file-stream) unix:PROT_READ unix:MAP_PRIVATE
(system:fd-stream-fd
| file-stream) 0))) mmap-ptr))

There was a problem reported and apparently fixed in

http://permalink.gmane.org/gmane.lisp.cmucl.devel/9910


My local CVS tree shows this entry for unix-glibc2.lisp
----------------------------
revision 1.37
date: 2006/09/06 01:07:09;  author: rtoy;  state: Exp; 
lines: +14 -4
o Add MAP_FAILED
o UNIX-MMAP cannot use the syscall macro because addresses
can be
  "negative".   (Steal implementation from
unix.lisp.)
----------------------------

However the CVS-Cmucl tree at common-lisp.net does not seem
to have
this fix. Perhaps it was lost during the restore after it
was
corrupted?

I'm not sure if this problem is related as I believed
UNIX-MMAP should
fail unconditionally without the fix.
--
Madhu