Is this a bug in 'read-vector?

I have been trying to read large binary files of floating
point data
using CMUCL (19c). I thought I would have to do it using
some form of
FFI and went to comp.lang.lisp for help getting that
working. I
succeeded. But Duane Rettig at Allegro suggested it would be
easier to
use 'read-vector. So I tried that as follows:

(let ((vec (make-array 10 :element-type 'double-float)))
  (with-open-file (os "d10.bin")
    (read-vector vec os)
    (print vec)))

where "d10.bin" is a double-float binary file
containing 10
elements. When I try to read the file it produces the
following error:

Type-error in KERNEL::OBJECT-NOT-DOUBLE-FLOAT-ERROR-HANDLER:
   #\Null is not of type DOUBLE-FLOAT

Here is the C-code code I used to produce the file:

#include <iostream>
#include <fstream>
#include <complex>
using namespace std;

int main()
{
  int n=10;
  double *d = new double[n];
  for (int i=0; i<n; i++) 
    d[i] = i;
  FILE *of = fopen("d10.bin", "wb");
  fwrite(f,8,n,of);
  fclose(of);
  return 0;
}


Here is Duane's sample code that he say works in Allegro:

copied from 
<http://groups.google.com/group/comp.lang.lisp
/browse_thread/thread/67876101085aee82/05a20cfcd11f8fbe?hl=e
n#05a20cfcd11f8fbe>
............................................................
.......
You must be using a simple-streams implementation from
another lisp.
Allegro CL doesn't have a KERNEL package.

What you're seeing above is a bug; you should report it to
that
lisp's support team.

It works fine in Allegro CL (for which you can download the
Express
Edition for free):

CL-USER(1): (defvar *x*
               (make-array 10 :element-type 'double-float
                              :initial-contents
                              (loop for i from 0.0d0 to
9.0d0
                                    collect i)))
*X*
CL-USER(2): *x*
#(0.0d0 1.0d0 2.0d0 3.0d0 4.0d0 5.0d0 6.0d0 7.0d0 8.0d0
9.0d0)
CL-USER(3): (with-open-file (s "z.dat"
:direction :io
                                       :if-exists :overwrite
                                       :if-does-not-exist
:create)
(write-vector *x* s))
80
CL-USER(4): (defvar *y* (make-array 10 :element-type
'double-float
                                       :initial-element
10.0d0))
*Y*
CL-USER(5): (with-open-file (s "z.dat")
(read-vector *y* s))
80
CL-USER(6): *y*
#(0.0d0 1.0d0 2.0d0 3.0d0 4.0d0 5.0d0 6.0d0 7.0d0 8.0d0
9.0d0)
CL-USER(7):

-- 
Duane Rettig
............................................................
.......



Is this really a bug? Or am I doing something else wrong?

Regards,

--Jeff

>>>>> "Jeffrey" == Jeffrey
Cunningham <jeffreycunningham.net> writes:

    Jeffrey> I have been trying to read large binary
files of floating point data
    Jeffrey> using CMUCL (19c). I thought I would have to
do it using some form of
    Jeffrey> FFI and went to comp.lang.lisp for help
getting that working. I
    Jeffrey> succeeded. But Duane Rettig at Allegro
suggested it would be easier to
    Jeffrey> use 'read-vector. So I tried that as
follows:

    Jeffrey> (let ((vec (make-array 10 :element-type
'double-float)))
    Jeffrey>   (with-open-file (os "d10.bin")
    Jeffrey>     (read-vector vec os)
    Jeffrey>     (print vec)))

    Jeffrey> where "d10.bin" is a
double-float binary file containing 10
    Jeffrey> elements. When I try to read the file it
produces the following error:

    Jeffrey> Type-error in
KERNEL::OBJECT-NOT-DOUBLE-FLOAT-ERROR-HANDLER:
    Jeffrey>    #\Null is not of type DOUBLE-FLOAT

[snip]

    Jeffrey> Is this really a bug? Or am I doing
something else wrong?

Yes, this does appear to be a bug in the implementation of
read-vector.

You can, however, achieve what you want by opening the file
with an
element-type of, say, (unsigned-byte 8), instead of the
default
'character.

I'll have to read some more to understand how read-vector
interacts
with the stream element type.  It seems, though, that the
element-type
of the vector overrides the element-type of the stream, more
or less.
Currently, a stream element type of character basically
causes the
code to read in characters.

Ray

>>>>> "Jeffrey" == Jeffrey
Cunningham <jeffreycunningham.net> writes:

    Jeffrey> I have been trying to read large binary
files of floating point data
    Jeffrey> using CMUCL (19c). I thought I would have to
do it using some form of
    Jeffrey> FFI and went to comp.lang.lisp for help
getting that working. I
    Jeffrey> succeeded. But Duane Rettig at Allegro
suggested it would be easier to
    Jeffrey> use 'read-vector. So I tried that as
follows:

    Jeffrey> (let ((vec (make-array 10 :element-type
'double-float)))
    Jeffrey>   (with-open-file (os "d10.bin")
    Jeffrey>     (read-vector vec os)
    Jeffrey>     (print vec)))

[snip]

    Jeffrey> Is this really a bug? Or am I doing
something else wrong?

Could you file a bug report at http://trac.common
-lisp.net/cmucl?

Thanks,

Ray

>>>>> "Jeffrey" == Jeffrey
Cunningham <jeffreycunningham.net> writes:

    Jeffrey> I have been trying to read large binary
files of floating point data
    Jeffrey> using CMUCL (19c). I thought I would have to
do it using some form of
    Jeffrey> FFI and went to comp.lang.lisp for help
getting that working. I
    Jeffrey> succeeded. But Duane Rettig at Allegro
suggested it would be easier to
    Jeffrey> use 'read-vector. So I tried that as
follows:

    Jeffrey> (let ((vec (make-array 10 :element-type
'double-float)))
    Jeffrey>   (with-open-file (os "d10.bin")
    Jeffrey>     (read-vector vec os)
    Jeffrey>     (print vec)))

    Jeffrey> where "d10.bin" is a
double-float binary file containing 10
    Jeffrey> elements. When I try to read the file it
produces the following error:

    Jeffrey> Type-error in
KERNEL::OBJECT-NOT-DOUBLE-FLOAT-ERROR-HANDLER:
    Jeffrey>    #\Null is not of type DOUBLE-FLOAT

First, thanks for entering a ticket for cmucl on this issue!

Second, I've looked a bit more.  The ticket has my
response, but in a
nutshell, CMUCL has two versions of read-vector.  The
version you used
is EXT:READ-VECTOR and has the bug.  But if you (require
'simple-streams), and use this (note the :class arg):

    (let ((vec (make-array 10 :element-type
'double-float)))
      (with-open-file (os "d10.bin" :class
'stream:file-simple-stream)
         (stream:read-vector vec os)
         (print vec)))

everything works as you expect.

I think we should make EXT:READ-VECTOR work the same as
STREAM:READ-VECTOR, especially since EXT:READ-VECTOR is
significantly
faster (order of magnitude or more) than STREAM:READ-VECTOR.

Ray

>>>>> "Raymond" == Raymond Toy
<raymond.toyericsson.com> writes:

    Raymond> I think we should make EXT:READ-VECTOR work
the same as
    Raymond> STREAM:READ-VECTOR, especially since
EXT:READ-VECTOR is significantly
    Raymond> faster (order of magnitude or more) than
STREAM:READ-VECTOR.

I've modified this so that ext:read-vector works in this
case.

However, this change also means that read-sequence no longer
signals
errors if you, say, read characters into an (unsigned-byte
8)
simple-array.

Should we still signal such errors?  (We still signal such
errors if
the array is not a specialized array.)

Ray