"ocaml_beginners"::[] reading big file -> how to make it faster?

On 6/5/06, Jonathan Roewen <jonathan.roewengmail.com> wrote:
> > Not really, if the running time of the whole
program
> > will increase... :(
> >
> > The reading was faster now, but the whole work
needs more time now... :(
> >
> > Strange :(
>
> Using mutable fields in the record makes it slower
overall?? That is
> certainly odd. Maybe the array.init version with
non-mutable records
> is better then.
>
> BTW: When using the strings, if you use accessor
functions taking an
> x,y argument, your function can pull out the data.
>
> E.g.:
>
> type picture_t = string * int * int (* width &
height *)
>
> val get : picture_t -> int -> int -> colour
>
> let get (buf,w,h) x y =
>  let base = (y * w + x) * 3 in {
>    red = int_of_char buf.[base];
>    green = int_of_char buf.[base + 1];
>    blue = int_of_char buf.[base + 2];
>  }
>
> Which of course incurs runtime overhead over (possibly)
much faster load times.
>
> For the matrix init version, it'd look something like:
>
> let load_picture chan =
>  (* code from before here *)
>  let linewidth = colornum * width in
>  let buffer = String.make linewidth ' ' in
>  let buf_pos = ref 0 in
>  let refill () =
>    buf_pos := 0;
>    really_input chan buffer 0 linewidth
>  in
>  let get_pixel x y =
>    if !buf_pos = linewidth then refill ();
>    buf_pos := !buf_pos + 3;
>    {
>      red = int_of_char buffer.[!buf_pos - 3];
>      blue = int_of_char buffer.[!buf_pos - 2];
>      green = int_of_char buffer.[!buf_pos - 1];
>    }
>  in
>  let picture = init_matrix width height get_pixel in
>  { xdim = width; ydim = height; colors = color; data =
picture }
>
> This is without any testing whatsoever .. but I think
you can see what
> it's doing, and how to correct any bugs if there are
any (there has to
> be at least one! :P). Yes, it's a bit of extra code,
but at least you
> build the matrix just once with no further
modifications, and get to
> keep your colour type mutable-free -- if that is indeed
the cause
> behind the apparently slower performance.
>
> Jonathan

Yes, there are most undoubtedly a few bugs  It's
dependent on
init_matrix filling the items in a particular order (row by
row), and
completely ignores the x,y arguments =P

A possible change is to read the entire file into memory in
one go
(OCaml's string size limits permitting), and then indexing
directly
into the string as in previous code. You
have the downside of using more memory (until GC reclaims
the string
buffer), but otherwise should be fine.

Jonathan


------------------------ Yahoo! Groups Sponsor
--------------------~--> 
Home is just a click away.  Make Yahoo! your home page now.
http://us.click.yahoo.com/DHchtC/3FxNAA/yQLSAA/saFolB/TM

------------------------------------------------------------
--------~-> 

Archives up to August 22, 2005 are also downloadable at http://www.connettivo.net/cntprojects/ocaml_beginners/
The archives of the very official ocaml list (the seniors'
one) can be found at http://caml.inria.fr
Attachments are banned and you're asked to be polite, avoid
flames etc. 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http:/
/groups.yahoo.com/group/ocaml_beginners/

<*> To unsubscribe from this group, send an email to:
    ocaml_beginners-unsubscribe@yahoogroups.com

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.c
om/info/terms/