"ocaml_beginners"::[] Making a general library from OCaml-code

Hello,

I want to write a library that can be accessed
like any other system library on a unix/linux system,
but I don't want to code it in C. I want to write it
in Ocaml, but want to use it as a system-library
that can be accessed by C-programs.

How to create this?

Do I have to use C-interfacing tricks to get such a thing
done?

TIA,
Oliver

Messages in this topic (1) Reply (via web post) | Start a new topic

.

On Tue, Jun 03, 2008 at 12:45:21PM +0200, Oliver Bandel wrote:
> I want to write a library that can be accessed
> like any other system library on a unix/linux system,
> but I don't want to code it in C. I want to write it
> in Ocaml, but want to use it as a system-library
> that can be accessed by C-programs.
>
> How to create this?

It's a bit tricky. I've never really pinned it down, but here are
some general comments:

- Look at the source to mod_caml. It builds a .so containing the
OCaml runtime. To do this I needed to patch the runtime (patch is
going into 3.11 apparently).
http://caml.inria.fr/mantis/view.php?id=3866

- To generate a shared library, you will need to generate PIC code.
In recent versions it's the default, but you may need to enable it
(ocamlopt -fPIC).

- You need to initialize the OCaml runtime.

- Beware of multiple C threads calling into the OCaml code.

BTW if you do work it out, *please* post a mini-project or the steps
you used on a web page somewhere.

Rich.

--
Richard Jones
Red Hat

Messages in this topic (2) Reply (via web post) | Start a new topic

.

Helo Richard,

Zitat von Richard Jones < rich%40annexia.org">richannexia.org>:

> On Tue, Jun 03, 2008 at 12:45:21PM +0200, Oliver Bandel wrote:
&gt; > I want to write a library that can be accessed
> > like any other system library on a unix/linux system,
> > but I don't want to code it in C. I want to write it
> > in Ocaml, but want to use it as a system-library
> > that can be accessed by C-programs.
> >
>; > How to create this?
&gt;
> It's a bit tricky. I've never really pinned it down, but here are
> some general comments:
>
&gt; - Look at the source to mod_caml. It builds a .so containing the
> OCaml runtime. To do this I needed to patch the runtime (patch is
> going into 3.11 apparently).
> http://caml.inria.fr/mantis/view.php?id=3866

Well, should I better wait until release of 3.11 for such kinds of
projects?!

>
> - To generate a shared library, you will need to generate PIC code.
&gt; In recent versions it's the default, but you may need to enable it
> (ocamlopt -fPIC).
>
> - You need to initialize the OCaml runtime.
[...]

So I have C-code, and from there calling my OCaml-code,
right? So there is no direct way for it, it seems.

>
&gt; - Beware of multiple C threads calling into the OCaml code.
&gt;
> BTW if you do work it out, *please* post a mini-project or the steps
&gt; you used on a web page somewhere.
[...]

Well, the stuff is not really huge, so I also can write it entirely
in C. That seems to me to be faster in the over-all process.

But maybe I will try it with OCaml later on.
Because it's a small lib, this also might be good for
exploring such kind of OCaml-based library....well,
we will see how I will decide in the future

If I possibly make it, I will make some web-description available
then.

But first I want to have the stuff available
for myself (I think in some hours I have it ready in C).

Ciao,
Oliver

Messages in this topic (3) Reply (via web post) | Start a new topic

.

Oliver Bandel a écrit :
>
> Hello,
&gt;
> I want to write a library that can be accessed
> like any other system library on a unix/linux system,
> but I don't want to code it in C. I want to write it
> in Ocaml, but want to use it as a system-library
> that can be accessed by C-programs.
>
> How to create this?
&gt;
> Do I have to use C-interfacing tricks to get such a thing
&gt; done?
&gt;
> TIA,
>; Oliver

The idea is to generate one C-object file from your Caml code.
The -output-obj command line option allows this.

You will alse need to register the entry points in your Caml code
(Callback module) in
order to be able to call it from C side.

Yes, a C part is necessary, in order to:
- initialize the Caml runtime system
- expose C API and call the relevant Caml code, eventually converting
data between both worlds

ODLL, written by Nicolas Canasse is not widely used, but automates this
for windows world. It may
be a good idea to have it evolve (or simply update it to current version
of OCaml if necessary). Maybe
after the natdynlink branch of ocaml tools will be published (I think
this will simplify all the process).

Hoping that this helps

Salutations

Matt

Messages in this topic (4) Reply (via web post) | Start a new topic

.

Oliver Bandel a écrit :
>
> Helo Richard,
>
>; Zitat von Richard Jones < rich%40annexia.org">richannexia.org>:
>
> > On Tue, Jun 03, 2008 at 12:45:21PM +0200, Oliver Bandel wrote:
&gt; > > I want to write a library that can be accessed
> > > like any other system library on a unix/linux system,
> > > but I don't want to code it in C. I want to write it
> > > in Ocaml, but want to use it as a system-library
> > > that can be accessed by C-programs.
> > >
>; > > How to create this?
&gt; >
>; > It's a bit tricky. I've never really pinned it down, but here are
> > some general comments:
> >
>; > - Look at the source to mod_caml. It builds a .so containing the
> > OCaml runtime. To do this I needed to patch the runtime (patch is
> > going into 3.11 apparently).
> > http://caml.inria.fr/mantis/view.php?id=3866
>;
> Well, should I better wait until release of 3.11 for such kinds of
> projects?!
>
&gt; >
>; > - To generate a shared library, you will need to generate PIC code.
&gt; > In recent versions it's the default, but you may need to enable it
> > (ocamlopt -fPIC).
> >
>; > - You need to initialize the OCaml runtime.
> [...]
&gt;
> So I have C-code, and from there calling my OCaml-code,
> right? So there is no direct way for it, it seems.
&gt;
> >
>; > - Beware of multiple C threads calling into the OCaml code.
&gt; >
>; > BTW if you do work it out, *please* post a mini-project or the steps
&gt; > you used on a web page somewhere.
> [...]

Mmmm. It's a long time since I should have done this...

>
&gt; Well, the stuff is not really huge, so I also can write it entirely
> in C. That seems to me to be faster in the over-all process.
>
>; But maybe I will try it with OCaml later on.
> Because it's a small lib, this also might be good for
> exploring such kind of OCaml-based library....well,
&gt; we will see how I will decide in the future
>
> If I possibly make it, I will make some web-description available
> then.
&gt;
> But first I want to have the stuff available
> for myself (I think in some hours I have it ready in C).

If your library is small and could be a good example, I can help
turning it into a .so.

I have some OMake code that helps.

Salutations

Matt

Messages in this topic (5) Reply (via web post) | Start a new topic

.

On Tuesday 03 June 2008 12:51:05 Richard Jones wrote:
&gt; BTW if you do work it out, *please* post a mini-project or the steps
&gt; you used on a web page somewhere.

If I might reiterate this: lots of people want to be able to build DLLs from
OCaml but nobody has ever written a guide explaining how it can be done even
though a couple of people have claimed to be able to do it.

I would also love to see such a guide.

--
Dr Jon D Harrop, Flying Frog Consultancy Ltd.
http://www.ffconsultancy.com/products/?e

Messages in this topic (6) Reply (via web post) | Start a new topic

.

On Tuesday 03 June 2008 13:06:55 Oliver Bandel wrote:
&gt; Well, should I better wait until release of 3.11 for such kinds of
> projects?!

Are you sure this will even be in 3.11?

> Well, the stuff is not really huge, so I also can write it entirely
> in C. That seems to me to be faster in the over-all process.
>
>; But maybe I will try it with OCaml later on.
> Because it's a small lib, this also might be good for
> exploring such kind of OCaml-based library....well,
&gt; we will see how I will decide in the future
>
> If I possibly make it, I will make some web-description available
> then.
&gt;
> But first I want to have the stuff available for myself (I think in some
>; hours I have it ready in C).

This kind of thing is vastly easier and more productive with a common language
run-time (assuming your users are also using a high-level language). This is
a major reason to build upon the JVM or Mono and is of particular importance
to commercial developers who want to sell DLLs.

--
Dr Jon D Harrop, Flying Frog Consultancy Ltd.
http://www.ffconsultancy.com/products/?e

Messages in this topic (7) Reply (via web post) | Start a new topic

.

Jon Harrop a écrit :
>
> On Tuesday 03 June 2008 12:51:05 Richard Jones wrote:
&gt; > BTW if you do work it out, *please* post a mini-project or the steps
&gt; > you used on a web page somewhere.

> If I might reiterate this: lots of people want to be able to build
DLLs from
>; OCaml but nobody has ever written a guide explaining how it can be
done even
>; though a couple of people have claimed to be able to do it.

Hum. All I'm doing is following the explanations of chapter 18 of the
manual!

Anyway, here is a stupid example.

******* Here is my Caml code

mattgros:~/projets/so$ cat module1.ml
let hello y = "Hello " ^ y ^ "! I hope everything is ok?"
mattgros:~/projets/so$ cat module2.ml
let s y = Module1.hello y
mattgros:~/projets/so$ cat caml_stub.ml
let _ = Callback.register "my beautiful function&quot; (fun name -> Module2.s
name ^ " (returned by caml_stub)")

******* Let's compile the caml code into my_caml_code.o

mattgros:~/projets/so$ ocamlopt -c module1.ml
mattgros:~/projets/so$ ocamlopt -c module2.ml
mattgros:~/projets/so$ ocamlopt -c caml_stub.ml
mattgros:~/projets/so$ ocamlopt -output-obj -o my_caml_code.o
module1.cmx module2.cmx caml_stub.cmx

******* Now the DLL itself (the caml runtime has to be started
manually. It is possible to have the caml_startup function be
called automatically when the DLL is loaded.)

mattgros:~/projets/so$ cat my_dll.c
#include <stdio.h>
#include <assert.h>
#include <string.h> /* strdup */
#include <caml/alloc.h>
#include <caml/memory.h>;
#include <caml/callback.h&gt;
#include <caml/mlvalues.h&gt;

static int my_dll_init_done = 0;

void my_dll_init (void)
{
char *vide[2];

vide[0] = "bof&quot;;
vide[1] = NULL;

if (!my_dll_init_done)
{
caml_startup (vide);
my_dll_init_done = 1;
}
}

int my_dll_is_init_done (void)
{
return my_dll_init_done;
}

void my_dll_try_it(char * name){
value res;
char * ch;

static value *cbk = NULL;
if (cbk == NULL)
cbk = caml_named_value(&quot;my beautiful function&quot;);

assert(cbk);

res = caml_callback(*cbk, caml_copy_string(name));

ch = strdup (String_val(res));

printf("%sn&quot;, ch);

free(ch);
}

******* ocamlc knows how to compile C code, and is kind enough to show
us if asked to be verbose
(Maybe I should have used ocamlopt, here, to be coherent)

mattgros:~/projets/so$ ocamlc -verbose -c my_dll.c
+ gcc -fno-defer-pop -Wall -D_FILE_OFFSET_BITS=64 -D_REENTRANT -fPIC
-c -I'/home/matt/godi/lib/ocaml/std-lib' 'my_dll.c'

******* And now link the shared library. Adding the camlruntime: here
the native one is used (asmrun), since we used ocamlopt. The -lm and
-ldl additionnal libraries to link are listed by ocamlc -config

mattgros:~/projets/so$ ocamlc -config
version: 3.10.1
standard_library_default: /home/matt/godi/lib/ocaml/std-lib
standard_library: /home/matt/godi/lib/ocaml/std-lib
standard_runtime: /home/matt/godi/bin/ocamlrun
ccomp_type: cc
bytecomp_c_compiler: gcc -fno-defer-pop -Wall -D_FILE_OFFSET_BITS=64
-D_REENTRANT -fPIC
bytecomp_c_linker: gcc -Wl,-E -L/home/matt/godi/lib
-Wl,-R/home/matt/godi/lib
bytecomp_c_libraries: -lm -ldl -lpthread
native_c_compiler: gcc -Wall -D_FILE_OFFSET_BITS=64 -D_REENTRANT
native_c_linker: gcc -L/home/matt/godi/lib -Wl,-R/home/matt/godi/lib
native_c_libraries: -lm -ldl
native_partial_linker: ld -r
ranlib: ranlib
cc_profile: -pg
architecture: i386
model: default
system: linux_elf
ext_obj: .o
ext_asm: .s
ext_lib: .a
ext_dll: .so
os_type: Unix
default_executable_name: a.out
systhread_supported: true

mattgros:~/projets/so$ gcc -shared -o mydll.so my_dll.o my_caml_code.o
-L`ocamlc -where` -lasmrun -lm -ldl

******* One C test file

mattgros:~/projets/so$ cat test.c
#include <stdlib.h>
#include <stdio.h>
#include <assert.h>
#include "my_dll.h"

void is_init_done(){
printf(&quot;DLL init check: %sn", (my_dll_is_init_done()) ? "Init done" :
"Init not done");
}

void do_init(){
printf(&quot;Doing DLL init...&quot;);
my_dll_init();
printf(&quot; donen";);
fflush(stdout);
}

int main(void){
is_init_done();
do_init();
is_init_done();

my_dll_try_it("Bob");

return EXIT_SUCCESS;
}
mattgros:~/projets/so$ gcc -o testdll mydll.so test.c -Wl,-R.
mattgros:~/projets/so$ ./testdll
DLL init check: Init not done
Doing DLL init... done
DLL init check: Init done
Hello Bob! I hope everything is ok? (returned by caml_stub)
mattgros:~/projets/so$ ldd testdll
linux-gate.so.1 => (0xb7fd8000)
mydll.so => ./mydll.so (0xb7fae000)
libc.so.6 => /lib/tls/i686/cmov/libc.so.6 (0xb7e4f000)
libm.so.6 => /lib/tls/i686/cmov/libm.so.6 (0xb7e2a000)
libdl.so.2 => /lib/tls/i686/cmov/libdl.so.2 (0xb7e26000)
/lib/ld-linux.so.2 (0xb7fd9000)

My last post to the Caml-list was a help request to automate those
compilation steps with ocamlbuild.
(I already have it with OMake).

Hoping this will help?

Salutations

Matt

Messages in this topic (8) Reply (via web post) | Start a new topic

.

Zitat von Matthieu Dubuget < matthieu.dubuget%40gmail.com">matthieu.dubugetgmail.com>:

> Jon Harrop a écrit :
> >
>; > On Tuesday 03 June 2008 12:51:05 Richard Jones wrote:
&gt; > > BTW if you do work it out, *please* post a mini-project or the
> steps
&gt; > > you used on a web page somewhere.
>
&gt;
> > If I might reiterate this: lots of people want to be able to build
&gt; DLLs from
>; > OCaml but nobody has ever written a guide explaining how it can be
> done even
>; > though a couple of people have claimed to be able to do it.
>
> Hum. All I'm doing is following the explanations of chapter 18 of the
> manual!
[...]

Well, just a few minutes ago I also looked again into the refman.
Chapter 18 seems to be a good starting point.

I will examine your examples (and read again the chapter 18) tomorrow...

...

The section-title "18.7 Advanced topic: callbacks from C to Caml"
is a littlebid irritating.

Should be entitled: "Calling OCaml from C", and the
callback-stuff should be mentioned as a sub-section.

In the text just below the point 18.7
is: "In this section, we show how C functions can call Caml
functions(...)".

Using the title "Calling OCaml from C" for 18.7 would be much more
inviting, because it's what the docs are talking about. So the
section-title is a minsmomer, IMHO.

I didn't needed "callbacks" so far, but calling Ocaml from C
is what I'm looking here. The reason I didn't found it, was, that
the section was misnomed (IMHO).

So, again, I can recommend to make the docs more clearly,
so that it's easier to find, what already is written down there.

And If the refman would be available in a nicely printed
book-format, this would also be fine.

I hate reading documentation at the screen only.

But this is only my personal opinion, and I think,
most computer-people's would disagree here, because
it's more "freaky" to ignore paper completely.

>
> Anyway, here is a stupid example.
[...]

Stupid is OK, if it helps people who think too complicated
(and therefore don't get the things done simply )
to get things done.

[...]
> Hoping this will help?
[...]

Will try it tomorrow.

This evening I want to view Big Buck Bunny again...
...and again...
...and again...
...and again...

...really good movie, I think:

http://www.bigbuckbunny.org/

Good night, best wishes
and have fun,

Oliver

P.S.: Don't become too desperate, when you need some days
until you can download BigBuckBunny; I have waited the last
three days to get it... I think the servers are really working
a lot... you also can buy it as DVD and use it as a present.

BUT IMHO it's really funny and cool... and completely developed
at the computer. But the characters are looking quite lively/real.

Messages in this topic (9) Reply (via web post) | Start a new topic

.

> This evening I want to view Big Buck Bunny again...



I was lucky then, or maybe downloaded just after it was ready:
no delay.

Have a nice day

Messages in this topic (10) Reply (via web post) | Start a new topic

.