|
List Info
Thread: Eventmachine 0.8.0 released with epoll, breaks Ruby file-descriptor limit
|
|
| Eventmachine 0.8.0 released with epoll,
breaks Ruby file-descriptor limit |
|
2007-07-11 11:33:43 |
|
The Eventmachine team is pleased to announce the release of version 0.8.0, available now on RubyForge as a tarball and as a gem.
Eventmachine is a framework for writing fast, scalable network-aware programs (clients, servers, or both) in Ruby or C/C++, without requiring the use of threads. EM is used today in a variety of production systems including Web, mail, and LDAP/RADIUS servers, SNMP agents, and custom protocol handlers. It's also used in Kirk Haines';s Swiftiply, a fast clustering proxy for Rails and other web frameworks.
The 0.8.0 release adds automatic support for epoll, on Linux 2.6 kernels. This enables Eventmachine programs to break through Ruby's limit of 1024 file and socket descriptors per process. In performance tests designed to simulate busy web servers, Eventmachine with epoll exhibits the behavior you would expect: with very large numbers (>20,000) of quiet connections and a few active ones, there is no noticeable degradation of responsiveness or total throughput, compared to cases with only a few connections.
We expect the next major EM release to be called 0.9.0. It will add Erlang-like features intended to make it simple for EM-based programs to send messages to each other, or to other processes. A typical use case for this capability is a "service" web site that must make calls to other web sites, web services, or other network services in order to fulfill requests from its own clients.
Thanks to all for your support of the EM project, and especially to Tony Arcieri, who tirelessly bugged me for weeks to add the epoll support, and to Kirk for helping to shake it out.
|
| Re: Eventmachine 0.8.0 released with
epoll, breaks Ruby file-descriptor li |
|
2007-07-11 11:57:32 |
Hi Francis,
Any work being done on non blocking database drivers for EM?
I know
that's been a major reason why I haven't used EM in more
applications.
The message passing stuff sounds cool though, can't wait
to see it.
Chris
_______________________________________________
Eventmachine-talk mailing list
Eventmachine-talk rubyforge.org
http://rubyforge.org/mailman/listinfo/eventmachine-talk
|
|
| Re: Eventmachine 0.8.0 released with
epoll, breaks Ruby file-descriptor li |
|
2007-07-11 12:11:01 |
Hi Francis,
Now that you have epoll support working (congrats BTW!), is
kqueue
(FreeBSD/Mac OS X) or /dev/poll (Solaris) support far off
the horizon?
Best regards,
--Michael
On 7/11/07, snacktime <snacktimegmail.com> wrote:
> Hi Francis,
>
> Any work being done on non blocking database drivers
for EM? I know
> that's been a major reason why I haven't used EM in
more applications.
> The message passing stuff sounds cool though, can't
wait to see it.
>
> Chris
> _______________________________________________
> Eventmachine-talk mailing list
> Eventmachine-talkrubyforge.org
> http://rubyforge.org/mailman/listinfo/eventmachine-talk
>
>
_______________________________________________
Eventmachine-talk mailing list
Eventmachine-talkrubyforge.org
http://rubyforge.org/mailman/listinfo/eventmachine-talk
|
|
| Re: Eventmachine 0.8.0 released with
epoll, breaks Ruby file-descriptor li |
|
2007-07-11 12:19:17 |
|
On 7/11/07, Michael S. Fischer < michaeldynamine.net">michaeldynamine.net> wrote:
Hi Francis,
Now that you have epoll support working (congrats BTW!), is kqueue
(FreeBSD/Mac OS X) or /dev/poll (Solaris) support far off the horizon?
A higher priority would probably be Windows/IOCP, to tell the truth. I'm completely amenable to adjusting the priorities, though, especially if someone would like to step up and help develop and test, hint hint!. kqueue and /dev/poll will be pretty easy for someone to fit into the epoll pattern. IOCP is a complete rewrite.
|
| Re: Eventmachine 0.8.0 released with
epoll, breaks Ruby file-descriptor li |
|
2007-07-11 12:23:14 |
|
On 7/11/07, snacktime < snacktimegmail.com">snacktimegmail.com> wrote:
Hi Francis,
Any work being done on non blocking database drivers for EM? I know
that's been a major reason why I haven't used EM in more applications.
The message passing stuff sounds cool though, can't wait to see it.
Chris
Chris, the "official" response to this is to run blocking DBMS calls with EventMachine#defer. That puts them on a thread pool managed by the EM reactor.
I'd be thrilled if someone were to implement evented DB drivers, though. You could do it in Ruby with the existing protocol handlers in lib/protocols.
|
| Re: Eventmachine 0.8.0 released with
epoll, breaks Ruby file-descriptor li |
|
2007-07-11 13:05:16 |
>
> Chris, the "official" response to this is to
run blocking DBMS calls with
> EventMachine#defer. That puts them on a thread pool
managed by the EM
> reactor.
>
> I'd be thrilled if someone were to implement evented DB
drivers, though. You
> could do it in Ruby with the existing protocol handlers
in lib/protocols.
>
I should have clarified. I meant the ones written as C
extensions.
Then again depending on the application you might not really
be losing
much with a slower pure ruby driver. More cpu time but no
io
blocking, so it's probably a win.
Chris
> _______________________________________________
> Eventmachine-talk mailing list
> Eventmachine-talkrubyforge.org
> http://rubyforge.org/mailman/listinfo/eventmachine-talk
>
_______________________________________________
Eventmachine-talk mailing list
Eventmachine-talkrubyforge.org
http://rubyforge.org/mailman/listinfo/eventmachine-talk
|
|
| Re: Eventmachine 0.8.0 released with
epoll, breaks Ruby file-descriptor li |
|
2007-07-11 13:31:29 |
|
On 7/11/07, snacktime < snacktimegmail.com">snacktimegmail.com> wrote:
>
>; Chris, the "official" response to this is to run blocking DBMS calls with
> EventMachine#defer. That puts them on a thread pool managed by the EM
> reactor.
>
> I'd be thrilled if someone were to implement evented DB drivers, though. You
> could do it in Ruby with the existing protocol handlers in lib/protocols.
>
I should have clarified. ;I meant the ones written as C extensions.
Then again depending on the application you might not really be losing
much with a slower pure ruby driver. More cpu time but no io
blocking, so it's probably a win.
But you're still talking about changing them so they use evented I/O, right?
|
| Re: Eventmachine 0.8.0 released with
epoll, breaks Ruby file-descriptor li |
|
2007-07-11 19:03:25 |
>
> But you're still talking about changing them so they
use evented I/O,
> right?
>
Yes. Not sure what you could gain by trying to do it in
pure ruby
though, as you would still just have to use a thread right?
I think
the best option would be to just add it to existing C
drivers where
the client library supports async io. Postgres does, I was
just
looking at their specs.
Chris
_______________________________________________
Eventmachine-talk mailing list
Eventmachine-talkrubyforge.org
http://rubyforge.org/mailman/listinfo/eventmachine-talk
|
|
| Re: Eventmachine 0.8.0 released with
epoll, breaks Ruby file-descriptor li |
|
2007-07-11 20:04:57 |
|
On 7/11/07, snacktime < snacktimegmail.com">snacktimegmail.com> wrote:
>
Yes. Not sure what you could gain by trying to do it in pure ruby
though, as you would still just have to use a thread right? I think
the best option would be to just add it to existing C drivers where
the client library supports async io. Postgres does, I was just
looking at their specs.
The only reason I thought of doing it in Ruby is to save time, but your idea makes a lot of sense too. Anyone out there want to give this a shot?
|
| Re: Eventmachine 0.8.0 released with
epoll, breaks Ruby file-descriptor li |
|
2007-07-11 21:07:25 |
On 7/11/07, Francis Cianfrocca <garbagecat10gmail.com> wrote:
> On 7/11/07, snacktime <snacktimegmail.com> wrote:
> > >
> >
> > Yes. Not sure what you could gain by trying to do
it in pure ruby
> > though, as you would still just have to use a
thread right? I think
> > the best option would be to just add it to
existing C drivers where
> > the client library supports async io. Postgres
does, I was just
> > looking at their specs.
>
>
> The only reason I thought of doing it in Ruby is to
save time, but your idea
> makes a lot of sense too. Anyone out there want to give
this a shot?
Interesting, there is already async support in the c driver
I'll have
to test it out. It uses rb_thread_select to read the fd, so
in theory
it should only block the thread that's making the call to
the driver,
not the whole process. Wish I had looked at this a long
time ago!
Chris
_______________________________________________
Eventmachine-talk mailing list
Eventmachine-talkrubyforge.org
http://rubyforge.org/mailman/listinfo/eventmachine-talk
|
|
[1-10]
|
|