Using Em within rails

So today at work it looks like I lost the debate on using an
rdbms
versus a distributed data store for our product, which btw
has 0 users
so far.   But anyways that's another story.

So along with this is the need to do async http requests
from within
rails.   Basically I have to make one http request per data
object I
want from the backend.  So normally  I will need to make 1-4
async
http requests, wait for them to return (or timeout), then
continue
processing as normal in the rails controller.  I think this
can
probably be done with EM.  Can anyone think of a reason not
to start
up Em within rails, perform the http requests, then stop the
reactor
and continue on?  Is there any significant overhead with
starting the
reactor?

Chris
_______________________________________________
Eventmachine-talk mailing list
Eventmachine-talkrubyforge.org
http://rubyforge.org/mailman/listinfo/eventmachine-talk

On Oct 23, 2007, at 2:01 AM, snacktime wrote:

> So today at work it looks like I lost the debate on
using an rdbms
> versus a distributed data store for our product, which
btw has 0 users
> so far.   But anyways that's another story.
>
> So along with this is the need to do async http
requests from within
> rails.   Basically I have to make one http request per
data object I
> want from the backend.  So normally  I will need to
make 1-4 async
> http requests, wait for them to return (or timeout),
then continue
> processing as normal in the rails controller.  I think
this can
> probably be done with EM.  Can anyone think of a reason
not to start
> up Em within rails, perform the http requests, then
stop the reactor
> and continue on?  Is there any significant overhead
with starting the
> reactor?

I'm no EM expert, but I have two alternative suggestions.

1. Use BackgrounDRb. It was tailor made for this exact
scenario. It  
hasn't gotten a lot of love in the last few months, but the
code works.

2. Merb allows the programmer to "render" a Proc
which releases the  
internal mutex (so it can handle a new request) and Mongrel
takes  
care of streaming the response back to the client in a
thread-safe  
manner. Of course, you can "chain" multiple Procs
together, so if  
your asynch requests require any kind of serialization then
this  
would enforce that.

(I'm a Merb newbie, so take the above with the appropriate
grain of  
salt.)

Other than that, EM would very likely work just fine. I
would highly  
recommend against starting/stopping it on each asynch call.
What  
would be the point? Open a socket connection between your
Rails  
controller and EM and tear down the connection when you are
done. But  
leave EM running. The overhead is the socket setup/teardown
instead  
of a *process* setup/teardown which is what you suggested.

cr

_______________________________________________
Eventmachine-talk mailing list
Eventmachine-talkrubyforge.org
http://rubyforge.org/mailman/listinfo/eventmachine-talk

On 10/23/07, Francis Cianfrocca <garbagecat10gmail.com> wrote:
> On 10/23/07, Chuck Remes <cremes.devlistmac.com> wrote:
> >  Other than that, EM would very likely work just
fine. I would highly
> > recommend against starting/stopping it on each
asynch call. What
> > would be the point? Open a socket connection
between your Rails
> > controller and EM and tear down the connection
when you are done. But
> > leave EM running. The overhead is the socket
setup/teardown instead
> > of a *process* setup/teardown which is what you
suggested.
>
>
> Another point:  I have an EM-based web framework that
was designed
> specifically for REST operations that would . In it,
every HTTP response is
> automatically an EM Deferrable, so it's trivial to do
what Chris needs to
> do: kick off one or more operations that require access
to external
> resources (like other web servers or distributed data
stores) and keep
> merrily processing other incoming client requests while
waiting for the
> responses.
>
> I've been hesitant to release this thing for a couple
of reasons. First,
> it's married to the EM HTTP frontend, and it really
should be modified to
> work with webrick, apache, lighty, nginx, or whatever.
And second, the Ruby
> world already has Rails and Merb and probably doesn't
need yet another Web
> framework, even though the emphasis of this one (REST
and external-services)
> is different. (Over in Javaland, there are hundreds
specialized web
> frameworks.)
>
> Any opinions?
>

Release it Francis, I have several use cases of a simple
HTTP frontend
without any heavy weight framework. I don't know, but unless
you
intend run entire rails on top of EventMachine as swiftply
of Kirk
Haines, its a bad idea to have EventMachine.run in your
controller.



For Chris, No you can't do async requests in rails. There is
a way
using mongrel_handlers and sleep. The code looks something
like this:


require 'active_record'

class StatusHandler < Mongrel::HttpHandler
   def process(request, response)
      id = request.params['PATH_INFO'].slice(1, 20)
      current = request.params['QUERY_STRING']

      while status(id) == current do
         sleep 0.2
      end

      response.start(200) do |head, out|
         head["Content-Type"] =
"text/html"
         out.write status(id)
      end
   end

   def status(id)
      connection.select_value("select status from
auctions where id=#{id.to_i}")
   end

   def connection
      ActiveRecord::Base.connection
   end
end

uri "/status", :handler => StatusHandler.new,
:in_front => true

Taken verbatim from :
http://adam.blogs.bitscribe.net/2007/05/08/c
omet-with-rails-mongrel/

All the best.



-- 
Let them talk of their oriental summer climes of
everlasting
conservatories; give me the privilege of making my own
summer with my
own coals.

http://gnufied.org
_______________________________________________
Eventmachine-talk mailing list
Eventmachine-talkrubyforge.org
http://rubyforge.org/mailman/listinfo/eventmachine-talk

On 10/23/07, snacktime <snacktimegmail.com> wrote:
> So today at work it looks like I lost the debate on
using an rdbms
> versus a distributed data store for our product, which
btw has 0 users
> so far.   But anyways that's another story.
>
> So along with this is the need to do async http
requests from within
> rails.   Basically I have to make one http request per
data object I
> want from the backend.  So normally  I will need to
make 1-4 async
> http requests, wait for them to return (or timeout),
then continue
> processing as normal in the rails controller.  I think
this can
> probably be done with EM.  Can anyone think of a reason
not to start
> up Em within rails, perform the http requests, then
stop the reactor
> and continue on?  Is there any significant overhead
with starting the
> reactor?

If you use evented_mongrel, then your app would already be
running
inside of an EM loop.  No need to start and stop the
reactor.


Kirk Haines
_______________________________________________
Eventmachine-talk mailing list
Eventmachine-talkrubyforge.org
http://rubyforge.org/mailman/listinfo/eventmachine-talk

On 10/23/07, Francis Cianfrocca <garbagecat10gmail.com> wrote:

> I've been hesitant to release this thing for a couple
of reasons. First,
> it's married to the EM HTTP frontend, and it really
should be modified to
> work with webrick, apache, lighty, nginx, or whatever.
And second, the Ruby
> world already has Rails and Merb and probably doesn't
need yet another Web
> framework, even though the emphasis of this one (REST
and external-services)
> is different. (Over in Javaland, there are hundreds
specialized web
> frameworks.)

I have a bunch of optimizations for the HTTP parser, as well
as a
change to allow the header to be processed before the
POST/PUT body is
touched, that I need to get over to you sometime, Francis. 
I've been
hacking on it heavily because I am using it as the basis for
a sort of
micro-framework specifically aimed at web app edge cases --
things
like upload handling, which are long, slow, IO intensive
requests, as
well as ajax, service APIs, and simple request handling that
benefits
more from being very fast than from from being feature
heavy.

On an AMD 4200+ CPU, it will do about 14k hello worlds a
second and on
an Intel 3.0Ghz Xeon CPU, it's doing over 20k/second right
now.


Kirk Haines
_______________________________________________
Eventmachine-talk mailing list
Eventmachine-talkrubyforge.org
http://rubyforge.org/mailman/listinfo/eventmachine-talk

> > I have a bunch of optimizations for the HTTP
parser, ..
> >
> > On an AMD 4200+ CPU, it will do about 14k hello
worlds a second and on
> > an Intel 3.0Ghz Xeon CPU, it's doing over
20k/second right now.
>
> That sounds great! It may be what's needed to get more
attention for the EM
> HTTP server.
>

Get Roy Fielding's attention - before he starts trying to
sell Waka:

"" I went through the readable bits with HTTP
and it turned out to be a big mistake.  Nobody reads HTTP in
real
practice, yet the overhead of parsing HTTP messages is
huge.""

http://lists.w3.org/Archives/Public/uri/2007Oct/0054.ht
ml

=0)

Cheers!

Duncan

__________________________
Duncan Cragg
The Financial Times Group (UK)
http://duncan-cragg.org
/blog/
_______________________________________________
Eventmachine-talk mailing list
Eventmachine-talkrubyforge.org
http://rubyforge.org/mailman/listinfo/eventmachine-talk