Prev: Rake versus Ruby?
Next: Read text file and set the values in the Variables
From: Tony Arcieri on 18 May 2010 22:44
[Note: parts of this message were removed to make it a legal post.]

On Tue, May 18, 2010 at 7:56 PM, Daniel DeLorme <dan-ml(a)dan42.com> wrote:

> What I can't figure out with EventMachine is how to have the "main
> thread" generate the layout while the subrequests are executing.
>

The problem here is inversion of control. EventMachine inverts control on
you, and it sucks. You can't just do subreq(...) and expect it to return a
value. In the best case, you have subreq call a block when it completes.
The familiar pattern of "call function, get value" no longer applies.

--
Tony Arcieri
Medioh! A Kudelski Brand

From: Roger Pack on 19 May 2010 11:36
Tony Arcieri wrote:
> On Tue, May 18, 2010 at 10:26 PM, Bill Kelly <billk(a)cts.com> wrote:
>
>> Sorry if I'm being redundant, but I wanted to point out that EventMachine
>> *can* support non-inverted control semantics, with a Fiber-based wrapper
>> layer.
>
>
> And I'm pretty sure I was the first person to ever implement a Ruby
> Fiber-based wrapper which provides normal flow control semantics on top
> of
> an IoC-driven event-based framework with Revactor, for what it's worth.

Yeah I was going to mention revactor as well.

There is "async sinatra" if that's any help.

http://github.com/raggi/async_sinatra
--
Posted via http://www.ruby-forum.com/.

From: Daniel DeLorme on 19 May 2010 07:29
Daniel N wrote:
> On 19 May 2010 11:56, Daniel DeLorme <dan-ml(a)dan42.com> wrote:
>> Ok, here's the context. I didn't put this in my OP because I figured
>> it would just bore everybody to tears.
>>
>> This is inside a rack request. The idea is that I'm assembling a web
>> page by doing a bunch of sub-requests for the various parts of the
>> page. So I'll have something like:
>>
>> action "index" do
>> @news = subreq("http://news.server")
>> @ad = subreq("http://ad.server")
>> @blog = subreq("http://blog.server")
>> @forum = subreq("http://forum.server")
>> end
>>
>> All these sub-requests are launched asynchronously and, while they are
>> executing, the app generates the layout within which the output of the
>> subrequests will be embedded. So I'll have something like:
>>
>> response = ['<html><body>',
>> '<div>',@ad,'</div>',
>> '<div>',@news,'</div>',
>> '<div>',@blog,'</div>',
>> '<div>',@forum,'</div>',
>> '</body></html>']
>>
>> And when rack finally outputs the response to the client it will block
>> on the various subrequests unless/until they have completed.
>>
>> What I can't figure out with EventMachine is how to have the "main
>> thread" generate the layout while the subrequests are executing.
>>
>>
> Ok now we're talking. So with rack you can't do true async with a callback.

Ah, but I never really wanted callbacks; that would be evented IO. The
various approaches to asynchronous IO are not terribly well defined, but
what I meant by nonblocking was simply:
resource.send_request #=> doesn't block waiting for response!
resource.get_response
This is not possible with Net::HTTP because those two phases of the http
request are bound into one monolithic get(url) operation.

> callback based async actually needs to block in order for the rack
> application you're in to return it's result. You _could_ do it by returning
> a custom object in the rack response that renders as much as possible while
> it waits for the response, and then renders that when it can, but that
> option may not be available depending on what framework you're using.

I guess I was not clear enough, but this approach is exactly what I
tried to explain above.

> You can use ESI. There's an esi for rack project on github by Joshua Hull
> which could be useful to you. http://github.com/joshbuddy/esi-for-rack You
> can also use esi outside of the rack request in apache or nginx, by
> responding first with a layout file containing esi tags pointing to the
> content to use. Ngins, Apache or the esi rack project can then assemble the
> page for you using the resources specified.

Oh wow this was *really* interesting. This sent me on an hours-long
exploration of Varnish+ESI and Nginx+SSI. It turns out that Nginx will
fetch the subrequests in parallel but Varnish (caching proxy) will not.
This is probably fine if most of the subrequests are already cached
(which admittedly is the point of a caching proxy) but if not... Nginx
is the winner.

This opens a lot of possibilities. For example I can imagine serving up
simple "skeleton" pages with heavy caching and then generate all the
user-specific parts through SSI.


From: Daniel N on 17 May 2010 23:41
[Note: parts of this message were removed to make it a legal post.]

On 13 May 2010 17:37, Daniel DeLorme <dan-ml(a)dan42.com> wrote:

> Does anyone know how to do the following, but without threads, purely with
> asynchronous IO?
>
> website = Thread.new{ Net::HTTP.get(URI.parse(url)) }
> template = compute_lots_of_stuff()
> puts template.sub("<content goes here>", website.value)
>
> I'm not sure I understand EventMachine, but it doesn't seem like this code
> fits with the "event loop" model. Besides, I don't want to react to every
> chunk of data that comes it; I just want the result at the end.
>
> Thanks.
>

Event machine is perfect for this kind of stuff. Weather it fits with the
rest of your web framework is more likely the thing that makes it an
unlikely selection (if you're using anything rack based for example)

http://eventmachine.rubyforge.org/EventMachine/Protocols/HttpClient.html
or
http://eventmachine.rubyforge.org/EventMachine/Protocols/HttpClient2.html

Give an example of how this is used.

HTH
Daniel

From: Zach Moazeni on 17 May 2010 23:17
You could check out http://github.com/pauldix/typhoeus

It allows you to create parallel HTTP requests pretty easily.

On May 13, 2010, at 3:37 AM, Daniel DeLorme wrote:

> Does anyone know how to do the following, but without threads, purely with asynchronous IO?
>
> website = Thread.new{ Net::HTTP.get(URI.parse(url)) }
> template = compute_lots_of_stuff()
> puts template.sub("<content goes here>", website.value)
>
> I'm not sure I understand EventMachine, but it doesn't seem like this code fits with the "event loop" model. Besides, I don't want to react to every chunk of data that comes it; I just want the result at the end.
>
> Thanks.
>


--
Zach Moazeni
http://simplechatter.com


First  |  Prev  |  Next  |  Last
Pages: 1 2 3
Prev: Rake versus Ruby?
Next: Read text file and set the values in the Variables