Prev: _chdrive / SetCurrentDirectory resetting the current directory on other drives?
Next: Microsoft Responds to the Evolution of Online Communities
From: Hector Santos on 3 May 2010 23:56
Folks, I revamping our internet hosting session-based RPC
client/server framework around IOCP and Async RPC and I have some
questions in create the new framework.

Background:

The current framework is a traditional client/server model where there
is main thread RPC client as a hosting listing server and each
connection spawns a new RPC client session thread. The RPC Server
offers 250+ functions API DLL library for RPC clients to use.

I have the socket IOCP layout worked out for socket connections and to
dispatch commands for the particular hosting server.

To use one command example, SEARCH, the dispatch is:

BOOL iocpDisp_Search(LPCLIENT_CTX lpCtx, char *szText)
{
//
// illustrating sync RPC call
//
SOCKET t = lpCtx->hClientSocket;
TMsgHeader msg = {0};
while (wcMailSearch(szText,msg)) {
// do something with msg
Sendf(t,"%u,%u\r\n",msg.Id, msg.Number);
msg.Id++;
}
return TRUE;
}

The goal is to make the above async RPC using IOCP notification.

I guess I am not sure of:

1) If I should have two I/O complete ports, one for sockets,
one for handling Async RPC calls.

2) How to keep/pass/separate/link the RPC_ASYNC_STATE instance with
the socket LPCLIENT_TXT instance. In other words, do I need to
initialize a unique RPC_ASYNC_STATE per async RPC function?

Overall, how should I prepare the structures for what is otherwise two
IOCP "streams"?

Would it be better to use Async RPC with a callback notification?

Thanks in advance

--
HLS
From: m on 4 May 2010 08:37
In my experience, a single pool of worker threads (IOCP) for both IO and
processing is usually the most efficient design. Having said that, there
are many reasons why separate pools might be better including better
modularity of code, and easier support for NUMA and processor groups. One
can also implement multiple pools to support class of service etc. and
session quotas. The key is to remember that no matter how many threads,
there are still only X CPUs in the system and each context switch adds
overhead. Conversely, a simple design that minimizes context switches, can
leave you exposes to denial of sevice.

"Hector Santos" <sant9442(a)nospam.gmail.com> wrote in message
news:usDrV3z6KHA.5808(a)TK2MSFTNGP02.phx.gbl...
> Folks, I revamping our internet hosting session-based RPC client/server
> framework around IOCP and Async RPC and I have some questions in create
> the new framework.
>
> Background:
>
> The current framework is a traditional client/server model where there is
> main thread RPC client as a hosting listing server and each connection
> spawns a new RPC client session thread. The RPC Server offers 250+
> functions API DLL library for RPC clients to use.
>
> I have the socket IOCP layout worked out for socket connections and to
> dispatch commands for the particular hosting server.
>
> To use one command example, SEARCH, the dispatch is:
>
> BOOL iocpDisp_Search(LPCLIENT_CTX lpCtx, char *szText)
> {
> //
> // illustrating sync RPC call
> //
> SOCKET t = lpCtx->hClientSocket;
> TMsgHeader msg = {0};
> while (wcMailSearch(szText,msg)) {
> // do something with msg
> Sendf(t,"%u,%u\r\n",msg.Id, msg.Number);
> msg.Id++;
> }
> return TRUE;
> }
>
> The goal is to make the above async RPC using IOCP notification.
>
> I guess I am not sure of:
>
> 1) If I should have two I/O complete ports, one for sockets,
> one for handling Async RPC calls.
>
> 2) How to keep/pass/separate/link the RPC_ASYNC_STATE instance with
> the socket LPCLIENT_TXT instance. In other words, do I need to
> initialize a unique RPC_ASYNC_STATE per async RPC function?
>
> Overall, how should I prepare the structures for what is otherwise two
> IOCP "streams"?
>
> Would it be better to use Async RPC with a callback notification?
>
> Thanks in advance
>
> --
> HLS

From: Hector Santos on 4 May 2010 20:35
m wrote:

> In my experience, a single pool of worker threads (IOCP) for both IO and
> processing is usually the most efficient design. Having said that,
> there are many reasons why separate pools might be better including
> better modularity of code, and easier support for NUMA and processor
> groups. One can also implement multiple pools to support class of
> service etc. and session quotas. The key is to remember that no matter
> how many threads, there are still only X CPUs in the system and each
> context switch adds overhead. Conversely, a simple design that
> minimizes context switches, can leave you exposes to denial of sevice.


There was two reasons: separation of I/O channels including one that
already exist (ncacn_ip_tcp) and not seeing how to associate an async
RPC "file handle object" with an iocp.

I'm not referring to RPC_ASYNC_STATE.u.IOC.hIOport but with
CreateIOCompletionPort(). I was presuming the RPC system manager has
logic to making the association and bind.

In other words, in this case, the RPC bind is ncacn_ip_tcp (socket) or
ncalrpc (some unknown kernel port, shared memory) and it is probably
already using an IOCP internally.

So far its working very well, but I'm still far from any real full
blown testing.

Thanks

--
HLS
From: m on 4 May 2010 21:29
AFAIK, there is no way ;) This is one of those 'other' reasons!

I am glad that you have it working at least for test

"Hector Santos" <sant9442(a)nospam.gmail.com> wrote in message
news:uhb1rr#6KHA.4804(a)TK2MSFTNGP02.phx.gbl...
> m wrote:
>
>> In my experience, a single pool of worker threads (IOCP) for both IO and
>> processing is usually the most efficient design. Having said that, there
>> are many reasons why separate pools might be better including better
>> modularity of code, and easier support for NUMA and processor groups.
>> One can also implement multiple pools to support class of service etc.
>> and session quotas. The key is to remember that no matter how many
>> threads, there are still only X CPUs in the system and each context
>> switch adds overhead. Conversely, a simple design that minimizes context
>> switches, can leave you exposes to denial of sevice.
>
>
> There was two reasons: separation of I/O channels including one that
> already exist (ncacn_ip_tcp) and not seeing how to associate an async RPC
> "file handle object" with an iocp.
>
> I'm not referring to RPC_ASYNC_STATE.u.IOC.hIOport but with
> CreateIOCompletionPort(). I was presuming the RPC system manager has logic
> to making the association and bind.
>
> In other words, in this case, the RPC bind is ncacn_ip_tcp (socket) or
> ncalrpc (some unknown kernel port, shared memory) and it is probably
> already using an IOCP internally.
>
> So far its working very well, but I'm still far from any real full blown
> testing.
>
> Thanks
>
> --
> HLS

 | 
Pages: 1
Prev: _chdrive / SetCurrentDirectory resetting the current directory on other drives?
Next: Microsoft Responds to the Evolution of Online Communities