Issue 2802: Support for polling in arrival order is needed (messaging-rtf) Source: UBS (Mr. Hans Kneubuehl, hans.kneubuehl(at)ubs.com) Nature: Severity: Summary: 1. The fact that the receiving of replies and the matching of replies to the corresponding requests is hard-wired represents a serious limitation for TII. Support for polling in arrival order is needed. Resolution: Close no change with explanation as above. Revised Text: Actions taken: July 14, 1999: received issue October 3, 2001: closed issue Discussion: I dont really understand the problem here. One always has a different poller for each sendp request sent, and one can poll each of them in turn for a finite or even 0 time. One even has a construct such as PollableSet to make it easy to go around a number of pollers. Close this Issue. This appears to require a very significant change to the specification to support a usage pattern that appears to have not been part of the original adopted specification. It seems that this sort of a change could perhaps have been admissible in the first R/FTF, but seems out of scope for a subsequent RTF. Propose closing no change End of Annotations:===== From: hans.kneubuehl@ubs.com X-OpenMail-Hops: 2 Date: Thu, 9 Mar 2000 19:00:00 +0100 Message-Id: Subject: 2802: Polling by Objref and Operation Name MIME-Version: 1.0 TO: messaging-rtf@omg.org, uabcsru@uab.ericsson.se Content-Disposition: inline; filename="BDY.TXT" ;Creation-Date="Thu, 9 Mar 2000 19:00:00 +0100" Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII ;Creation-Date="Thu, 9 Mar 2000 19:00:00 +0100" X-UIDL: MG>!!a65e9f5d!!'a#e9 >From Proposed Resolutions for Vote 1. Draft 1 - 25th February 2000: >Issue 2802: Support for polling in arrival order is needed (messaging-rtf) > >Summary: 1. The fact that the receiving of replies and the matching of replies to the >corresponding requests is hard-wired represents a serious limitation for TII. >Support for polling in arrival order is needed. > >Proposed Resolution >I dont really understand the problem here. One always has a different poller for each >sendp request sent, and one can poll each of them in turn for a finite or even 0 >time. One even has a construct such as PollableSet to make it easy to go around a number >of pollers. >Close this Issue (Note that "polling in arrival order" has already led to confusion, I therefore call the requirement "polling by objref and operation name".) I want to try to illustrate the problem. The issue is that certain application architecture can only be done in a very awkward way with the current specs. Let's illustrate this with the following scenario: A large number of client applications is using an intermediate server A from where asynchronous TII-requests to an enterprise server are produced. The TII-ORB transports the reply back to the same node as the intermediate server from where the replies are retrieved by a daemon application B. For some reason B has to use the polling model, e.g. it has to do other things than retrieving replies and thus wants to have strict control of what it is doing at what time. (i) How can this be solved with the current specs? In order to implement this application architecture with the current specs, B has to use equivalent pollers to the one A receives back from the sendp calls. Oneway to achieve this (assuming that issue 2803 is solved) is that A constantly externalizes the resulting pollers to a persistent store. B then constantly creates pollers from the externalized data, adds them to a pollable set, checks if any replies are ready for retrieval, retrieves and processes the replies. Add transactions in order to ensure that no retrievel of any reply is lost. Add multiple instances of B and you need some locking between the PollableSet::is_ready and the actual poll call. (ii) How can this be solved with the extension I am proposing? In order to implement the same architecture, B can use special pollers that are not tied to a particular request, but are only initialized with the target objref of the request and the operation name. With this feature B is not required to constantly update a pollable set with pollers from externalized data. B just constantly creates a new poller, initializes it with target objref and operation name and retrieves the next reply available matching this objref and operation name. If B wants to associate the retrieved replies to the original requests, then the same as in (i), A constantly has to externalize the resulting pollers to the persistent store, from where B can create the equivalent pollers. When B retrieves a reply using an "objref and operation name"-initialized poller, the poller gets updated with the complete correlation information from the retrieved reply which then allows to compare it to the pollers initialized from the externalized data. -> As the above variants illustrate, the current specs require a very awkward implementation of B. -> B wants to retrieve all the requests, but in order to do this, it is forced to explicitly retreive each reply by an equivalent poller received on sending the request. -> B wants to retrieve the replies as soon as they are available, but in order to do this, it has to consequently go and update its pollable set. -> B is very complex because of limitations of the standard on how replies can be retrieved from a persistent router, and not because the application logic is requiring this. Personally, I don't think this is such an exotic scenario and therefore I would expect that CORBA Messaging gets more useful in this area. My proposal is therefore: - drop the readonly qualifiers of the attributes operation_target and operation_name of the generic poller valuetypes - define the semantics such that pollers can be initialized just by objref and operation name, and that such pollers then can be used to retrieve any reply that matches this objref and operation name ('Polling by objref and operation name' is completely analogous to sending the request: send: you specify target, operation name and input arguments poll: you specify target, operation name and output arguments) What are other people thinking about this? Regards Hans -- Hans Kneubuehl, UBS AG, P.O. Box, 8098 Zurich, Switzerland phone: +41 1 238 28 96, fax: +41 1 238 30 11 Sender: Chris.Smith@uab.ericsson.se Message-ID: <38CCE40A.D9858EE@uab.ericsson.se> Date: Mon, 13 Mar 2000 13:50:18 +0100 From: Chris Smith X-Mailer: Mozilla 4.07 [en] (X11; I; Linux 2.0.36 i686) MIME-Version: 1.0 To: hans.kneubuehl@ubs.com CC: messaging-rtf@omg.org Subject: Re: 2802: Polling by Objref and Operation Name References: Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=us-ascii X-UIDL: ~Of!!-b%"!'X[!!hW^d9 > (Note that "polling in arrival order" has already led to confusion, I therefore > call the requirement "polling by objref and operation name".) > > I want to try to illustrate the problem. The issue is that certain application > architecture can only be done in a very awkward way with the current specs. > snip... > (i) How can this be solved with the current specs? > In order to implement this application architecture with the current > specs, B > has to use equivalent pollers to the one A receives back from the > sendp calls. > Oneway to achieve this (assuming that issue 2803 is solved) is that > A > constantly externalizes the resulting pollers to a persistent > store. B then > constantly creates pollers from the externalized data, adds them to > a pollable > set, checks if any replies are ready for retrieval, retrieves and > processes the > replies. Alternatively, client A can pass the Poller to client B directly. > (ii) How can this be solved with the extension I am proposing? > In order to implement the same architecture, B can use special > pollers that are > not tied to a particular request, but are only initialized with the > target > objref of the request and the operation name. With this feature B is > not > required to constantly update a pollable set with pollers from > externalized > data. B just constantly creates a new poller, initializes it with > target objref > and operation name and retrieves the next reply available matching > this objref > and operation name. The big problem I have with this is that the information that one requires to identify client requests coming from client A is much more than target object ref and operation. With your proposal, client B can obtain replies for requests issued from any client that happened to be using the same intermediate router. This is a very big departure from the normal CORBA model. From: hans.kneubuehl@ubs.com X-OpenMail-Hops: 2 Date: Tue, 14 Mar 2000 13:55:27 +0100 Message-Id: Subject: Re: 2802: Polling by Objref and Operation Name MIME-Version: 1.0 TO: uabcsru@uab.ericsson.se CC: messaging-rtf@omg.org Content-Disposition: inline; filename="BDY.TXT" ;Creation-Date="Tue, 14 Mar 2000 13:55:27 +0100" Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII ;Creation-Date="Tue, 14 Mar 2000 13:55:27 +0100" X-UIDL: aFJ!!#h8e9M>ld9kd2e9 > -----Original Message----- > From: uabcsru [mailto:uabcsru@uab.ericsson.se] > Sent: Monday, March 13, 2000 1:50 PM > > (Note that "polling in arrival order" has already led to > confusion, I therefore > > call the requirement "polling by objref and operation name".) Another good term is "polling a stream of replies". This shows well the issue with the current specs: it requires that the stream consumer has to constantly feed the "stream" with the pollers received from sending the requests. > > I want to try to illustrate the problem. The issue is that > certain application > > architecture can only be done in a very awkward way with > the current specs. > > > snip... > > > (i) How can this be solved with the current specs? > > In order to implement this application architecture with > the current specs, B > > has to use equivalent pollers to the one A receives back > from the sendp calls. > > Oneway to achieve this (assuming that issue 2803 is solved) > is that A > > constantly externalizes the resulting pollers to a > persistent store. B then > > constantly creates pollers from the externalized data, adds > them to a pollable > > set, checks if any replies are ready for retrieval, > retrieves and processes the > > replies. > > Alternatively, client A can pass the Poller to client B directly. Yes, but that was not the point of my issue. The point is that B wants to consume a stream of replies, but the current specs requires a very awkward implementation of B in order to achieve it. B is forced to constantly update a pollable set with (equivalent) pollers that have been received on sending the request. I would like CORBA Messaging to provide for a mechanism where applications in the category of B just can retrieve the "next" reply (where "next" is qualified with target objref and operation name). > > (ii) How can this be solved with the extension I am proposing? > > In order to implement the same architecture, B can use > special pollers that are > > not tied to a particular request, but are only initialized > with the target > > objref of the request and the operation name. With this > feature B is not > > required to constantly update a pollable set with pollers > from externalized > > data. B just constantly creates a new poller, initializes > it with target objref > > and operation name and retrieves the next reply available > matching this objref > > and operation name. > > The big problem I have with this is that the information that one > requires to identify client requests coming from client A is much > more than target object ref and operation. With your proposal, > client B can obtain replies for requests issued from any client > that happened to be using the same intermediate router. This is > a very big departure from the normal CORBA model. Ok, I see your problem, but I think it is no problem: this information can stored in the Objref. I can best illustrate this by explaining how we have solved this in our implementation of TII. We take IORs that contain the routing information for how to call the target synchronously (part used by the target router) and extend it with the routing information for how to transport the request via message queues. Basically, we extend the IOR with a queue name. For different applications, different IORs for the same target but with different queue names are configured. Thus, we allow even a lot of different clients to use the same intermediate router, but they all use a target IOR with a differing queue name such that B only receives replies from A.) I think the IOR is the most natural place to do this, because its purpose is really to provide the necessary routing information to the ORB in order to find out where to transport requests. So it is natural to extend it with async routing information needed for TII. We don't have implemented Message Routing Interoperability, but I think with a pure Messaging::Router-implementation, different clients would have to be routed to different client routers such that B only receives replies from A. In fact, our implementation takes security into account, too. Even if other clients are routed to the same queue, client B only see replies from clients for which it is allowed to see them (we transport principal data as part of the service context). Does this solve the problem? Regards Hans -- Hans Kneubuehl, UBS AG, P.O. Box, 8098 Zurich, Switzerland phone: +41 1 238 28 96, fax: +41 1 238 30 11 Sender: Chris.Smith@uab.ericsson.se Message-ID: <38CE703B.F7AA7FD@uab.ericsson.se> Date: Tue, 14 Mar 2000 18:00:43 +0100 From: Chris Smith X-Mailer: Mozilla 4.07 [en] (X11; I; Linux 2.0.36 i686) MIME-Version: 1.0 To: hans.kneubuehl@ubs.com CC: messaging-rtf@omg.org Subject: Re: 2802: Polling by Objref and Operation Name References: Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=us-ascii X-UIDL: -5k!!jd=!!d]0!!=@?e9 > Another good term is "polling a stream of replies". This shows well the issue > with the current specs: it requires that the stream consumer has to constantly > feed the "stream" with the pollers received from sending the requests. That is the part of the spec that I would like to defend :-) If you let a client obtain replies to requests that it knew nothing about, then you could have chaos. See below for more comments. > > > (ii) How can this be solved with the extension I am proposing? > > > In order to implement the same architecture, B can use > > special pollers that are > > > not tied to a particular request, but are only initialized > > with the target > > > objref of the request and the operation name. With this > > feature B is not > > > required to constantly update a pollable set with pollers > > from externalized > > > data. B just constantly creates a new poller, initializes > > it with target objref > > > and operation name and retrieves the next reply available > > matching this objref > > > and operation name. > > > > The big problem I have with this is that the information that one > > requires to identify client requests coming from client A is much > > more than target object ref and operation. With your proposal, > > client B can obtain replies for requests issued from any client > > that happened to be using the same intermediate router. This is > > a very big departure from the normal CORBA model. > > Ok, I see your problem, but I think it is no problem: this information can stored in the Objref. > > I can best illustrate this by explaining how we have solved this in > our > implementation of TII. We take IORs that contain the routing > information for > how to call the target synchronously (part used by the target > router) and > extend it with the routing information for how to transport the > request via > message queues. Basically, we extend the IOR with a queue name. For > different > applications, different IORs for the same target but with different > queue names > are configured. Thus, we allow even a lot of different clients to > use the same > intermediate router, but they all use a target IOR with a differing > queue name > such that B only receives replies from A.) > > I think the IOR is the most natural place to do this, because its > purpose is > really to provide the necessary routing information to the ORB in > order to find > out where to transport requests. So it is natural to extend it with > async > routing information needed for TII. I know that you CAN do these things, but we are specifying a standard here. That means that mechanisms MUST work and MUST ALWAYS work independant of how they are implemented (as long as they conform to the specification). > > We don't have implemented Message Routing Interoperability, but I > think with a > pure Messaging::Router-implementation, different clients would have > to be > routed to different client routers such that B only receives replies > from A. That is not the way the Messaging specification is written though. Replies are sent to the right place on the basis of actually having a Poller, not on the basis of the original client request runtime knowing to send it to a particular router. How would the runtime portably and interoperably know that certain requests should go to one router, and other requests go to another router. > > In fact, our implementation takes security into account, too. Even > if other > clients are routed to the same queue, client B only see replies from > clients > for which it is allowed to see them (we transport principal data as > part of the > service context). > > Does this solve the problem? No, not for the standard (in my opinion). Its obvious how you CAN do these things for a certain constrained environment, for certain type of applications, but the Messaging spec is supposed to be generic. Do you have something concrete for putting into the Vote 1 tomorrow? > > Regards > Hans > -- > Hans Kneubuehl, UBS AG, P.O. Box, 8098 Zurich, Switzerland > phone: +41 1 238 28 96, fax: +41 1 238 30 11 Date: Tue, 03 Apr 2001 18:46:56 -0400 From: Jishnu Mukerji Organization: Hewlett-Packard EIAL, Florham Park NJ USA X-Mailer: Mozilla 4.73 [en] (WinNT; U) X-Accept-Language: en MIME-Version: 1.0 To: orb_revision@omg.org Subject: Issue 2802 proposed resolution Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=us-ascii X-UIDL: JgNe9-T+!!RUOd9Tme!! Status: RO Issue 2802: Support for polling in arrival order is needed (messaging-rtf) Click here for this issue's archive. Source: UBS (Mr. Hans Kneubuehl, hans.kneubuehl@ubs.com) Nature: Severity: Summary: The fact that the receiving of replies and the matching of replies to the corresponding requests is hard-wired represents a serious limitation for TII. Support for polling in arrival order is needed. Resolution: This appears to require a very significant change to the specification to support an usage pattern that appears to have not been part of the original adopted specification. It seems that this sort of a change could perhaps have been admissible in the first R/FTF, but seems out of scope for a subsequent RTF. Propose closing no change. Revised Text: Actions taken: Close no change with explanation as above. _____________________________________________________________________ The proposed resolution above will appear in the next vote unless people feel that it needs further discussion. Thanks, Jishnu.