Issue 5266: ForwardRequest is impossible to detect in clients (corba-rtf) Source: Oracle (Dr. Andrew Piper, andyp(at)bea.com) Nature: Uncategorized Issue Severity: Summary: REQUIREMENT To be able to use interceptors, and in particular ForwardRequest, in a client to perform active, per-request, load-balancing. PROBLEM It is not possible to detect in an interceptor whether ForwardRequest has previously been thrown for the same client request. Thus it is possible for a client to go into an infinite loop throwing ForwardRequest. DISCUSSION The basic problem is that although for a single client request the request-scoped PICurrent is shared across across interceptor invocations - even when ForwardRequest is thrown - it is not possible to *modify* this information in an interceptor to indicate to a future invocation that the invocation has been seen. The two relevant parts of the spec here are: 21.3.6.5 For retries, depending on the policies in effect, a new request may or may not follow when a retry has been indicated. If a new request does follow, while this request is a new request, with respect to Interceptors, there is one point of correlation between the original request and the retry: because control has not returned to the client, the request scoped PortableInterceptor::Current for both the original request and the retrying request is the same (see Chapter 21, Portable Interceptors on page 21-32). 21.4.2 Before an invocation is made, PICurrent is obtained via a call to ORB::resolve_initial_references ("PICurrent") From within the interception points, the data on PICurrent that has moved from the thread scope to the request scope is available via the get_slot operation on the RequestInfo object. A PICurrent can still be obtained via resolve_initial_references, but that is the Interceptor's thread scope PICurrent. See section 21.4.4.4, Request Scope vs Thread Scope on page 21-36 for a detailed discussion of the scope of PICurrent. Thus modifications to the thread's PICurrent are lost on retries and modifications to the request's PICurrent are not possible. PROPOSED RESOLUTION I have made several different attempts at coming up with a portable way of solving this problem without changing the spec, but have failed. It seems to me that it really should be possible for the interceptor to know that a retry is in effect and I can think of a number of different solutions to this: 1. add: void set_slot (in SlotId id, in any data) raises (InvalidSlot); to RequestInfo. This would allow interceptors to transfer information between invokes of the same client request and thus a retry could be detected. 2. Add a new function to RequestInfo to indicate that a forward is in operation. The minimalist fix here would be to allow forward_reference() to be accessed in send_request() as well as in receive_other(). i.e. returning the object from the previous ForwardRequest if that has been thrown. I'm ambivalent about which of these is best but for the sake of simplicity I'm going to plump for (1) because this is already allowed in ServerRequestInfo. So: - Change the IDL in 21.3.12 to include void set_slot (in SlotId id, in any data) raises (InvalidSlot); - After 21.3.12.12 move in the text from 21.3.14.6 - Change the IDL in 21.3.14 to remove set_slot() Resolution: Revised Text: Actions taken: May 2, 2002: received issue April 11, 2012: Deferred Discussion: End of Annotations:===== X-Sender: andyp@san-francisco.beasys.com X-Mailer: QUALCOMM Windows Eudora Version 4.3.2 Date: Wed, 01 May 2002 16:28:06 -0700 To: corba-rtf@omg.org From: Andy Piper Subject: ForwardRequest is impossible to detect in clients Cc: issues@omg.org, ec@bea.com REQUIREMENT To be able to use interceptors, and in particular ForwardRequest, in a client to perform active, per-request, load-balancing. PROBLEM It is not possible to detect in an interceptor whether ForwardRequest has previously been thrown for the same client request. Thus it is possible for a client to go into an infinite loop throwing ForwardRequest. DISCUSSION The basic problem is that although for a single client request the request-scoped PICurrent is shared across across interceptor invocations - even when ForwardRequest is thrown - it is not possible to *modify* this information in an interceptor to indicate to a future invocation that the invocation has been seen. The two relevant parts of the spec here are: 21.3.6.5 For retries, depending on the policies in effect, a new request may or may not follow when a retry has been indicated. If a new request does follow, while this request is a new request, with respect to Interceptors, there is one point of correlation between the original request and the retry: because control has not returned to the client, the request scoped PortableInterceptor::Current for both the original request and the retrying request is the same (see Chapter 21, Portable Interceptors on page 21-32). 21.4.2 Before an invocation is made, PICurrent is obtained via a call to ORB::resolve_initial_references ("PICurrent") From within the interception points, the data on PICurrent that has moved from the thread scope to the request scope is available via the get_slot operation on the RequestInfo object. A PICurrent can still be obtained via resolve_initial_references, but that is the Interceptor's thread scope PICurrent. See section 21.4.4.4, Request Scope vs Thread Scope on page 21-36 for a detailed discussion of the scope of PICurrent. Thus modifications to the thread's PICurrent are lost on retries and modifications to the request's PICurrent are not possible. PROPOSED RESOLUTION I have made several different attempts at coming up with a portable way of solving this problem without changing the spec, but have failed. It seems to me that it really should be possible for the interceptor to know that a retry is in effect and I can think of a number of different solutions to this: 1. add: void set_slot (in SlotId id, in any data) raises (InvalidSlot); to RequestInfo. This would allow interceptors to transfer information between invokes of the same client request and thus a retry could be detected. 2. Add a new function to RequestInfo to indicate that a forward is in operation. The minimalist fix here would be to allow forward_reference() to be accessed in send_request() as well as in receive_other(). i.e. returning the object from the previous ForwardRequest if that has been thrown. I'm ambivalent about which of these is best but for the sake of simplicity I'm going to plump for (1) because this is already allowed in ServerRequestInfo. So: - Change the IDL in 21.3.12 to include void set_slot (in SlotId id, in any data) raises (InvalidSlot); - After 21.3.12.12 move in the text from 21.3.14.6 - Change the IDL in 21.3.14 to remove set_slot() andy Date: Wed, 01 May 2002 23:57:21 +0000 From: Harold Carr X-Mailer: Mozilla 4.51 [en] (WinNT; U) X-Accept-Language: en To: Andy Piper CC: corba-rtf@omg.org, issues@omg.org, ec@bea.com Subject: Re: ForwardRequest is impossible to detect in clients Hello Andy, > PROPOSED RESOLUTION > > 1. add: > void set_slot (in SlotId id, in any data) raises (InvalidSlot); > to RequestInfo. This would allow interceptors to transfer information > between invokes of the same client request and thus a retry could be detected. I'm not a TMI expert, but as I recall set_slot is not allowed on ClientRequestInfo because of Messaging/TMI. Otherwise an interceptor may set_slots which effect a different request. > > 2. Add a new function to RequestInfo to indicate that a forward is in > operation. The minimalist fix here would be to allow forward_reference() to > be accessed in send_request() as well as in receive_other(). i.e. returning > the object from the previous ForwardRequest if that has been thrown. Rather than allow forward_reference to be accessed in send_request I would suggest something more general, something like: // Only allowed in receive_other void ClientRequestInfo::set_retry_info(Any any) // Only allowed in send_request and send_poll Any ClientRequestInfo::get_retry_info() If not set then returns tk_null (like empty slots). Cheers, Harold X-Sender: andyp@san-francisco.beasys.com X-Mailer: QUALCOMM Windows Eudora Version 4.3.2 Date: Wed, 01 May 2002 17:12:21 -0700 To: Harold Carr From: Andy Piper Subject: Re: ForwardRequest is impossible to detect in clients Cc: corba-rtf@omg.org, issues@omg.org, ec@bea.com At 11:57 PM 5/1/2002 +0000, Harold Carr wrote: Hello Andy, > PROPOSED RESOLUTION > > 1. add: > void set_slot (in SlotId id, in any data) raises (InvalidSlot); > to RequestInfo. This would allow interceptors to transfer information > between invokes of the same client request and thus a retry could be detected. I'm not a TMI expert, but as I recall set_slot is not allowed on ClientRequestInfo because of Messaging/TMI. Otherwise an interceptor may set_slots which effect a different request. > > 2. Add a new function to RequestInfo to indicate that a forward is in > operation. The minimalist fix here would be to allow forward_reference() to > be accessed in send_request() as well as in receive_other(). i.e. returning > the object from the previous ForwardRequest if that has been thrown. Rather than allow forward_reference to be accessed in send_request I would suggest something more general, something like: // Only allowed in receive_other void ClientRequestInfo::set_retry_info(Any any) You would have to allow this in send_request() also I believe. It is my understanding that only other interceptors get receive_other() - at least thats what happens in the JDK :) Certainly the problem requires a solution that allows setting of information in send_request(). // Only allowed in send_request and send_poll Any ClientRequestInfo::get_retry_info() If not set then returns tk_null (like empty slots). I'm fine with this. I guess it irks me that PICurrent for the request is explicitly called-out in the spec as the thing that carries the request information along and yet there is no way of leveraging that. andy Sender: jbiggar@Resonate.com Date: Wed, 01 May 2002 17:20:53 -0700 From: Jonathan Biggar X-Mailer: Mozilla 4.79 [en] (X11; U; SunOS 5.6 sun4u) X-Accept-Language: en To: Harold Carr CC: Andy Piper , corba-rtf@omg.org, ec@bea.com Subject: Re: ForwardRequest is impossible to detect in clients Harold Carr wrote: > > PROPOSED RESOLUTION > > > > > 1. add: > > void set_slot (in SlotId id, in any data) raises (InvalidSlot); > > to RequestInfo. This would allow interceptors to transfer information > > between invokes of the same client request and thus a retry could be detected. > > I'm not a TMI expert, but as I recall set_slot is not allowed on > ClientRequestInfo because of Messaging/TMI. Otherwise an interceptor > may set_slots which effect a different request. Hmmm, are you saying that RequestInfo isn't bound only to the particular request and the proposed set_slot could affect some other request as well? > > 2. Add a new function to RequestInfo to indicate that a forward is in > > operation. The minimalist fix here would be to allow forward_reference() to > > be accessed in send_request() as well as in receive_other(). i.e. returning > > the object from the previous ForwardRequest if that has been thrown. > > Rather than allow forward_reference to be accessed in send_request I > would suggest something more general, something like: > > // Only allowed in receive_other > void ClientRequestInfo::set_retry_info(Any any) > > // Only allowed in send_request and send_poll > Any ClientRequestInfo::get_retry_info() > > If not set then returns tk_null (like empty slots). Won't this fail if more than one client interceptor needs special forwarding handling? -- Jon Biggar Floorboard Software jon@floorboard.com jon@biggar.org X-Sender: andyp@san-francisco.beasys.com X-Mailer: QUALCOMM Windows Eudora Version 4.3.2 Date: Wed, 01 May 2002 17:35:30 -0700 To: Jonathan Biggar , Harold Carr From: Andy Piper Subject: Re: ForwardRequest is impossible to detect in clients Cc: corba-rtf@omg.org, ec@bea.com At 05:20 PM 5/1/2002 -0700, Jonathan Biggar wrote: Harold Carr wrote: > > PROPOSED RESOLUTION > > > > > 1. add: > > void set_slot (in SlotId id, in any data) raises (InvalidSlot); > > to RequestInfo. This would allow interceptors to transfer information > > between invokes of the same client request and thus a retry could be detected. > > I'm not a TMI expert, but as I recall set_slot is not allowed on > ClientRequestInfo because of Messaging/TMI. Otherwise an interceptor > may set_slots which effect a different request. Hmmm, are you saying that RequestInfo isn't bound only to the particular request and the proposed set_slot could affect some other request as well? > > 2. Add a new function to RequestInfo to indicate that a forward is in > > operation. The minimalist fix here would be to allow forward_reference() to > > be accessed in send_request() as well as in receive_other(). i.e. returning > > the object from the previous ForwardRequest if that has been thrown. > > Rather than allow forward_reference to be accessed in send_request I > would suggest something more general, something like: > > // Only allowed in receive_other > void ClientRequestInfo::set_retry_info(Any any) > > // Only allowed in send_request and send_poll > Any ClientRequestInfo::get_retry_info() > > If not set then returns tk_null (like empty slots). Won't this fail if more than one client interceptor needs special forwarding handling? Right, set_slot() would handle this generically. andy Date: Thu, 02 May 2002 14:52:15 +0000 From: Harold Carr X-Mailer: Mozilla 4.51 [en] (WinNT; U) X-Accept-Language: en To: Andy Piper CC: corba-rtf@omg.org, issues@omg.org, ec@bea.com Subject: Re: ForwardRequest is impossible to detect in clients Hello Andy > >Rather than allow forward_reference to be accessed in send_request I > >would suggest something more general, something like: > > > >// Only allowed in receive_other > >void ClientRequestInfo::set_retry_info(Any any) > > You would have to allow this in send_request() also I believe. It is my > understanding that only other interceptors get receive_other() - at least > thats what happens in the JDK :) Certainly the problem requires a solution > that allows setting of information in send_request(). I take it that you are doing some load-balancing logic in send_request then have that point raise ForwardRequest? And you want to see that ForwardRequest in the subsequent ForwardRequest, right? Before taking the time to design this feedback loop I would point out two details: 1. orbrev/02-02-01 21.3.6.5 says that "for retries, depending on the policies in effect, a new request may or may not follow when a retry has been indicated. I think, to ensure portability of the feedback loop, would need to be more specific. Something like: for retries a new request SHALL follow when a retry has been indicated unless the following policies are in effect: . 2. Why do you even need the feedback mechanism? Each time you enter send_request you do your load balancing logic. If the logic determines a forward request is in order it would raise it. Otherwise not. I can imagine a simple load balancer that knows the reference of the "generic" server. When it saw that generic reference in send_request it would raise forward request to a specific server. The retry entry to send_request would note that the target reference was not the generic one so would not raise forward request again. I can imagine an even simpler load balancer where the interceptor just had a list of specific servers. Each time a request entered send_request it would just round-robin through that list and raise forward request. In this case it would need a feedback mechanism to notify itself that it already picked. It seems that my previous scenario above is almost equivalent and works without feedback by using the "generic" reference marker to compare against. Before proceeding more with this "need feedback" issue I think it would be useful to have a detailed use case. I don't think just saying that "needed for load balancing" is enough. Particularly since #2 above gives a scenario that works with the current specification (modulo #1). CHeers, Harold Date: Thu, 02 May 2002 14:59:11 +0000 From: Harold Carr X-Mailer: Mozilla 4.51 [en] (WinNT; U) X-Accept-Language: en To: Jonathan Biggar CC: Andy Piper , corba-rtf@omg.org, ec@bea.com Subject: Re: ForwardRequest is impossible to detect in clients Hello Jonathan, > > I'm not a TMI expert, but as I recall set_slot is not allowed on > > ClientRequestInfo because of Messaging/TMI. Otherwise an > > interceptor > > may set_slots which effect a different request. > > Hmmm, are you saying that RequestInfo isn't bound only to the > > particular > request and the proposed set_slot could affect some other request as > well? I'm hoping some Messaging/TMI expert will provide the scenario for the set_slot restriction to save us the time to attempt to recreate the scenario. (I've always wished specs would contain rationale so "old" people didn't have to try to page and new people didn't have to reinvent. The C++ ARM and the first edition of the Common Lisp books were great in this regard). > > // Only allowed in receive_other > > void ClientRequestInfo::set_retry_info(Any any) > > > > // Only allowed in send_request and send_poll > > Any ClientRequestInfo::get_retry_info() > > > > If not set then returns tk_null (like empty slots). > > Won't this fail if more than one client interceptor needs special > forwarding handling? Yes, so it would need "slots". Then it would either need a slot allocator or one could say that any slots ids allocated for PICurrent also were valid for retry_info. However, as I said in my previous message, I would prefer to see a detailed use case before trying to design much more. As useful as the feedback mechansim may be it may not be strictly necessary. Cheers, Harold X-Sender: andyp@san-francisco.beasys.com X-Mailer: QUALCOMM Windows Eudora Version 4.3.2 Date: Thu, 02 May 2002 11:56:45 -0700 To: Harold Carr From: Andy Piper Subject: Re: ForwardRequest is impossible to detect in clients Cc: corba-rtf@omg.org, issues@omg.org, ec@bea.com At 02:52 PM 5/2/2002 +0000, Harold Carr wrote: I take it that you are doing some load-balancing logic in send_request then have that point raise ForwardRequest? And you want to see that ForwardRequest in the subsequent ForwardRequest, right? Yes. Before taking the time to design this feedback loop I would point out two details: 1. orbrev/02-02-01 21.3.6.5 says that "for retries, depending on the policies in effect, a new request may or may not follow when a retry has been indicated. I think, to ensure portability of the feedback loop, would need to be more specific. Something like: for retries a new request SHALL follow when a retry has been indicated unless the following policies are in effect: . I'm not convinced. In the case that a retry does NOT happen there is no problem (apart from the fact that the method never gets invoked :) since there is no retry there can be no infinite loop. Certainly when an interceptor throws ForwardRequest the assumption is that a retry will ensue otherwise Forwardrequest has limited value :) So you could be specific about the interceptor only if necessary. 2. Why do you even need the feedback mechanism? Each time you enter send_request you do your load balancing logic. If the logic determines a forward request is in order it would raise it. Otherwise not. I can imagine a simple load balancer that knows the reference of the "generic" server. When it saw that generic reference in send_request it would raise forward request to a specific server. The retry entry to send_request would note that the target reference was not the generic one so would not raise forward request again. So the problem is that once ForwardRequest has been thrown once that object becomes the target (rather than jsut the effective_target). Its true that you could solve this by requiring the target be unchanged by throwing of ForwardRequest. I can imagine an even simpler load balancer where the interceptor just had a list of specific servers. Each time a request entered send_request it would just round-robin through that list and raise forward request. In this case it would need a feedback mechanism to notify itself that it already picked. Right this is the problem. How do you solve the feedback problem? It seems that my previous scenario above is almost equivalent and works without feedback by using the "generic" reference marker to compare against. I think not based on the change to the meaning of target() on subsequent invokes. Before proceeding more with this "need feedback" issue I think it would be useful to have a detailed use case. I don't think just saying that "needed for load balancing" is enough. Particularly since #2 above gives a scenario that works with the current specification (modulo #1). So I disagree. I don't think we have a workable solution yet, but I am more than happy to use the spec as-is if that is possible. andy Date: Thu, 02 May 2002 19:15:16 +0000 From: Harold Carr X-Mailer: Mozilla 4.51 [en] (WinNT; U) X-Accept-Language: en To: Andy Piper CC: corba-rtf@omg.org, issues@omg.org, ec@bea.com Subject: Re: ForwardRequest is impossible to detect in clients Hello Andy, > >1. orbrev/02-02-01 21.3.6.5 says that "for retries, depending on the > >policies in effect, a new request may or may not follow when a retry has > >been indicated. > > > >I think, to ensure portability of the feedback loop, would need to be > >more specific. Something like: for retries a new request SHALL follow > >when a retry has been indicated unless the following policies are in > >effect: . > > I'm not convinced. In the case that a retry does NOT happen there is no > problem (apart from the fact that the method never gets invoked :) since > there is no retry there can be no infinite loop. Certainly when an > interceptor throws ForwardRequest the assumption is that a retry will ensue > otherwise Forwardrequest has limited value :) So you could be specific > about the interceptor only if necessary. I'm not sure what you mean by you're not convinced. All I am suggesting is to precisely specify when a retry does NOT happen so one does not have to ASSUME that it will. They spec will say it SHALL happen unless certain policies are in effect. > > >2. Why do you even need the feedback mechanism? Each time you enter > >send_request you do your load balancing logic. If the logic determines > >a forward request is in order it would raise it. Otherwise not. I can > >imagine a simple load balancer that knows the reference of the "generic" > >server. When it saw that generic reference in send_request it would > >raise forward request to a specific server. The retry entry to > >send_request would note that the target reference was not the generic > >one so would not raise forward request again. > > So the problem is that once ForwardRequest has been thrown once that object > becomes the target (rather than jsut the effective_target). Its true that > you could solve this by requiring the target be unchanged by throwing of > ForwardRequest. I think you have a misunderstanding. The Target DOES NOT CHANGE. The effective_target changes to the object specified in the ForwardRequest. So the solution in #2 should work. > >Before proceeding more with this "need feedback" issue I think it would > >be useful to have a detailed use case. I don't think just saying that > >"needed for load balancing" is enough. Particularly since #2 above > >gives a scenario that works with the current specification (modulo #1). Cheers, Harold X-Sender: andyp@san-francisco.beasys.com X-Mailer: QUALCOMM Windows Eudora Version 4.3.2 Date: Fri, 03 May 2002 11:50:17 -0700 To: Harold Carr From: Andy Piper Subject: Re: ForwardRequest is impossible to detect in clients Cc: corba-rtf@omg.org, ec@bea.com [removed issues from the CC list] At 07:15 PM 5/2/2002 +0000, Harold Carr wrote: > I'm not convinced. In the case that a retry does NOT happen there is no > problem (apart from the fact that the method never gets invoked :) since > there is no retry there can be no infinite loop. Certainly when an > interceptor throws ForwardRequest the assumption is that a retry will ensue > otherwise Forwardrequest has limited value :) So you could be specific > about the interceptor only if necessary. I'm not sure what you mean by you're not convinced. All I am suggesting is to precisely specify when a retry does NOT happen so one does not have to ASSUME that it will. They spec will say it SHALL happen unless certain policies are in effect. I think we are misunderstanding each other :). I am totally happy with the spec saying this (in fact I would like the spec to say this). I just thought this was orthogonal to the issue at hand. > >2. Why do you even need the feedback mechanism? Each time you enter > >send_request you do your load balancing logic. If the logic determines > >a forward request is in order it would raise it. Otherwise not. I can > >imagine a simple load balancer that knows the reference of the "generic" > >server. When it saw that generic reference in send_request it would > >raise forward request to a specific server. The retry entry to > >send_request would note that the target reference was not the generic > >one so would not raise forward request again. > > So the problem is that once ForwardRequest has been thrown once that object > becomes the target (rather than jsut the effective_target). Its true that > you could solve this by requiring the target be unchanged by throwing of > ForwardRequest. I think you have a misunderstanding. The Target DOES NOT CHANGE. The effective_target changes to the object specified in the ForwardRequest. So the solution in #2 should work. Here's what happens for me: request1 send_request() target == effective_target() throw ForwardRequest send_request() target != effective_target() invoke request2 send_request() target != effective_target() OOPS! So sorry, I knew there was a problem but I inverted the description of it :) andy Date: Fri, 03 May 2002 19:18:57 +0000 From: Harold Carr X-Mailer: Mozilla 4.51 [en] (WinNT; U) X-Accept-Language: en To: Andy Piper CC: corba-rtf@omg.org, ec@bea.com Subject: Re: ForwardRequest is impossible to detect in clients Hello Andy, Thanks for providing more detail. I still think it would be useful to completely specify your use case and requirements - we kind of have to guess what you want. > Here's what happens for me: > > request1 > send_request() > target == effective_target() > throw ForwardRequest > send_request() > target != effective_target() > invoke > > request2 > send_request() > target != effective_target() > OOPS! > > So sorry, I knew there was a problem but I inverted the description > of it :) This seems to imply that you want to load-balance on every client invocation, right? That's why I said above a clearly specified use case would be useful - so everyone understands what you are trying to do. If one wants to just load balance each client to a different target then the scenario I described in earlier messages works fine. A solution for load balancing on each client invocation may require a feedback mechanism. Cheers, H X-Sender: andyp@san-francisco.beasys.com X-Mailer: QUALCOMM Windows Eudora Version 4.3.2 Date: Fri, 03 May 2002 13:05:15 -0700 To: Harold Carr From: Andy Piper Subject: Re: ForwardRequest is impossible to detect in clients Cc: corba-rtf@omg.org, ec@bea.com At 07:18 PM 5/3/02 +0000, Harold Carr wrote: Thanks for providing more detail. I still think it would be useful to completely specify your use case and requirements - we kind of have to guess what you want. Ok. > Here's what happens for me: > > request1 > send_request() > target == effective_target() > throw ForwardRequest > send_request() > target != effective_target() > invoke > > request2 > send_request() > target != effective_target() > OOPS! > > So sorry, I knew there was a problem but I inverted the description >of it :) This seems to imply that you want to load-balance on every client invocation, right? That's why I said above a clearly specified use case Yes. would be useful - so everyone understands what you are trying to do. Ok. If one wants to just load balance each client to a different target then the scenario I described in earlier messages works fine. A solution for load balancing on each client invocation may require a feedback mechanism. Right. The simple use case is. A client wishes to support dynamic request-level failover and dynamic per-request load-balancing between a set of servers providing indentical services. This is so that the client can select different targets based on load conditions or other criteria. Scenario 1: A client has a list of targets for a particular object reference. On each request it selects an appropriate target based on criteria that it knows about. It then invokes on that target. Scenario 2: A client using scenario 1 gets a COMM_FAILURE from the selected target. It then selects a different target and retires the request. andy Date: Thu, 9 May 2002 09:56:00 -0700 (PDT) From: Harold Carr To: andyp@bea.com Cc: corba-rtf@omg.org, ec@bea.com Subject: Re: ForwardRequest is impossible to detect in clients Hello Andy, >The simple use case is. > >A client wishes to support dynamic request-level failover and dynamic >per-request load-balancing between a set of servers providing >indentical >services. This is so that the client can select different targets >based on >load conditions or other criteria. > >Scenario 1: > A client has a list of targets for a particular object >reference. >On each request it selects an appropriate target based on criteria >that it > knows about. It then invokes on that target. > >Scenario 2: > A client using scenario 1 gets a COMM_FAILURE from the >selected target. It then selects a different target and >retires the >request. Thanks for providing a little more detail. Although I'm sympathetic to PI providing a "feedback" loop I'm reluctant to add such a loop until an important use case needs it. I discussed your scenario with Ken Cavanaugh and he suggested an alternate solution: Scenario 1: Create a local interface instance which is managed by a POA with a ServantLocator. The ServantLocator would contain the list of real targets. The client would invoke on the local interface instance reference. The ServantLocator would raise ForwardRequest to the appropriate "real" target. Scenario 2: Have a ClientRequestInterceptor::receive_exception point raise a ForwardRequest to the local interface instance. Optional for scenario 2: Have a ClientRequestInterceptor::receive_exception point interact with the local interface instance/ServantLocator infrastructure to notify it of the COMM_FAILURE (to maybe take it out of the list, restart it, or whatever). Bottom line: the use case presented seems to be better handled using the POA rather than PI. And it has the benefit of being available now (no need to wait for a spec change and then updated implementations). Hope that helps, Harold X-Sender: siegel@192.67.184.65 X-Mailer: QUALCOMM Windows Eudora Version 4.3.2 Date: Fri, 10 May 2002 08:24:54 -0400 To: Jonathan Biggar , Harold Carr From: Jon Siegel Subject: Re: ForwardRequest is impossible to detect in clients Cc: andyp@bea.com, corba-rtf@omg.org, ec@bea.com Hi all -- One more thing about Jonathan Biggar's comment 'way down at the bottom of the copied notes (and you should check me out on this, because I haven't done a lot of programming myself lately although I do keep up with the specs): If you add server requirements to the client, you require linking with the larger server libraries and enlarge the client footprint. Some applications use polling-mode retrieval for events, notifications, and asynchronous invocations to avoid this, and thus fit in small devices (and there seem to be more and more types of these, getting more important in the marketplace, as time goes by). Solutions to the forwardrequest problem should preserve the ability to use polling and avoid linking with these larger libraries. (If I've overlooked something and this comment is a little off the wall, just say so :-) Hope this helps -- Jon Siegel At 10:23 AM 5/9/02 -0700, Jonathan Biggar wrote: >Harold Carr wrote: >> Thanks for providing a little more detail. Although I'm sympathetic >> to PI providing a "feedback" loop I'm reluctant to add such a loop >> until an important use case needs it. >> >> I discussed your scenario with Ken Cavanaugh and he suggested an >> alternate solution: >> >> Scenario 1: >> >> Create a local interface instance which is managed by a POA with a >> ServantLocator. The ServantLocator would contain the list of real >> targets. The client would invoke on the local interface instance >> reference. The ServantLocator would raise ForwardRequest to the >> appropriate "real" target. >> >> Scenario 2: >> >> Have a ClientRequestInterceptor::receive_exception point raise a >> ForwardRequest to the local interface instance. >> >> Optional for scenario 2: >> >> Have a ClientRequestInterceptor::receive_exception point interact with >> the local interface instance/ServantLocator infrastructure to notify >> it of the COMM_FAILURE (to maybe take it out of the list, restart it, >> or whatever). >> >> Bottom line: the use case presented seems to be better handled using >> the POA rather than PI. And it has the benefit of being available now >> (no need to wait for a spec change and then updated implementations). > >The downside of this is that you have to make the client into a server >in it's own right, as well as write the code for the proxy interface >implementation. Someone could modify an IDL compiler to generate the >latter, but it takes an extra level of expertise. ================================================================== Dr. Jon Siegel email: siegel@omg.org Vice President, Technology Transfer http://www.omg.org Object Management Group First Needham Place Phone: 781-444-0404 250 First Avenue, Suite 201 Fax: 781-444-0320 Needham, MA 02494 USA ================================================================== X-Sender: andyp@san-francisco.beasys.com X-Mailer: QUALCOMM Windows Eudora Version 4.3.2 Date: Mon, 13 May 2002 05:40:00 -0700 To: Harold Carr From: Andy Piper Subject: Re: ForwardRequest is impossible to detect in clients Cc: corba-rtf@omg.org, ec@bea.com Hi Harold As others have pointed out the use of the server-side programming model is problematic in this instance. Also it seems to me a peculiarly bad fit to the problem at hand - surely PI's were introduced *specifically* to address these sorts of use cases. Also I remain unconvinced that the POA solution would actually work since this is all happening in a J2EE RMI-IIOP environment where the CORBA programming model is completely hidden from the client. It seems to me that I would have to jump through a lot of hoops to get this to work and its not clear that it would be transparent to the client. I don't see that the use case I am presenting is unreasonable and moreover it seems to me that the necessary changes to support it are slight. Should the spec not be serving people's problems rather than the other way round? ;) Thanks andy At 09:56 AM 5/9/2002 -0700, Harold Carr wrote: Hello Andy, >The simple use case is. > >A client wishes to support dynamic request-level failover and dynamic >per-request load-balancing between a set of servers providing >indentical >services. This is so that the client can select different targets >based on >load conditions or other criteria. > >Scenario 1: > A client has a list of targets for a particular object >reference. >On each request it selects an appropriate target based on criteria >that it > knows about. It then invokes on that target. > >Scenario 2: > A client using scenario 1 gets a COMM_FAILURE from the >selected target. It then selects a different target and >retires the >request. Thanks for providing a little more detail. Although I'm sympathetic to PI providing a "feedback" loop I'm reluctant to add such a loop until an important use case needs it. I discussed your scenario with Ken Cavanaugh and he suggested an alternate solution: Scenario 1: Create a local interface instance which is managed by a POA with a ServantLocator. The ServantLocator would contain the list of real targets. The client would invoke on the local interface instance reference. The ServantLocator would raise ForwardRequest to the appropriate "real" target. Scenario 2: Have a ClientRequestInterceptor::receive_exception point raise a ForwardRequest to the local interface instance. Optional for scenario 2: Have a ClientRequestInterceptor::receive_exception point interact with the local interface instance/ServantLocator infrastructure to notify it of the COMM_FAILURE (to maybe take it out of the list, restart it, or whatever). Bottom line: the use case presented seems to be better handled using the POA rather than PI. And it has the benefit of being available now (no need to wait for a spec change and then updated implementations). Hope that helps, Harold X-Sender: andyp@san-francisco.beasys.com X-Mailer: QUALCOMM Windows Eudora Version 4.3.2 Date: Mon, 13 May 2002 05:40:40 -0700 To: Jonathan Biggar , Harold Carr From: Andy Piper Subject: Re: ForwardRequest is impossible to detect in clients Cc: corba-rtf@omg.org, ec@bea.com At 10:23 AM 5/9/2002 -0700, Jonathan Biggar wrote: The downside of this is that you have to make the client into a server in it's own right, as well as write the code for the proxy interface implementation. Someone could modify an IDL compiler to generate the latter, but it takes an extra level of expertise. And this is in RMI-IIOP land which makes it doubly difficult if not impossible. andy Date: Tue, 14 May 2002 16:34:51 +0000 From: Harold Carr X-Mailer: Mozilla 4.51 [en] (WinNT; U) X-Accept-Language: en To: Andy Piper CC: corba-rtf@omg.org, ec@bea.com Subject: Re: ForwardRequest is impossible to detect in clients Hello Andy, > As others have pointed out the use of the server-side programming model is > problematic in this instance. Also it seems to me a peculiarly bad fit to > the problem at hand - surely PI's were introduced *specifically* to address > these sorts of use cases. The PI initial submission and FTF/RTF focused almost exclusively on security and transactions. As I said in a previous mail, at that time I actually pushed for some feedback mechanism at those times and was shot down. Now I find myself in the odd position of being conservative and pushing back against something I had proposed at one time myself. > > Also I remain unconvinced that the POA solution would actually work > since > this is all happening in a J2EE RMI-IIOP environment where the CORBA > programming model is completely hidden from the client. It seems to > me that > I would have to jump through a lot of hoops to get this to work and > its not > clear that it would be transparent to the client. If you used an ORBInitializer to set things up it would be transparent to the client, even in a J2EE RMI-IIOP environment. > > I don't see that the use case I am presenting is unreasonable and > moreover > it seems to me that the necessary changes to support it are > slight. Should > the spec not be serving people's problems rather than the other way > round? ;) I am simply trying to explore alternatives until it is clear there is an important use case the spec does not cover. Plus, I was able to give you a suggestion that would work today - no need to wait for a spec change and implementation updates. Cheers, Harold X-Sender: andyp@san-francisco.beasys.com X-Mailer: QUALCOMM Windows Eudora Version 4.3.2 Date: Wed, 22 May 2002 16:40:39 -0700 To: Harold Carr From: Andy Piper Subject: Re: ForwardRequest is impossible to detect in clients Cc: corba-rtf@omg.org, ec@bea.com Hi Harold, Sorry for the delay in replying - I've been away. At 09:56 AM 5/9/02 -0700, Harold Carr wrote: Thanks for providing a little more detail. Although I'm sympathetic to PI providing a "feedback" loop I'm reluctant to add such a loop until an important use case needs it. I guess I don't see why this isn't an "important" use-case. The inteceptors client API has a clear deficiency for which I have proposed a solution. I would prefer that we discussed the solution on its merits rather than whether my particular use of interceptors is acceptable. The problems that interceptors were originally designed to solve (transactions and CSIv2) could have been easily solved with more specific solutions, however - quite rightly - a general solution to the problem was proposed, one that would have benefits in other problem domains. I do not see how we can justifiably discriminate based on usage when the spec is clearly designed to be generic and used in scenarios perhaps never intended by the spec writers. I discussed your scenario with Ken Cavanaugh and he suggested an alternate solution: Scenario 1: Create a local interface instance which is managed by a POA with a ServantLocator. The ServantLocator would contain the list of real targets. The client would invoke on the local interface instance reference. The ServantLocator would raise ForwardRequest to the appropriate "real" target. The information required is present in service contexts and IOR tagged components. I presume that to get at this information I would have to write an interceptor as well as the local interface / POA. This seems to be a complicated solution whereas I would contend my solution is relatively simple. Bottom line: the use case presented seems to be better handled using the POA rather than PI. And it has the benefit of being available now (no need to wait for a spec change and then updated implementations). I already have a non-portable solution to this problem. I am more concerned that the spec be fixed if it is deficient (which I believe it is) than whether or not I have a solution right now. andy X-Sender: andyp@san-francisco.beasys.com X-Mailer: QUALCOMM Windows Eudora Version 4.3.2 Date: Wed, 22 May 2002 17:17:34 -0700 To: Harold Carr From: Andy Piper Subject: Re: ForwardRequest is impossible to detect in clients Cc: corba-rtf@omg.org, ec@bea.com Hi Harold, [I answered the wrong mail before] At 04:34 PM 5/14/02 +0000, Harold Carr wrote: The PI initial submission and FTF/RTF focused almost exclusively on security and transactions. As I said in a previous mail, at that time I actually pushed for some feedback mechanism at those times and was shot down. Now I find myself in the odd position of being conservative and pushing back against something I had proposed at one time myself. As I said in my previous mail since the spec is generic I don't see why we should constrain ourselves to thinking about transactions and CSIv2. If you used an ORBInitializer to set things up it would be transparent to the client, even in a J2EE RMI-IIOP environment. But I would still need interceptors as well. I am simply trying to explore alternatives until it is clear there is an important use case the spec does not cover. Plus, I was able to give you a suggestion that would work today - no need to wait for a spec change and implementation updates. Thanks for your concern :) As I said in my previous mail I am solely concerned that the spec does not have any holes wrt a useful feature. I already have solutions to my particular implementation problem. andy Date: Tue, 22 Oct 2002 16:01:01 -0400 From: Jishnu Mukerji Organization: Software Global Business Unit, Hewlett-Packard X-Mailer: Mozilla 4.79 [en] (Win98; U) X-Accept-Language: en To: corba-rtf@omg.org Subject: Proposed resolution for 5266 The following resolution will appear in the next vote unless there is sinificant technical objection to it. Jishnu. ____________________________________________________________________ Issue 5266: ForwardRequest is impossible to detect in clients (corba-rtf) Click http://cgi.omg.org/issues/issue5266.txt for this issue's archive. Source: BEA Systems (Dr. Andrew Piper, andyp@bea.com) Nature: Uncategorized Issue Severity: Summary: REQUIREMENT To be able to use interceptors, and in particular ForwardRequest, in a client to perform active, per-request, load-balancing. PROBLEM It is not possible to detect in an interceptor whether ForwardRequest has previously been thrown for the same client request. Thus it is possible for a client to go into an infinite loop throwing ForwardRequest. DISCUSSION The basic problem is that although for a single client request the request-scoped PICurrent is shared across across interceptor invocations - even when ForwardRequest is thrown - it is not possible to *modify* this information in an interceptor to indicate to a future invocation that the invocation has been seen. etc...... Resolution: The extensive discussion in the archive suggests that the significant change proposed here does not quite achieve what was intended and there are other ways of acheiving what was intended without making significant changes to the specification. So close no change. Revised Text: Actions taken: Close no change X-Sender: andyp@san-francisco.beasys.com X-Mailer: QUALCOMM Windows Eudora Version 5.0 Date: Thu, 24 Oct 2002 09:51:41 -0700 To: Jishnu Mukerji , corba-rtf@omg.org From: Andy Piper Subject: Re: Proposed resolution for 5266 At 04:01 PM 10/22/2002 -0400, Jishnu Mukerji wrote: Resolution: The extensive discussion in the archive suggests that the significant change proposed here does not quite achieve what was intended and there are other ways of acheiving what was intended without making significant changes to the specification. So close no change. I'm somewhat less than in agreement with this. I believe the change would achieve what is required. The major problem as I understand it is unspecified conflict with the messaging spec. I would be interested in the opinions of others on this. andy Date: Thu, 24 Oct 2002 13:33:13 -0400 From: Jishnu Mukerji Organization: Software Global Business Unit, Hewlett-Packard X-Mailer: Mozilla 4.79 [en] (Win98; U) X-Accept-Language: en To: Andy Piper Cc: corba-rtf@omg.org Subject: Re: Proposed resolution for 5266 Andy Piper wrote: > > At 04:01 PM 10/22/2002 -0400, Jishnu Mukerji wrote: > >Resolution: The extensive discussion in the archive suggests that the > >significant change proposed here does not quite achieve what was > >intended and there are other ways of acheiving what was intended > >without making significant changes to the specification. So close no > >change. > > I'm somewhat less than in agreement with this. I believe the change would > achieve what is required. The major problem as I understand it is > unspecified conflict with the messaging spec. I would be interested in the > opinions of others on this. > > andy Best way to get a definitive sense of the RTF on this matter is to try to close it no change and see how they vote.:-) There will be many other issues that will have such a test applied to them this time around. Jishnu. Sender: jbiggar@Resonate.com Date: Thu, 24 Oct 2002 10:43:10 -0700 From: Jonathan Biggar X-Mailer: Mozilla 4.8 [en] (X11; U; SunOS 5.8 sun4u) X-Accept-Language: en To: corba-rtf@omg.org Subject: Re: Proposed resolution for 5266 Andy Piper wrote: > > At 04:01 PM 10/22/2002 -0400, Jishnu Mukerji wrote: > >Resolution: The extensive discussion in the archive suggests that the > >significant change proposed here does not quite achieve what was > >intended and there are other ways of acheiving what was intended > >without making significant changes to the specification. So close no > >change. > > I'm somewhat less than in agreement with this. I believe the change would > achieve what is required. The major problem as I understand it is > unspecified conflict with the messaging spec. I would be interested in the > opinions of others on this. I think I'm with Andy. There is an issue to be solved here, and the proposed workarounds have their own problems. -- Jon Biggar Floorboard Software jon@floorboard.com jon@biggar.org Date: Thu, 24 Oct 2002 14:41:07 -0400 From: Jishnu Mukerji Organization: Software Global Business Unit, Hewlett-Packard X-Mailer: Mozilla 4.79 [en] (Win98; U) X-Accept-Language: en To: Jonathan Biggar Cc: corba-rtf@omg.org Subject: Re: Proposed resolution for 5266 Jonathan Biggar wrote: > > Andy Piper wrote: > > > > At 04:01 PM 10/22/2002 -0400, Jishnu Mukerji wrote: > > >Resolution: The extensive discussion in the archive suggests that the > > >significant change proposed here does not quite achieve what was > > >intended and there are other ways of acheiving what was intended > > >without making significant changes to the specification. So close no > > >change. > > > > I'm somewhat less than in agreement with this. I believe the change would > > achieve what is required. The major problem as I understand it is > > unspecified conflict with the messaging spec. I would be interested in the > > opinions of others on this. > > I think I'm with Andy. There is an issue to be solved here, and the > proposed workarounds have their own problems. Well, if a concrete resolution that can be voted on materializes in the next four weeks we will certainly vote on that. I am not planning to put this one up for an NC vote immediately, since someone has shown an interest to presumably come up with a resolution, but I would like to keep some pressure on for making progress. Now for a more general meta-discussion....... What I don't want to see happen to a whole host of issues is for them to go back into the dormant state because no one really cares to spend any real time to come up with a resolution, at least not until someone tries to close them no change, and then there is excitement for 25 minutes and then back to dormant, until the next time. If something is truly important, there should really be willingness to spend more time on it by more than a few people, specially when they are talking of adding significant new features to the specifcation - something that an RTF is suposedly not supposed to do. In those cases with feature enhancement proposals, the very fact that many serious vendors are not participating in the discussion is indicative to me that they would rather not see it be done, and beyond a certain amount of discussion members need to be given a chance to make that known through a vote. Along these lines, I would also like to get a feel for what we should do with all issues that have been open for more than 12 months. I intend to start combing through them and see which ones have a prayer in hell of getting resolved and which ones need to be punted for lack of interest in getting a fix or lack of feasibility of the same. Comments? Thoughts? Jishnu. X-Sender: andyp@san-francisco.beasys.com X-Mailer: QUALCOMM Windows Eudora Version 5.0 Date: Thu, 05 Dec 2002 14:27:37 -0800 To: corba-rtf@omg.org From: Andy Piper Subject: Proposed resolution to 5266 So having looked at all the previous discussions it seem that the second of my two original solutions may be less contentious. I have also tried to capture here some of points raised by others. This is a first second pass ;) to see how it sits with people. Thanks andy This is issue # 5266 Andy Piper ForwardRequest is impossible to detect in clients REQUIREMENT To be able to use interceptors, and in particular ForwardRequest, in a client to perform active, per-request, load-balancing. In general to be able to use ForwardRequest locally more than once in an interceptor. PROBLEM It is not possible to detect in an interceptor whether ForwardRequest has previously been thrown for the same client request. Thus it is possible for a client to go into an infinite loop throwing ForwardRequest. Any solution must work for an arbitrary number of installed interceptors. Any solution should not require servers or server libraries in the client. DISCUSSION The basic problem is that although for a single client request the request-scoped PICurrent is shared across across interceptor invocations - even when ForwardRequest is thrown - it is not possible to *modify* this information in an interceptor to indicate to a future invocation that the invocation has been seen. It is also not possible to use target() and effective_target() to distinguish between the first and subsequent invocations since target() is not replaced with effective_target() for subsequent invokes and therefore it is not possible to distinguish whether the subsequent invoke is the result of ForwardRequest or a totally new request. This problem also occurs in the case of multiple interceptors relying on this behavior. The two relevant parts of the spec here are: 21.3.6.5 For retries, depending on the policies in effect, a new request may or may not follow when a retry has been indicated. If a new request does follow, while this request is a new request, with respect to Interceptors, there is one point of correlation between the original request and the retry: because control has not returned to the client, the request scoped PortableInterceptor::Current for both the original request and the retrying request is the same (see Chapter 21, Portable Interceptors on page 21-32). 21.4.2 Before an invocation is made, PICurrent is obtained via a call to ORB::resolve_initial_references ("PICurrent") From within the interception points, the data on PICurrent that has moved from the thread scope to the request scope is available via the get_slot operation on the RequestInfo object. A PICurrent can still be obtained via resolve_initial_references, but that is the Interceptor's thread scope PICurrent. See section 21.4.4.4, Request Scope vs Thread Scope on page 21-36 for a detailed discussion of the scope of PICurrent. Thus modifications to the thread's PICurrent are lost on retries and modifications to the request's PICurrent are not possible. PROPOSED RESOLUTION I have made several different attempts at coming up with a portable way of solving this problem without changing the spec, but have failed. It seems to me that it really should be possible for the interceptor to know that a retry is in effect and I can think of a number of different solutions to this: 1. add: void set_slot (in SlotId id, in any data) raises (InvalidSlot); to RequestInfo. This would allow interceptors to transfer information between invokes of the same client request and thus a retry could be detected. 2. Add a new function to RequestInfo to indicate that a forward is in operation or not. The minimalist fix here would be to allow forward_reference() to be accessed in send_request() as well as in receive_other(). i.e. returning the object from the previous ForwardRequest if that has been thrown and null otherwise. Since (1) has generated so much controversy I will now try and plump for (2) which requires no interface changes. - Change 21.3.6.5 to: ... for retries, depending on the policies in effect, a new request shall follow when a retry has been indicated unless the following policies are in effect: . - Change 21.3.12.11 to: In the receive_other interception point if the reply_status attribute is LOCATION_FORWARD, then this attribute will contain the object to which the request will be forwarded. In the send_request interception point, if the interceptor is being invoked as the result of a previously raised ForwardRequest, this attribute will contain the object referenced by that previous ForwardRequest. Otherwise it will be null. This is not the same as effective_target since effective_target persists across client invokes, whereas forward_reference does not. For completeness here is what would be required for (1): - Change the IDL in 21.3.12 to include void set_slot (in SlotId id, in any data) raises (InvalidSlot); - After 21.3.12.12 move in the text from 21.3.14.6 - Change 21.3.6.5 to: ... for retries, depending on the policies in effect, a new request shall follow when a retry has been indicated unless the following policies are in effect: . - Change the IDL in 21.3.14 to remove set_slot() Date: Fri, 06 Dec 2002 17:46:50 +0000 From: Harold Carr X-Mailer: Mozilla 4.79 [en] (WinNT; U) X-Accept-Language: en To: Andy Piper CC: corba-rtf@omg.org Subject: Re: Proposed resolution to 5266 Hello Andy, If there was a controversy (from me) it wasn't between the form of retry info (set_slot versus reply_status). It was whether we should do this at all. While the initial spec was being formed I pushed for such a retry feedback mechanism and was shot down by everyone. So when you brought it up again I found myself being in the ironic position of pushing back - mostly to see if anyone would back me up on the pushback and state technical reasons as to why we shouldn't add such a mechanism. I still do not know why no one supported the feedback mechanism in the first place. That said, and since no one else seems to object (other than me the last time you brought this up) I support such a mechanism. However, as long as we are going to the trouble, it seems we should make the feedback mechanism as general as possible. In other words, set_slot(id, any), but with a twist. Whenever an allocate_slot_id is called, it would allocate two "slots" - the existing one (in PICurrent : both RSC and TSC) and a new "RequestInfo" slot that only lives on RequestInfo. RequestInfo::get_request_info_slot(...) Requestinfo::set_request_info_slot(...) These slots are disjoint from the PICurrent slots. One can set and get at any time. I remember set_slot was specifically left off of RequestInfo - but I cannot remember the exact reason - something to do with messaging I think. Perhaps someone else can remember the technical problem. If there is no problem then maybe set_slot that is not disjoint is ok. Cheers, Harold X-Sender: andyp@san-francisco.beasys.com X-Mailer: QUALCOMM Windows Eudora Version 5.0 Date: Fri, 06 Dec 2002 10:43:43 -0800 To: Harold Carr From: Andy Piper Subject: Re: Proposed resolution to 5266 Cc: corba-rtf@omg.org Hi Harold, At 05:46 PM 12/6/2002 +0000, Harold Carr wrote: If there was a controversy (from me) it wasn't between the form of retry info (set_slot versus reply_status). It was whether we should do this at all. While the initial spec was being formed I pushed for such a retry feedback mechanism and was shot down by everyone. So when you brought it up again I found myself being in the ironic position of pushing back - mostly to see if anyone would back me up on the pushback and state technical reasons as to why we shouldn't add such a mechanism. I still do not know why no one supported the feedback mechanism in the first place. Me neither, I would have supported you :) That said, and since no one else seems to object (other than me the last time you brought this up) I support such a mechanism. Great! However, as long as we are going to the trouble, it seems we should make the feedback mechanism as general as possible. In other words, set_slot(id, any), but with a twist. Whenever an allocate_slot_id is called, it would allocate two "slots" - the existing one (in PICurrent : both RSC and TSC) and a new "RequestInfo" slot that only lives on RequestInfo. RequestInfo::get_request_info_slot(...) Requestinfo::set_request_info_slot(...) These slots are disjoint from the PICurrent slots. One can set and get at any time. I remember set_slot was specifically left off of RequestInfo - but I cannot remember the exact reason - something to do with messaging I think. Perhaps someone else can remember the technical problem. If there is no problem then maybe set_slot that is not disjoint is ok. You proposal seems to avoid this entirely so I say go with that. What I'm not sure about is how this affects 21.3.1 (and others). The interceptor spec goes to great pains to indicate that interceptors run independently. Your proposal opens up the possibility of cross-interceptor communication even though we really only require that an interceptor communicate with itself across invocations. Do you think that the former should be: a) disallowed. b) allowed but with no guarantees about ordering. c) implied but not required? Also since the original set_slot() sets the RSC, I'm not totally sure how this is different to the slot in RequestInfo since RequestInfo is implicitly request scoped. I'm worried that if there is a problem with set_slot() then there may also be a problem with set_request_info_slot() since the scope is the same. I'm also wondering whether the root problem (that no-one can remember) would be do to with the visibility of the results of set_slot between interceptors. It may be that we can avoid this be toning down the language of set_slot() for client interceptors by plumping for (c) above (i.e. the only requirement is that invocations of the same interceptor see the results of set_slot - this would still allow interceptors to run in different threads). Thoughts? andy X-Sender: andyp@san-francisco.beasys.com X-Mailer: QUALCOMM Windows Eudora Version 5.0 Date: Thu, 12 Dec 2002 16:10:28 -0800 To: corba-rtf@omg.org From: Andy Piper Subject: Proposed resolution to 5266 After further discussions with Harold here is what I have come up with. I would like to get a sense of the RTF in the next vote. andy This is issue # 5266 Andy Piper ForwardRequest is impossible to detect in clients REQUIREMENT To be able to use interceptors, and in particular ForwardRequest, in a client to perform active, per-request, load-balancing. In general to be able to use ForwardRequest locally more than once in an interceptor. PROBLEM It is not possible to detect in an interceptor whether ForwardRequest has previously been thrown for the same client request. Thus it is possible for a client to go into an infinite loop throwing ForwardRequest. Any solution must work for an arbitrary number of installed interceptors. Any solution should not require servers or server libraries in the client. DISCUSSION The basic problem is that although for a single client request the request-scoped PICurrent is shared across across interceptor invocations - even when ForwardRequest is thrown - it is not possible to *modify* this information in an interceptor to indicate to a future invocation that the invocation has been seen. It is also not possible to use target() and effective_target() to distinguish between the first and subsequent invocations since target() is not replaced with effective_target() for subsequent invokes and therefore it is not possible to distinguish whether the subsequent invoke is the result of ForwardRequest or a totally new request. This problem also occurs in the case of multiple interceptors relying on this behavior. The two relevant parts of the spec here are: 21.3.6.5 For retries, depending on the policies in effect, a new request may or may not follow when a retry has been indicated. If a new request does follow, while this request is a new request, with respect to Interceptors, there is one point of correlation between the original request and the retry: because control has not returned to the client, the request scoped PortableInterceptor::Current for both the original request and the retrying request is the same (see Chapter 21, Portable Interceptors on page 21-32). 21.4.2 Before an invocation is made, PICurrent is obtained via a call to ORB::resolve_initial_references ("PICurrent") From within the interception points, the data on PICurrent that has moved from the thread scope to the request scope is available via the get_slot operation on the RequestInfo object. A PICurrent can still be obtained via resolve_initial_references, but that is the Interceptor's thread scope PICurrent. See section 21.4.4.4, Request Scope vs Thread Scope on page 21-36 for a detailed discussion of the scope of PICurrent. Thus modifications to the thread's PICurrent are lost on retries and modifications to the request's PICurrent are not possible. POSSIBLE RESOLUTIONS I have made several different attempts at coming up with a portable way of solving this problem without changing the spec, but have failed. It seems to me that it really should be possible for the interceptor to know that a retry is in effect and I can think of a number of different solutions to this: 1. add: void set_slot (in SlotId id, in any data) raises (InvalidSlot); to RequestInfo. This would allow interceptors to transfer information between invokes of the same client request and thus a retry could be detected. 2. Add a new function to RequestInfo to indicate that a forward is in operation or not. The minimalist fix here would be to allow forward_reference() to be accessed in send_request() as well as in receive_other(). i.e. returning the object from the previous ForwardRequest if that has been thrown and null otherwise. 3. Add new functions to RequestInfo: any get_request_info_slot(in SlotId id) raises (InvalidSlot); void set_request_info_slot(in SlotId id, in any data) raises (InvalidSlot); This is the same as (1) but does not have any implications for PICurrent. Having discussed this some more with Harold, it seems (3) follows the path of least resistance. PROPOSED RESOLUTION - Change 21.3.6.5 to: ... for retries, depending on the policies in effect, a new request shall follow when a retry has been indicated unless the following policies are in effect: . - Change the IDL in 21.3.12 to include: any get_request_info_slot(in SlotId id) raises (InvalidSlot); void set_request_info_slot(in SlotId id, in any data) raises (InvalidSlot); - add 21.3.12.15 and 21.3.12.16: 21.3.12.15 get_request_info_slot This operation returns the data from the given slot of the RequestInfo that is in the scope of the current request. This allows interceptors to communicate with each other about the state of a request. Since interceptor invocations within a request are serialized but unordered, an interceptor should generally only communicate with itself. If the given slot has not been set during the current request, then an any containing a type code with a TCKind value of tk_null is returned. If the ID does not define an allocated slot, InvalidSlot is raised. Parameters id The SlotId of the slot that is to be returned. Return Value The slot data, in the form of an any, obtained with the given identifier. 21.3.12.16 set_request_info_slot This operation allows an Interceptor to set a slot in the RequestInfo that is in the scope of the current request. If data already exists in that slot, it will be overwritten. If the ID does not define an allocated slot, InvalidSlot is raised. Parameters id The SlotId of the slot. data The data, in the form of an any, to store in that slot. - Change the table 21-1 in 21.3.13 to include: get_request_info_slot yes yes yes yes yes set_request_info_slot yes yes yes yes yes - Change the table 21-2 in 21.3.14 to include: get_request_info_slot yes yes yes yes yes set_request_info_slot yes yes yes yes yes - add a new section 21.4.4.x (insert after 21.4.4.1 possibly): 21.4.4.2 RequestInfo slots In order to accommodate communication between interceptors independent of PICurrent, allocate_slot_id will also allocate a RequestInfo slot. This can be manipulated via set_request_info_slot and get_request_info_slot. These functions have no affect on the slots in the TSC. Sender: jon@floorboard.com Date: Thu, 12 Dec 2002 18:03:23 -0800 From: Jonathan Biggar X-Mailer: Mozilla 4.8 [en] (X11; U; SunOS 5.7 sun4u) X-Accept-Language: en To: Andy Piper CC: corba-rtf@omg.org Subject: Re: Proposed resolution to 5266 Andy, There doesn't appear to be any text here that states that the contents of the request_info_slot table is carried over from an initial request to a retry. Is this the intent? -- Jon Biggar Floorboard Software jon@floorboard.com jon@biggar.org Andy Piper wrote: > PROPOSED RESOLUTION > > - Change 21.3.6.5 to: > > ... for retries, depending on the policies in effect, a new request > shall follow when a retry has been indicated unless the following > policies are in effect: . > > - Change the IDL in 21.3.12 to include: > > any get_request_info_slot(in SlotId id) raises (InvalidSlot); > void set_request_info_slot(in SlotId id, in any data) raises (InvalidSlot); > > - add 21.3.12.15 and 21.3.12.16: > > 21.3.12.15 get_request_info_slot > > This operation returns the data from the given slot of the RequestInfo > that is in the scope of the current request. This allows interceptors > to communicate with each other about the state of a request. Since > interceptor invocations within a request are serialized but unordered, > an interceptor should generally only communicate with itself. > > If the given slot has not been set during the current request, then an > any containing a type code with a TCKind value of tk_null is returned. > > If the ID does not define an allocated slot, InvalidSlot is raised. > > Parameters > > id The SlotId of the slot that is to be returned. > > Return Value The slot data, in the form of an any, obtained with the > given identifier. > > 21.3.12.16 set_request_info_slot > > This operation allows an Interceptor to set a slot in the RequestInfo > that is in the scope of the current request. If data already exists in > that slot, it will be overwritten. > > If the ID does not define an allocated slot, InvalidSlot is raised. > > Parameters > > id The SlotId of the slot. > > data The data, in the form of an any, to store in that slot. > > - Change the table 21-1 in 21.3.13 to include: > > get_request_info_slot yes yes yes yes yes > set_request_info_slot yes yes yes yes yes > > - Change the table 21-2 in 21.3.14 to include: > > get_request_info_slot yes yes yes yes yes > set_request_info_slot yes yes yes yes yes > > - add a new section 21.4.4.x (insert after 21.4.4.1 possibly): > > 21.4.4.2 RequestInfo slots > > In order to accommodate communication between interceptors independent > of PICurrent, allocate_slot_id will also allocate a RequestInfo > slot. This can be manipulated via set_request_info_slot and > get_request_info_slot. These functions have no affect on the slots in > the TSC. X-Sender: andyp@san-francisco.beasys.com X-Mailer: QUALCOMM Windows Eudora Version 5.0 Date: Thu, 12 Dec 2002 18:37:29 -0800 To: Jonathan Biggar From: Andy Piper Subject: Re: Proposed resolution to 5266 Cc: corba-rtf@omg.org At 06:03 PM 12/12/2002 -0800, Jonathan Biggar wrote: There doesn't appear to be any text here that states that the contents of the request_info_slot table is carried over from an initial request to a retry. Is this the intent? Not by design. I think I was using the term "request" to be a client request rather than remote invoke. What would you suggest? My other doubt is whether this should just be on ClientRequestInfo rather than RequestInfo (since ServerRequestInfo has set_slot). andy