Issue 1409: Multiple threads calling destroy() once destroy() operation has begun (port-rtf) Source: (, ) Nature: Uncategorized Issue Severity: Summary: Summary: There is also a problem with multiple threads calling destroy() once a destroy() operation has begun. This is particularly important when they set wait_for_completion to true, or if they call destroy with a different etherealize_objects value. My current thought on this is that the first thread which calls destroy determines the etherealization policy for the destroy. If subsequent threads call destroy() while destruction is in progress they will wait for completion if wait_for_completion is true, otherwise they return immediately, and let the intial thread which called destroy() complete the destruction. Resolution: Revised Text: Actions taken: June 1, 1998: received issue February 23, 1999: closed issue Discussion: End of Annotations:===== Return-Path: Sender: gscott@inprise.com Date: Fri, 29 May 1998 11:45:09 -0700 From: "George M. Scott" Organization: Borland International, Inc. To: issues@omg.org CC: port-rtf@omg.org Subject: POA lifecycle issues in MT environments POA lifecycle (creation and destruction) as defined currently has lots of potential problem in multi-threaded environments. I have outlined a couple of the problems below. We believe the semantics should be clarified so POA-based servers will be portable in MT environments. There is also a problem with multiple threads calling destroy() once a destroy() operation has begun. This is particularly important when they set wait_for_completion to true, or if they call destroy with a different etherealize_objects value. My current thought on this is that the first thread which calls destroy determines the etherealization policy for the destroy. If subsequent threads call destroy() while destruction is in progress they will wait for completion if wait_for_completion is true, otherwise they return immediately, and let the intial thread which called destroy() complete the destruction. Return-Path: Sender: "George Scott" Date: Wed, 01 Jul 1998 19:01:35 -0700 From: "George M. Scott" Organization: Inprise Corporation To: port-rtf@omg.org Subject: Proposed resolutions for Issues 1407-1410, 1428 The following are the proposals to address issues 1407-1410, and 1428. The proposals are based on our discussions from the previous POA conference call, and should hopefully be acceptable. I'm attempting to follow Dan's format for consistency. George Issue 1408: POA destory() is ill-defined Issue 1409: Multiple threads calling destroy() once destroy() has begun Nature: Revision Summary: POA destroy is not defined sufficiently enough to prevent multiple activations of the same POA name in the same process. POA::destory semantics are not defined for multiple threads calling destroy. Resolution: Accepted for Corba RTF 2.3 Revision: The following text will replace the current describing the destroy operation in Section 9.3.8, Page 9-31. This operation destroys the POA and all descendant POAs. Descendant POAs are destroyed (recursively) before the destruction of the containing POA. The POA so destroyed (that is, the POA with its name) may be re-created later in the same process. (This differs from the POAManager::deactivate operation that does not allow a re-creation of its associated POA in the same process. When a POA is destroyed, any requests that have started execution continue to completion. Any requests that have not started execution are processed as if the POA were in the holding state, until execution of all active requests has completed. Once all active requests have completed, all queued requests (if any) will behave as if they were newly arrived, that is, the POA will attempt to cause recreation of the POA by invoking one or more adapter activators. If the etherealize_objects parameter is TRUE, the POA has the RETAIN policy, and a servant manager is registered with the POA, the etherealize operation on the servant manager will be called for each active object in the Active Object Map. Etherealization will not occur until all active requests have completed execution. The apparent destruction of the POA occurs before any calls to etherealize are made. Thus, for example, an etherealize method that attempts to invoke operations on the POA will receive the OBJECT_NOT_EXIST exception. {editorial note - below I have merged text from these two issues and issue 1428 because they change the same text. If one of these resolutions fails, we will need to edit the following text) The wait_for_completion parameter is handled as follows: - If wait_for_completion is TRUE and the current thread is not in a POA-dispatched invocation context, the destroy operation will return only after all active requests have completed and all invocations of etherealize have completed. - If wait_for_completion is TRUE and the current thread is in a POA-dispatched invocation context, then the BAD_INV_ORDER exception is thrown and POA destruction does not occur. - If wait_for_completion is FALSE, the destroy operation destroys all POAs but does not wait for active requests to complete nor for etherealization to occur. If destroy is called multiple times before destruction is complete (because there are active requests), the etherealize_objects parameter will only apply to the first call of destroy, subsequent calls with conflicting etherealize_objects settings will use the value of the etherealize_objects from the first call. The wait_for_completion parameter will be handled as defined above for each individual call (some callers may choose to block, while others may not). Return-Path: Date: Mon, 06 Jul 1998 13:55:46 -0400 From: Paul H Kyzivat Organization: NobleNet To: "George M. Scott" CC: port-rtf@omg.org Subject: Re: Proposed resolutions for Issues 1407-1410, 1428 References: <359AE9FF.441D15F7@inprise.com> George M. Scott wrote: > > The following are the proposals to address issues 1407-1410, > and 1428. The proposals are based on our discussions from > the previous POA conference call, and should hopefully be > acceptable. > > I'm attempting to follow Dan's format for consistency. > > George > > ----------------------------------------------------------- > Issue 1407: Interaction of find_POA(), AdapterActivators and > create_POA() > Nature: Clarification > > Summary: There is a race condition in the POA when an > AdapterActivator is invoked as a result of a > call to find_POA() or an over-the-wire request, and > simultaneously another thread calls create_POA() > attempting to create the same POA manually. > > Resolution: Accepted for Corba RTF 2.3 > > Revision: Section 9.3.3, Page 9-20. After first paragraph > in page beginning with "if unknown_adapter raises..." > add the following Note: > > Note - It is possible for another thread to create the same > POA the AdapterActivator is being asked to create if > AdapterActivators are used in conjunction with other threads > calling create_POA with the same POA name. To avoid potential > race conditions, it is recommended that AdapterActivators and > manual creation of POAs (via the create_POA call) not be used > in conjunction to create a particular POA. > > ------------------------------------------------------------- > Issue 1408: POA destory() is ill-defined > Issue 1409: Multiple threads calling destroy() once destroy() > has begun > Nature: Revision > > Summary: POA destroy is not defined sufficiently enough to > prevent multiple activations of the same POA name in > the same process. POA::destory semantics are not defined > for multiple threads calling destroy. > > Resolution: Accepted for Corba RTF 2.3 > > Revision: The following text will replace the current describing > the destroy operation in Section 9.3.8, Page 9-31. > > This operation destroys the POA and all descendant POAs. > Descendant POAs are destroyed (recursively) before the destruction > of the containing POA. The POA so destroyed (that is, the POA > with its name) may be re-created later in the same process. > (This differs from the POAManager::deactivate operation that does > not allow a re-creation of its associated POA in the same process. > > When a POA is destroyed, any requests that have started execution > continue to completion. Any requests that have not started > execution are processed as if the POA were in the holding state, > until execution of all active requests has completed. Once > all active requests have completed, all queued requests (if > any) will behave as if they were newly arrived, that is, the POA > will attempt to cause recreation of the POA by invoking one or > more adapter activators. > > If the etherealize_objects parameter is TRUE, the POA has the > RETAIN policy, and a servant manager is registered with the POA, > the etherealize operation on the servant manager will be called > for each active object in the Active Object Map. Etherealization > will not occur until all active requests have completed execution. > The apparent destruction of the POA occurs before any calls > to etherealize are made. Thus, for example, an etherealize > method that attempts to invoke operations on the POA will > receive the OBJECT_NOT_EXIST exception. > > {editorial note - below I have merged text from these two issues > and issue 1428 because they change the same text. If > one of these resolutions fails, we will need to edit the following > text) > > The wait_for_completion parameter is handled as follows: > > - If wait_for_completion is TRUE and the current > thread is not in a POA-dispatched invocation context, the > destroy operation will return only after all active requests > have completed and all invocations of etherealize have > completed. > > - If wait_for_completion is TRUE and the current thread is in a > POA-dispatched invocation context, then the BAD_INV_ORDER > exception is thrown and POA destruction does not occur. > > - If wait_for_completion is FALSE, the destroy operation > destroys all POAs but does not wait for active requests to > complete nor for etherealization to occur. > > If destroy is called multiple times before destruction is > complete (because there are active requests), the > etherealize_objects parameter will only apply to the first > call of destroy, subsequent calls with conflicting > etherealize_objects settings will use the value of the > etherealize_objects from the first call. The wait_for_completion > parameter will be handled as defined above for each individual > call (some callers may choose to block, while others may not). > This seems reasonable. > -------------------------------------------------------------- > Issue 1410: Changing default servant, etc. after POA activation > Nature: Revision > > Summary: Changing default servants, servant managers, etc. > after POA activation can be problematic. After some > discussion it was agreed that this is only truly > problematic for ServantLocators which need to have > matching preinvoke/postinvoke operations. Failing > to do so could result in memory leaks (due to the > Cookie parameter) or other strange side effects, > including inproper transaction management. > > Resolution: Accepted for Corba 2.3 RTF > > Revision: There are two current proposed revisions, which are > outlined below. Inprise will vote for proposal (a) > over proposal (b). > > (a) > Change the second paragraph describing the set_servant_manager > operation on page 9-33, Section 9.3.8 to the following: > > "This operation sets the default servant manager associated > with the POA. This operation may only be invoked once after > a POA has been created. Attempting to set the servant manager > after one has already been set will result in the BAD_INV_ORDER > exception being thrown." I find this overly restrictive. > > (b) > Add a third paragraph to the description of set_servant_manager > on page 9-33, Section 9.3.8: > > "Changing a ServantLocator will result in newly received requests > calling preinvoke and postinvoke on the new ServantLocator. > However, currently active requests will call postinvoke on the > ServantLocator on which they called preinvoke, which may not be > the same as the current ServantLocator." I also find this overly restrictive. It imposes a particular implementation discipline to cover an obscure case that will rarely arise in practice and which a developer can work around if it does arise. I would assert: - in most cases there will be no need to change the locator, - when it is necessary, the developer will probably know that the change is safe, - when the developer doesn't know it is safe, the POAManager state can be set to holding before making the change to guarantee safety. So, I don't think any change is needed, except for some words of warning, replacing the ones above: "If a ServantLocator is replaced while operations are outstanding, then the old ServantLocator may be called for preinvoke and the replacement for postinvoke of the same request. If this is unacceptable, then the POA's POAManager may be placed in the HOLDING state prior to making the change." > > ---------------------------------------------------------------- > Issue 1428: Blocking POA Operations > Nature: Revision > > Summary: Several operations added to CORBA as part of the > Portability submission provide blocking behavior which > can result in deadlock in a large number of cases. > These calls include POA::destroy, ORB::shutdown, > POAManager::deactivate, POAManager::hold_requests, > POAManager::discard_requests. > > Resolution: Accepted for Corba 2.3 RTF > > Revision: The following changes are proposed: > > Replace the second sentence in the paragraph of section 4.9.4, > page 4-20, which begins with "If the wait_for_completion ..." > with the following: > > "If the wait_for_completion parameter is TRUE and the current > thread is not in an ORB-dispatched invocation context, this > operation blocks until all ORB processing (including request > processing and object deactivation or other operations associated > with object adapters) has completed. If the wait_for_completion > parameter is TRUE and the current thread is in an ORB-dispatched > invocation context, then the BAD_INV_ORDER exception is thrown." > > Replace the phrase "If the parameter is TRUE" in the 2nd > paragraph, 2nd sentence of the hold_requests description in > Section 9.3.2, page 9-18, with the phrase "If the parameter > is TRUE and the current thread is not in a POA-dispatched > invocation context". Add the sentenence "If the parameter > is TRUE and the current thread is in a POA-dispatched invocation > context then the BAD_INV_ORDER exception is raised and the > state is not changed." > > Replace the phrase "If the parameter is TRUE" in the 2nd > paragraph, 2nd sentence of the discard_requests description in > Section 9.3.2, page 9-18, with the phrase "If the parameter > is TRUE and the current thread is not in a POA-dispatched > invocation context". Add the sentenence "If the parameter > is TRUE and the current thread is in a POA-dispatched invocation > context then the BAD_INV_ORDER exception is raised and the > state is not changed." to the end of the paragraph. > > Replace the phrase "If the parameter is TRUE" in the 3rd > paragraph, 2nd sentence of the deactivate description in > Section 9.3.2, page 9-18, with the phrase "If the parameter > is TRUE and the current thread is not in a POA-dispatched > invocation context". Add the sentenence "If the parameter > is TRUE and the current thread is in a POA-dispatched invocation > context then the BAD_INV_ORDER exception is raised and the > state is not changed." to the end of the paragraph. > > (editorial note- I changed the text for POA::destory in the > resolution of issues 1408 and 1409 above, so it is not shown > here but a similar change to the above is necessary. > I also felt that 1409 raised an issue which also affects > POAManager::deactivate, so the following change is also > proposed) > > Add the following paragraph to the end of the description of > deactivate in section 9.3.2, page 9-19. > > "If deactivate is called multiple times before destruction is > complete (because there are active requests), the > etherealize_objects parameter will only apply to the first > call of deactivate, subsequent calls with conflicting > etherealize_objects settings will use the value of the > etherealize_objects from the first call. The wait_for_completion > parameter will be handled as defined above for each individual > call (some callers may choose to block, while others may not)." This seems reasonable Return-Path: Date: Wed, 08 Jul 1998 00:57:46 -0700 From: "Jon Goldberg" To: port-rtf@omg.org Subject: Re: Proposed resolutions for Issues 1407-1410, 1428 References: <359AE9FF.441D15F7@inprise.com> Hi Folks- I've modified and am now resubmitting George's original proposal for issues 1408-9, 1428 to take into account the conference call feedback from July 7 (tuesday). Based on my notes, 1428 actually was not changed from the conference call but I've included it in this message since they are all inter-dependent. My assessment of the changes is: 1. description of destroy changed to explicitly talk about setting the POA and its descendents as if they have already been destroyed prior to etherealization beginning for any active objects of any of those POAs. This ensures that all of the POAs pending destruction will reject/queue (see #3 below) requests before any etherealization is started. In the initial proposal, each child POA was independently destroyed before destruction of the parent POA. This allows a possibility of infinite looping in the destruction if etherealization for two child POAs cause each other to be re-created on each POA destruction. 2. the current description compares the POA's temporary behavior as if it is in the "holding state". We agreed to change this terminology to describe it as if the POA's POAManager(s) are in a state, since POAs themselves do not have state. 3. We also concluded that the state in question should be "inactive", not "holding". The implication of this change is that for the duration of the destruction (which may include etherealization), any requests for that POA will now result in an exception. Once the destruction is complete, any new requests might cause the POA to be recreated. Although this avoids deadlocks where etherealization attempts to access another POA which is pending destruction, it means clients will see inconsistent results depending on when destruction occurs for a POA. If the state is "holding", these requests will be queued and the client will always see a consistent result, although the request may take longer to complete depending on when POA destruction occurs. In addition, with change #1 above, if the state is "holding" deadlocks can occur if one POA's etherealization depends on another POA being destructed. I'm stating this implication here to make sure that the RTF voters are all aware of the issue. 4. We decided to mandate bottom-up etherealization. This means that a POA always attempts etherialization on its descendant POAs before proceeding with etherealization of its own active objects. take care, Jon ---------------------------------------------------------------- Issue 1408: POA destroy() is ill-defined Issue 1409: Multiple threads calling destroy() once destroy() has begun Nature: Revision Summary: POA destroy is not defined sufficiently enough to prevent multiple activations of the same POA name in the same process. POA::destory semantics are not defined for multiple threads calling destroy. Resolution: Accepted for Corba RTF 2.3 Revision: The following text will replace the current describing the destroy operation in Section 9.3.8, Page 9-31. This operation destroys the POA and all descendant POAs. The POA so destroyed (that is, the POA with its name) may be re-created later in the same process. (This differs from the POAManager::deactivate operation that does not allow a re-creation of its associated POA in the same process. After a deactivate, re-creation is allowed only if the POA is later destroyed.) When a POA is destroyed, all descendent POAs are also destroyed. During destruction, the POA and all its descendant POAs behave as if their POAManagers are in the inactive state. Note that these POAManagers are not actually placed in the inactive state. It is only the POAs pending destruction that behave in this manner, summarized as follows: * Any requests that that have started execution continue to completion. * Any requests that have not started execution, or that arrive while POA destruction is in progress, are rejected as described in "Inactive State" within section 9.3.2, "POAManager Interface". * Once destruction is complete, for any new requests (if any) that arrive the POA will attempt to cause recreation of the POA by invoking one or more adapter activators. If the etherealize_objects parameter is TRUE, the POA has the RETAIN policy, and a servant manager is registered with the POA, the etherealize operation on the servant manager will be called for each active object in the Active Object Map. Etherealization will not occur until all active requests have completed execution. The apparent destruction of the POA occurs before any calls to etherealize are made. Thus, for example, an etherealize method that attempts to invoke operations on the POA (or any of its descendant POAs) will receive the OBJECT_NOT_EXIST exception. Destruction of a POA always attempts etherialization on its descendant POAs before proceeding with etherealization of its own active objects. {George Scott original editorial note - below I have merged text from these two issues and issue 1428 because they change the same text. If one of these resolutions fails, we will need to edit the following text) The wait_for_completion parameter is handled as follows: - If wait_for_completion is TRUE and the current thread is not in a POA-dispatched invocation context, the destroy operation will return only after all active requests have completed and all invocations of etherealize have completed. - If wait_for_completion is TRUE and the current thread is in a POA-dispatched invocation context, then the BAD_INV_ORDER exception is thrown and POA destruction does not occur. - If wait_for_completion is FALSE, the destroy operation destroys all POAs but does not wait for active requests to complete nor for etherealization to occur. If destroy is called multiple times before destruction is complete (because there are active requests), the etherealize_objects parameter will only apply to the first call of destroy, subsequent calls with conflicting etherealize_objects settings will use the value of the etherealize_objects from the first call. The wait_for_completion parameter will be handled as defined above for each individual call (some callers may choose to block, while others may not). Return-Path: Sender: jon@floorboard.com Date: Wed, 08 Jul 1998 08:13:06 -0700 From: Jonathan Biggar To: Jon Goldberg CC: port-rtf@omg.org Subject: Re: Proposed resolutions for Issues 1407-1410, 1428 References: <359AE9FF.441D15F7@inprise.com> <35A3267A.181DCAD8@inprise.com> Jon Goldberg wrote: > > Hi Folks- > > I've modified and am now resubmitting George's original proposal for > issues 1408-9, 1428 to take into account the conference call > feedback > from July 7 (tuesday). Based on my notes, 1428 actually was not > changed from the conference call but I've included it in this > message > since they are all inter-dependent. My assessment of the changes > is: > > 1. description of destroy changed to explicitly talk about setting > the POA and its descendents as if they have already been destroyed > prior to etherealization beginning for any active objects of any > of those POAs. This ensures that all of the POAs pending > destruction > will reject/queue (see #3 below) requests before any etherealization > is started. In the initial proposal, each child POA was > independently > destroyed before destruction of the parent POA. This allows a > possibility of infinite looping in the destruction if > etherealization > for two child POAs cause each other to be re-created on each > POA destruction. > > 2. the current description compares the POA's temporary behavior as > if > it is in the "holding state". We agreed to change this terminology > to > describe it as if the POA's POAManager(s) are in a state, since POAs > themselves do not have state. > > 3. We also concluded that the state in question should be > "inactive", > not "holding". The implication of this change is that for the > duration of the destruction (which may include etherealization), any > requests for that POA will now result in an exception. Once the > destruction is complete, any new requests might cause the POA to be > recreated. Although this avoids deadlocks where etherealization > attempts to access another POA which is pending destruction, it > means > clients will see inconsistent results depending on when destruction > occurs for a POA. If the state is "holding", these requests will be > queued and the client will always see a consistent result, although > the request may take longer to complete depending on when POA > destruction occurs. In addition, with change #1 above, if the state > is > "holding" deadlocks can occur if one POA's etherealization depends > on > another POA being destructed. I'm stating this implication here to > make sure that the RTF voters are all aware of the issue. I don't like this much because it does cause visibility of the destroy process to the clients, particularly if the destroy comes from a call to ORB::shutdown(). However, I don't see a good way around this. Perhaps it would be a good idea to also state that when ORB::shutdown() is called that the input side of the protocols (GIOP) are blocked before the POAs are destroyed in order to avoid client interaction while the shutdown is processing? Then the process would look like this: 1. Someone calls ORB::shutdown(). 2. All server side protocol connections are blocked from processing further input. 3. All pending requests are drained from the ORB. 4. Once the last outstanding reply is sent on a server connection, the connection is shutdown (via CloseConnection if GIOP). 4. All POAs are destroyed. 5. Any other ORB level cleanup is done. Of course this could be left as a quality of ORB implementation issue, but that is how I would implement this. > 4. We decided to mandate bottom-up etherealization. This means that a > POA always attempts etherialization on its descendant POAs before > proceeding with etherealization of its own active objects. Seems reasonable, since it is likely that child POAs are using servant managers or adapter activators that are servants of parent POAs, and this avoids race conditions for those. -- Jon Biggar Floorboard Software jon@floorboard.com jon@biggar.org Return-Path: Sender: "Jon Goldberg" Date: Wed, 08 Jul 1998 10:56:26 -0700 From: Jon Goldberg To: Jonathan Biggar CC: port-rtf@omg.org Subject: Re: Proposed resolutions for Issues 1407-1410, 1428 References: <359AE9FF.441D15F7@inprise.com> <35A3267A.181DCAD8@inprise.com> <35A38C82.5C41F600@floorboard.com> Jonathan Biggar wrote: > > Jon Goldberg wrote: > > > > Hi Folks- > > > > I've modified and am now resubmitting George's original proposal > for > > issues 1408-9, 1428 to take into account the conference call > feedback > > from July 7 (tuesday). Based on my notes, 1428 actually was not > > changed from the conference call but I've included it in this > message > > since they are all inter-dependent. My assessment of the changes > > is: > > > > 1. description of destroy changed to explicitly talk about setting > > the POA and its descendents as if they have already been destroyed > > prior to etherealization beginning for any active objects of any > > of those POAs. This ensures that all of the POAs pending > destruction > > will reject/queue (see #3 below) requests before any > etherealization > > is started. In the initial proposal, each child POA was > independently > > destroyed before destruction of the parent POA. This allows a > > possibility of infinite looping in the destruction if > etherealization > > for two child POAs cause each other to be re-created on each > > POA destruction. > > > > 2. the current description compares the POA's temporary behavior > as if > > it is in the "holding state". We agreed to change this > terminology to > > describe it as if the POA's POAManager(s) are in a state, since > POAs > > themselves do not have state. > > > > 3. We also concluded that the state in question should be > "inactive", > > not "holding". The implication of this change is that for the > > duration of the destruction (which may include etherealization), > any > > requests for that POA will now result in an exception. Once the > > destruction is complete, any new requests might cause the POA to > be > > recreated. Although this avoids deadlocks where etherealization > > attempts to access another POA which is pending destruction, it > means > > clients will see inconsistent results depending on when > destruction > > occurs for a POA. If the state is "holding", these requests will > be > > queued and the client will always see a consistent result, > although > > the request may take longer to complete depending on when POA > > destruction occurs. In addition, with change #1 above, if the > state is > > "holding" deadlocks can occur if one POA's etherealization depends > on > > another POA being destructed. I'm stating this implication here > to > > make sure that the RTF voters are all aware of the issue. > > I don't like this much because it does cause visibility of the > destroy > process to the clients, particularly if the destroy comes from a > call to > ORB::shutdown(). However, I don't see a good way around this. > Perhaps > it would be a good idea to also state that when ORB::shutdown() is > called that the input side of the protocols (GIOP) are blocked > before > the POAs are destroyed in order to avoid client interaction while > the > shutdown is processing? Then the process would look like this: > > 1. Someone calls ORB::shutdown(). > > 2. All server side protocol connections are blocked from processing > further input. > > 3. All pending requests are drained from the ORB. > > 4. Once the last outstanding reply is sent on a server connection, > the > connection is shutdown (via CloseConnection if GIOP). > > 4. All POAs are destroyed. In the case of ORB::shutdown this may work because all new requests will be rejected and therefore POAs can be destroyed without deadlock or infinite looping. If the ORB isn't being shutdown, however (we are just destroying a POA and its descendants), new requests can still come in and we're back in the same hole as before. > 5. Any other ORB level cleanup is done. > > Of course this could be left as a quality of ORB implementation > issue, > but that is how I would implement this. After further consideration, I no longer think it is tolerable for the POAs to behave as if INACTIVE. The common case is POA destruction without mutual reference to sibling POAs, and this should not be penalized by opening up a window for client requests to be rejected. Furthermore, I no longer think it is possible to solve this deadlock problem. I think the best we can do is as follows: 1. use the scheme outlined in my revised proposal which first marks the POA and its descendants as pending destruction. 2. keep the original meaning of 'pending destruction', which means they behave as if their POAManagers are in the HOLDING state. The POAs will not behave as if their POAManagers are in the INACTIVE state because this surfaces the destruction to clients which is the wrong behavior. 3. declare that POA etherealization which depends on access to other POAs that are pending destruction is dangerous and should be avoided since it can cause deadlocks. I think #1 and #2 are necessary to maintain atomicity (a POA should not be recreated if its parent POA is pending destruction) and to protect clients. any thoughts? -Jon Return-Path: Date: Thu, 09 Jul 1998 12:40:46 -0400 From: Paul H Kyzivat Organization: NobleNet To: Jon Goldberg CC: Jonathan Biggar , port-rtf@omg.org Subject: Re: Proposed resolutions for Issues 1407-1410, 1428 References: <359AE9FF.441D15F7@inprise.com> <35A3267A.181DCAD8@inprise.com> <35A38C82.5C41F600@floorboard.com> <35A3B2CA.694FC370@inprise.com> Jon Goldberg wrote: > After further consideration, I no longer think it is tolerable for > the POAs to behave as if INACTIVE. The common case is POA > destruction > without mutual reference to sibling POAs, and this should not be > penalized by opening up a window for client requests to be rejected. I agree. > > Furthermore, I no longer think it is possible to solve this deadlock > problem. I think the best we can do is as follows: > > 1. use the scheme outlined in my revised proposal which first marks > the POA and its descendants as pending destruction. > > 2. keep the original meaning of 'pending destruction', which means > they > behave as if their POAManagers are in the HOLDING state. The > POAs will not behave as if their POAManagers are in the INACTIVE > state > because this surfaces the destruction to clients which is the > wrong behavior. > > 3. declare that POA etherealization which depends on access to other > POAs that are pending destruction is dangerous and should be avoided > since it can cause deadlocks. I am not sure I fully understand how you want this to work, but I think there is a problem: It should be possible to destroy all the POAs by destroying the root POA, and have something reasonable happen. This is implicitly what happens during shutdown, and it ought to be possible to do the same thing explicitly. But doing something reasonable in this case means that etherealization ought to take place (except perhaps in pathological cases). If destroying a parent POA first marks its entire tree as destroyed, and then starts etherealizing things, the POAs at the bottom of the tree can't use servant managers that are nearer to the root. The best way I can see to make this work reasonably is to have destroy on a POA first work bottom-up, destroying the most derived children first while leaving the higher up POAs free to service calls to any servant managers they may contain. Once a POA has given its children a chance this way, it can finally make itself unusable and etherealize its own children. This process eventually percolates all the way to the the POA that the first destroy request was sent to. There are still failure modes - when a POA has a ServantActivator in a peer or child POA, or an etherealize method that activates new objects in another POA. In the worst cases etherealization of some servants fails. I would just like to get the common cases working reasonably. > > I think #1 and #2 are necessary to maintain atomicity (a POA should > not be recreated if its parent POA is pending destruction) and > to protect clients. > > any thoughts? Yes - above. Return-Path: Date: Fri, 10 Jul 1998 19:18:36 -0700 From: "Jon Goldberg" To: port-rtf@omg.org Subject: Revision of 1408-9, 1428 Hi Folks- We seemed to have reached some consensus in the last conference call on the behavior of destroy (1408-9) and wait_for_completion (1428). The following proposal is assumed to withdraw any previous proposals and to invalidate previous votes. Please consider the new proposal and cast your vote. Even if you have already voted on a previous proposal, you need to vote again because Dan is wiping the slate clean. 1. The new proposal is mostly George's original which had the following characteristics: a. the atomicity for a destroy() is a single POA. If you destroy a parent POA, it will *first* destroy its children and then destroy itself. While the children are being destroyed, there is no indication that the parent POA is pending destruction. b. the behavior of a POA pending destruction is as if its POAManager is in the holding state. New requests will be queued. 2. That proposal is now modified to indicate that the parent POA destroys its children recursively, and then destroys itself once there are no more children. This covers the case where a new child is created (or recreated) during destruction. 3. I realized the following new problem while writing this message: a. Object A calls (remote) object B which calls back into Object A. b. If destroy is called on A's POA before it calls over to object B, the rules we discussed previously will cause deadlock since destruction is defined as pending until all active requests have completed. This call pattern is too common to have this deadlock allowed. Therefore, I've modified the proposal such that the POA pending destruction does *not* wait for all currently outstanding requests to complete. Instead, it sets itself as if its POAManager is in the holding state and then starts etherealization. If any currently executing requests then recursively call back into that POA, they will block until the etherealization is complete. This should be fine even in multi-threaded environments, assuming etherealization no longer explicitly destroys the Servant (which is a language-mapping issue). If the etherealization happens to cause a call back into that same POA, there will be a deadlock. The new proposal calls this out as a warning about complicated processing during etherealization. The proposal for 1428 is very stable at this point and I'm hoping it will be adopted quickly. I only modified the phrase "POA-dispatched execution context" to "execution context dispatched by some POA" for clarity. fire away, Jon ---------------------------------------------------------------- Issue 1408: POA destroy() is ill-defined Issue 1409: Multiple threads calling destroy() once destroy() has begun Nature: Revision Summary: POA destroy is not defined sufficiently enough to prevent multiple activations of the same POA name in the same process. POA::destroy semantics are not defined for multiple threads calling destroy. Resolution: Accepted for Corba RTF 2.3 Revision: The following text will replace the current describing the destroy operation in Section 9.3.8, Page 9-31. This operation destroys the POA and all descendant POAs. All descendant POAs are destroyed (recursively) before the destruction of the containing POA. The POA so destroyed (that is, the POA with its name) may be re-created later in the same process. (This differs from the POAManager::deactivate operation that does not allow a re-creation of its associated POA in the same process. After a deactivate, re-creation is allowed only if the POA is later destroyed.) When a POA is destroyed, any requests that have started execution continue to completion. Any requests that have not started execution are processed as if the POA's POAManager were in the holding state, until POA destruction is complete. Once POA destruction is complete, all queued requests (if any) will behave as if they were newly arrived, that is, the POA will attempt to cause recreation of the POA by invoking one or more adapter activators. POA destruction does not block until all active requests complete execution, as this can cause deadlock. If the etherealize_objects parameter is TRUE, the POA has the RETAIN policy, and a servant manager is registered with the POA, the etherealize operation on the servant manager will be called for each active object in the Active Object Map. Etherealization can occur while active requests are still executing on those servants. The POA behaves as if its POAManager were in the holding state while all calls to etherealize are made. Therefore, an etherealize method that attempts to invoke operations on the POA will deadlock. POA destruction is considered complete once all active objects have been etherealized. {editorial note - below I have merged text from these two issues and the relevant text from issue 1428 because they change the same text. If one of these resolutions fails, we will need to edit the following text) The wait_for_completion parameter is handled as follows: - If wait_for_completion is TRUE and the current thread is not in an invocation context dispatched from any POA, the destroy operation will return only after all active requests have completed and all invocations of etherealize have completed. - If wait_for_completion is TRUE and the current thread is in an invocation context dispatched from any POA, then the BAD_INV_ORDER exception is thrown and POA destruction does not occur. - If wait_for_completion is FALSE, the destroy operation destroys the POA but does not wait for active requests to complete nor for etherealization to occur. If destroy is called multiple times before destruction is complete (because there are active requests), the etherealize_objects parameter will only apply to the first call of destroy. Subsequent calls with conflicting etherealize_objects settings will use the value of the etherealize_objects from the first call. The wait_for_completion parameter will be handled as defined above for each individual call (some callers may choose to block, while others may not). ---------------------------------------------------------------- Issue 1428: Blocking POA Operations Nature: Revision Summary: Several operations added to CORBA as part of the Portability submission provide blocking behavior which can result in deadlock in a large number of cases. These calls include POA::destroy, ORB::shutdown, POAManager::deactivate, POAManager::hold_requests, POAManager::discard_requests. Resolution: Accepted for Corba 2.3 RTF Revision: The following changes are proposed: Replace the second sentence in the paragraph of section 4.9.4, page 4-20, which begins with "If the wait_for_completion ..." with the following: "If the wait_for_completion parameter is TRUE and the current thread is not in an invocation context dispatched by some ORB, this operation blocks until all ORB processing (including request processing and object deactivation or other operations associated with object adapters) has completed. If the wait_for_completion parameter is TRUE and the current thread is in an invocation context dispatched by some ORB, then the BAD_INV_ORDER exception is thrown." Replace the phrase "If the parameter is TRUE" in the 2nd paragraph, 2nd sentence of the hold_requests description in Section 9.3.2, page 9-18, with the phrase "If the parameter is TRUE and the current thread is not in an invocation context dispatched by some POA". Add the sentence "If the parameter is TRUE and the current thread is in an invocation context dispatched by some POA then the BAD_INV_ORDER exception is raised and the state is not changed." Replace the phrase "If the parameter is TRUE" in the 2nd paragraph, 2nd sentence of the discard_requests description in Section 9.3.2, page 9-18, with the phrase "If the parameter is TRUE and the current thread is not in an invocation context dispatched by some POA". Add the sentenence "If the parameter is TRUE and the current thread is in an invocation context dispatched by some POA then the BAD_INV_ORDER exception is raised and the state is not changed." to the end of the paragraph. Replace the phrase "If the parameter is TRUE" in the 3rd paragraph, 2nd sentence of the deactivate description in Section 9.3.2, page 9-18, with the phrase "If the parameter is TRUE and the current thread is not in an invocation context dispatched by some POA". Add the sentenence "If the parameter is TRUE and the current thread is in an invocation context dispatched by some POA then the BAD_INV_ORDER exception is raised and the state is not changed." to the end of the paragraph. (editorial note- I changed the text for POA::destroy in the resolution of issues 1408 and 1409 above, so it is not shown here but a similar change to the above is necessary. I also felt that 1409 raised an issue which also affects POAManager::deactivate, so the following change is also proposed) Add the following paragraph to the end of the description of deactivate in section 9.3.2, page 9-19. "If deactivate is called multiple times before destruction is complete (because there are active requests), the etherealize_objects parameter will only apply to the first call of deactivate, subsequent calls with conflicting etherealize_objects settings will use the value of the etherealize_objects from the first call. The wait_for_completion parameter will be handled as defined above for each individual call (some callers may choose to block, while others may not)." ---------------------------------------------------------------- Return-Path: Sender: jon@floorboard.com Date: Fri, 10 Jul 1998 20:13:34 -0700 From: Jonathan Biggar To: Jon Goldberg CC: port-rtf@omg.org Subject: Re: Revision of 1408-9, 1428 References: <35A6CB7C.DCD123B@inprise.com> Jon Goldberg wrote: > > Hi Folks- > > We seemed to have reached some consensus in the last conference call > on > the behavior of destroy (1408-9) and wait_for_completion (1428). > The > following proposal is assumed to withdraw any previous proposals and > to invalidate previous votes. Please consider the new proposal and > cast your vote. Even if you have already voted on a previous > proposal, you need to vote again because Dan is wiping the slate > clean. I vote yes on both proposals. > 1. The new proposal is mostly George's original which had > the following characteristics: > a. the atomicity for a destroy() is a single POA. If you > destroy a parent POA, it will *first* destroy its children > and then destroy itself. While the children are being destroyed, > there is no indication that the parent POA is pending destruction. > b. the behavior of a POA pending destruction is as if its > POAManager > is in the holding state. New requests will be queued. > > 2. That proposal is now modified to indicate that the parent POA > destroys its children recursively, and then destroys itself once > there are no more children. This covers the case where a new child > is created (or recreated) during destruction. > > 3. I realized the following new problem while writing this message: > > a. Object A calls (remote) object B which calls back into > Object A. > b. If destroy is called on A's POA before it calls over to > object B, the rules we discussed previously will cause > deadlock since destruction is defined as pending until > all active requests have completed. > > This call pattern is too common to have this deadlock allowed. > Therefore, I've modified the proposal such that the POA pending > destruction does *not* wait for all currently outstanding requests > to > complete. Instead, it sets itself as if its POAManager is in the > holding state and then starts etherealization. If any currently > executing requests then recursively call back into that POA, they > will > block until the etherealization is complete. This should be fine > even > in multi-threaded environments, assuming etherealization no longer > explicitly destroys the Servant (which is a language-mapping issue). > If the etherealization happens to cause a call back into that same > POA, there will be a deadlock. The new proposal calls this out as a > warning about complicated processing during etherealization. I think we are going to end up chasing our tails forever if we try to work around every possible deadlock condition. I think we have this tuned about as good as it's going to get. -- Jon Biggar Floorboard Software jon@floorboard.com jon@biggar.org Return-Path: Date: Sun, 12 Jul 1998 22:08:09 -0400 From: Bob Kukura Organization: IONA Technologies To: Jon Goldberg CC: port-rtf@omg.org Subject: Re: Revision of 1408-9, 1428 References: <35A6CB7C.DCD123B@inprise.com> I want to vote YES on these, but can't. I have to vote NO (for Martin) on 140[89] because, unless I am completely misinterpreting Jon's last-minute deadlock workaround, it seems to be allowing a POA to call etherealize() on a Servant while that Servant is still processing a request dispatched by that POA. If I am completely misinterpreting the text, then I still have to vote NO because the proposed text is ambiguous. Assuming I understand the intent, this seems like a fundamental change to the "serialization rules" for incarnation and etherealization, and to the role etherealize() plays in a server application. My initial reaction is that, with this change, etherealization becomes totally meaningless, and might as well be deprecated. The etherealize() call was intended to be used by applications as an indication of when a Servant is no longer used by a particular POA - allowing the application to save the Servant's state, free the Servant's storage, or do whatever else the application needed to do when that POA was done with that Servant. But I don't think any of these uses are possible any longer with this change, since the Servant may still be processing requests when etherealize() is called. We voted YES on the reference counting proposal on the basis that use of reference counting remains optional and the existing programs are not invalidated. But this additional change seems to force whatever work used to be done in etherialize() to be moved into _remove_ref(). Not only would this break existing programs, but the POA would lose significant functionality, since there would then be no way to share a Servant among several POAs and know for certain when a particular POA is done with that Servant. I have to vote NO on the latest proposal for 1428 because it introduces cross-ORB dependencies that are not necessary. It describes ORB::shutdown()'s wait_for_completion flag as being valid when "the current thread is not in an invocation context dispatched by some ORB". I see no reason why any invocation context of any ORB instance other than the one being shutdown should matter. Similarly, the wait_for_completion flag should only be invalid in POA operations when called from invocation contexts dispatched by some POA belonging to the same ORB as the POA on which the operation was invoked. -Bob Jon Goldberg wrote: > > Hi Folks- > > We seemed to have reached some consensus in the last conference call > on > the behavior of destroy (1408-9) and wait_for_completion (1428). > The > following proposal is assumed to withdraw any previous proposals and > to invalidate previous votes. Please consider the new proposal and > cast your vote. Even if you have already voted on a previous > proposal, you need to vote again because Dan is wiping the slate > clean. > > 1. The new proposal is mostly George's original which had > the following characteristics: > a. the atomicity for a destroy() is a single POA. If you > destroy a parent POA, it will *first* destroy its children > and then destroy itself. While the children are being destroyed, > there is no indication that the parent POA is pending destruction. > b. the behavior of a POA pending destruction is as if its > POAManager > is in the holding state. New requests will be queued. > > 2. That proposal is now modified to indicate that the parent POA > destroys its children recursively, and then destroys itself once > there are no more children. This covers the case where a new child > is created (or recreated) during destruction. > > 3. I realized the following new problem while writing this message: > > a. Object A calls (remote) object B which calls back into > Object A. > b. If destroy is called on A's POA before it calls over to > object B, the rules we discussed previously will cause > deadlock since destruction is defined as pending until > all active requests have completed. > > This call pattern is too common to have this deadlock allowed. > Therefore, I've modified the proposal such that the POA pending > destruction does *not* wait for all currently outstanding requests > to > complete. Instead, it sets itself as if its POAManager is in the > holding state and then starts etherealization. If any currently > executing requests then recursively call back into that POA, they > will > block until the etherealization is complete. This should be fine > even > in multi-threaded environments, assuming etherealization no longer > explicitly destroys the Servant (which is a language-mapping issue). > If the etherealization happens to cause a call back into that same > POA, there will be a deadlock. The new proposal calls this out as a > warning about complicated processing during etherealization. > > The proposal for 1428 is very stable at this point and I'm > hoping it will be adopted quickly. I only modified the > phrase "POA-dispatched execution context" to "execution context > dispatched by some POA" for clarity. > > fire away, > Jon > > ---------------------------------------------------------------- > Issue 1408: POA destroy() is ill-defined > Issue 1409: Multiple threads calling destroy() once destroy() > has begun > Nature: Revision > > Summary: POA destroy is not defined sufficiently enough to > prevent multiple activations of the same POA name in > the same process. POA::destroy semantics are not defined > for multiple threads calling destroy. > > Resolution: Accepted for Corba RTF 2.3 > > Revision: The following text will replace the current describing > the destroy operation in Section 9.3.8, Page 9-31. > > This operation destroys the POA and all descendant POAs. All > descendant POAs are destroyed (recursively) before the destruction > of > the containing POA. The POA so destroyed (that is, the POA with its > name) may be re-created later in the same process. (This differs > from > the POAManager::deactivate operation that does not allow a > re-creation > of its associated POA in the same process. After a deactivate, > re-creation is allowed only if the POA is later destroyed.) > > When a POA is destroyed, any requests that have started execution > continue to completion. Any requests that have not started > execution are processed as if the POA's POAManager were in the > holding state, until POA destruction is complete. Once > POA destruction is complete, all queued requests (if > any) will behave as if they were newly arrived, that is, the POA > will attempt to cause recreation of the POA by invoking one or > more adapter activators. POA destruction does not block > until all active requests complete execution, as this can > cause deadlock. > > If the etherealize_objects parameter is TRUE, the POA has the RETAIN > policy, and a servant manager is registered with the POA, the > etherealize operation on the servant manager will be called for each > active object in the Active Object Map. Etherealization can occur > while active requests are still executing on those servants. The > POA > behaves as if its POAManager were in the holding state while all > calls > to etherealize are made. Therefore, an etherealize method that > attempts to invoke operations on the POA will deadlock. POA > destruction is considered complete once all active objects > have been etherealized. > > {editorial note - below I have merged text from these two issues and > the relevant text from issue 1428 because they change the same text. > If one of these resolutions fails, we will need to edit the > following > text) > > The wait_for_completion parameter is handled as follows: > > - If wait_for_completion is TRUE and the current > thread is not in an invocation context dispatched from any POA, > the destroy operation will return only after all active requests > have completed and all invocations of etherealize have > completed. > > - If wait_for_completion is TRUE and the current thread is in an > invocation context dispatched from any POA, then the > BAD_INV_ORDER > exception is thrown and POA destruction does not occur. > > - If wait_for_completion is FALSE, the destroy operation > destroys the POA but does not wait for active requests to > complete nor for etherealization to occur. > > If destroy is called multiple times before destruction is > complete (because there are active requests), the > etherealize_objects parameter will only apply to the first > call of destroy. Subsequent calls with conflicting > etherealize_objects settings will use the value of the > etherealize_objects from the first call. The wait_for_completion > parameter will be handled as defined above for each individual > call (some callers may choose to block, while others may not). > > ---------------------------------------------------------------- > Issue 1428: Blocking POA Operations > Nature: Revision > > Summary: Several operations added to CORBA as part of the > Portability submission provide blocking behavior which > can result in deadlock in a large number of cases. > These calls include POA::destroy, ORB::shutdown, > POAManager::deactivate, POAManager::hold_requests, > POAManager::discard_requests. > > Resolution: Accepted for Corba 2.3 RTF > > Revision: The following changes are proposed: > > Replace the second sentence in the paragraph of section 4.9.4, > page 4-20, which begins with "If the wait_for_completion ..." > with the following: > > "If the wait_for_completion parameter is TRUE and the current > thread is not in an invocation context dispatched by some ORB, this > operation blocks until all ORB processing (including request > processing and object deactivation or other operations associated > with object adapters) has completed. If the wait_for_completion > parameter is TRUE and the current thread is in an invocation context > dispatched by some ORB, then the BAD_INV_ORDER exception is thrown." > > Replace the phrase "If the parameter is TRUE" in the 2nd > paragraph, 2nd sentence of the hold_requests description in > Section 9.3.2, page 9-18, with the phrase "If the parameter > is TRUE and the current thread is not in an invocation context > dispatched by some POA". Add the sentence "If the parameter > is TRUE and the current thread is in an invocation context > dispatched by some POA then the BAD_INV_ORDER exception is raised > and the state is not changed." > > Replace the phrase "If the parameter is TRUE" in the 2nd > paragraph, 2nd sentence of the discard_requests description in > Section 9.3.2, page 9-18, with the phrase "If the parameter > is TRUE and the current thread is not in an invocation context > dispatched by some POA". Add the sentenence "If the parameter > is TRUE and the current thread is in an invocation context > dispatched by some POA then the BAD_INV_ORDER exception is raised > and the state is not changed." to the end of the paragraph. > > Replace the phrase "If the parameter is TRUE" in the 3rd > paragraph, 2nd sentence of the deactivate description in > Section 9.3.2, page 9-18, with the phrase "If the parameter > is TRUE and the current thread is not in an invocation context > dispatched by some POA". Add the sentenence "If the parameter > is TRUE and the current thread is in an invocation context > dispatched > by some POA then the BAD_INV_ORDER exception is raised and the > state is not changed." to the end of the paragraph. > > (editorial note- I changed the text for POA::destroy in the > resolution of issues 1408 and 1409 above, so it is not shown > here but a similar change to the above is necessary. > I also felt that 1409 raised an issue which also affects > POAManager::deactivate, so the following change is also > proposed) > > Add the following paragraph to the end of the description of > deactivate in section 9.3.2, page 9-19. > > "If deactivate is called multiple times before destruction is > complete (because there are active requests), the > etherealize_objects parameter will only apply to the first > call of deactivate, subsequent calls with conflicting > etherealize_objects settings will use the value of the > etherealize_objects from the first call. The wait_for_completion > parameter will be handled as defined above for each individual > call (some callers may choose to block, while others may not)." > > ---------------------------------------------------------------- Return-Path: Sender: jon@floorboard.com Date: Sun, 12 Jul 1998 20:01:53 -0700 From: Jonathan Biggar To: Bob Kukura CC: Jon Goldberg , port-rtf@omg.org Subject: Re: Revision of 1408-9, 1428 References: <35A6CB7C.DCD123B@inprise.com> <35A96C09.C54EBC6@iona.com> Bob Kukura wrote: > > I want to vote YES on these, but can't. > > I have to vote NO (for Martin) on 140[89] because, unless I am > completely misinterpreting Jon's last-minute deadlock workaround, it > seems to be allowing a POA to call etherealize() on a Servant while > that > Servant is still processing a request dispatched by that POA. If I > am > completely misinterpreting the text, then I still have to vote NO > because the proposed text is ambiguous. > > Assuming I understand the intent, this seems like a fundamental > change > to the "serialization rules" for incarnation and etherealization, > and to > the role etherealize() plays in a server application. My initial > reaction is that, with this change, etherealization becomes totally > meaningless, and might as well be deprecated. The etherealize() > call > was intended to be used by applications as an indication of when a > Servant is no longer used by a particular POA - allowing the > application > to save the Servant's state, free the Servant's storage, or do > whatever > else the application needed to do when that POA was done with that > Servant. But I don't think any of these uses are possible any > longer > with this change, since the Servant may still be processing requests > when etherealize() is called. > > We voted YES on the reference counting proposal on the basis that > use of > reference counting remains optional and the existing programs are > not > invalidated. But this additional change seems to force whatever > work > used to be done in etherialize() to be moved into _remove_ref(). > Not > only would this break existing programs, but the POA would lose > significant functionality, since there would then be no way to share > a > Servant among several POAs and know for certain when a particular > POA is > done with that Servant. I have to withdraw my yes vote, because Bob's argument is convincing. The POA must wait until all requests on an object are completed before it can etherealize it. If that causes deadlocks for poorly written applications, so be it. > I have to vote NO on the latest proposal for 1428 because it introduces > cross-ORB dependencies that are not necessary. It describes > ORB::shutdown()'s wait_for_completion flag as being valid when "the > current thread is not in an invocation context dispatched by some ORB". > I see no reason why any invocation context of any ORB instance other > than the one being shutdown should matter. Similarly, the > wait_for_completion flag should only be invalid in POA operations when > called from invocation contexts dispatched by some POA belonging to the > same ORB as the POA on which the operation was invoked. But you can get the situation where a process 1 calls process 2 which calls back to process 1 which then calls back to process 2 with an operation that calls shutdown on process 2. This will deadlock. Perhaps this is too complicated an should just be warned about, but it isn't just local calls that can cause a deadlock. It just much easier from a documentation and implementation point of view to simply disallow shutdown(TRUE) inside any POA dispatched request. -- Jon Biggar Floorboard Software jon@floorboard.com jon@biggar.org Return-Path: Sender: jon@floorboard.com Date: Sun, 12 Jul 1998 20:19:03 -0700 From: Jonathan Biggar To: Bob Kukura , Jon Goldberg , port-rtf@omg.org Subject: Re: Revision of 1408-9, 1428 References: <35A6CB7C.DCD123B@inprise.com> <35A96C09.C54EBC6@iona.com> <35A978A1.104341A7@floorboard.com> Jonathan Biggar wrote: > I have to withdraw my yes vote, because Bob's argument is > convincing. > The POA must wait until all requests on an object are completed > before > it can etherealize it. If that causes deadlocks for poorly written > applications, so be it. To clarify my vote, if the proposal is changed back to require requests to complete before the etherealization of the object, then I will vote yes. -- Jon Biggar Floorboard Software jon@floorboard.com jon@biggar.org Return-Path: From: "Daniel R. Frantz" To: "'Bob Kukura'" Cc: Subject: RE: Revision of 1408-9, 1428 Date: Mon, 13 Jul 1998 08:19:04 -0400 X-MSMail-Priority: Normal Importance: Normal X-MimeOLE: Produced By Microsoft MimeOLE V4.72.2106.4 X-MIME-Autoconverted: from 8bit to quoted-printable by beasys.com id FAA07096 >-----Original Message----- >From: Bob Kukura [mailto:kukura@iona.com] >Sent: Sunday, July 12, 1998 10:08 PM >To: Jon Goldberg >Cc: port-rtf@omg.org >Subject: Re: Revision of 1408-9, 1428 > > >I want to vote YES on these, but can't. > >I have to vote NO (for Martin) on 140[89] because, unless I am >completely misinterpreting Jon's last-minute deadlock workaround, it >seems to be allowing a POA to call etherealize() on a Servant >while that >Servant is still processing a request dispatched by that POA. ... >The etherealize() call >was intended to be used by applications as an indication of when a >Servant is no longer used by a particular POA - allowing the application >to save the Servant's state, free the Servant's storage, or do whatever >else the application needed to do when that POA was done with that >Servant. Hmmm... The current spec already does what you don't like. I think your understanding of etherealize is incorrect. A POA calls etherealize only as the result of taking a single ObjectId from the AOM, not strictly for the purpose of deleting servants. That means that etherealize can indeed save the Servant's state for this particular object. It doesn't necessarily mean that the POA is done with the Servant. The current spec makes a distinction between a Servant executing multiple requests on the same object and a Servant executing requests on other objects, so that may be the source of confusion. (see below) So, if a POA can already call etherealize when a Servant is still processing, the proposals for 1408-9 and 1428 don't change etherealize at all. They are only trying to clean up some problems regarding POA::destroy(). If 1408-9,28 aren't good enough for destroy(), we should wait for the next round, but if they clean up destroy() without changing etherealize, it is useful to put them in now. I think they don't change etherealize and they're good enough. I vote YES. Now, what does etherealize really mean? Under deactivate, p. 9-35 This operation causes the association of the Object Id specified by the oid parameter and its servant to be removed from the Active Object Map. If a servant manager is associated with the POA, ServantLocator::etherealize will be invoked with the oid and the servant. . . Note cator::etherealize may be invoked multiple times with the same servant when the other objects are deactivated. It is the responsibility of the object implementation to refrain from destroying the servant while it is active with any Id. The very signature of etherealize clearly shows the intent to allow etherealization while other request processing is still going on. Etherealize is on pages 9-22/23. void etherealize ( in ObjectId oid, in POA adapter, in Servant serv, in boolean cleanup_in_progress, in boolean remaining_activations ); and then the description of "remaining_activation" says (last paragraph in the section). In a multi-threaded environment, the POA makes certain guarantees that allow servant managers to safely destroy servants. Specifically, the servant's entry in the Active Object Map corresponding to the target object is removed before etherealize() is called. Because calls to incarnate() and etherealize() are serialized, this prevents new requests for the target object from being invoked on the servant during etherealization. After removing the entry from the Active Object Map, if the POA determines before invoking etherealize() that other requests for the same target object are already in progress on the servant, it delays the call to etherealize() until all active methods for the target object have completed. Therefore, when etherealize() is called, the servant manager can safely destroy the servant if it wants to, unless the remaining_activations argument is TRUE. The last two sentence seem pretty clear: the POA will wait till all processing is done for the single object that was deactivated, but won't wait till processing is done on all objects. If it is the case the processing is done on all objects, then remaining_activation is FALSE and the etherealize can delete the Servant, but not otherwise. In the case of POA::destroy(), the POA will deactivate each object, leading to an etherealize call for each object in the AOM. Eventually will call etherealize when there are no more objects in the AOM for that servant, so that etherealize can indeed delete the servant. Dan Return-Path: Date: Mon, 13 Jul 1998 09:51:23 -0400 From: Paul H Kyzivat Organization: NobleNet To: Jon Goldberg CC: port-rtf@omg.org Subject: Re: Revision of 1408-9, 1428 References: <35A6CB7C.DCD123B@inprise.com> I agree that calling etherealize with operations outstanding is a bad idea. In addition, the following doesn't seem possible: > - If wait_for_completion is FALSE, the destroy operation > destroys the POA but does not wait for active requests to > complete nor for etherealization to occur. The new proposal has recursive destruction of child POAs happening first, during which time the parent seems to be fully functional. I believe that in effect, the destruction of the child POAs must be done with wait_for_completion TRUE, to guarantee that they are done using the parent POAs for etherealization. For that to work, it means that if destroy is called on a POA with wait_for_completion FALSE, the call may return with the POA still appearing to be functional, undestroyed. However a sequence of events has been initiated that will eventually lead to its being destroyed, once all of its children are destroyed. This only requires a change to the wording in the section quoted above, something like: - If wait_for_completion is FALSE, the destroy operation initiates destruction of the POA but does not wait for active requests to complete nor for etherealization to occur. Upon return from the operation the POA may not yet be destroyed, but will become destroyed after the destruction and etherealization of child POAs is complete. Return-Path: Date: Mon, 13 Jul 1998 11:04:09 -0400 From: Bob Kukura Organization: IONA Technologies To: "Daniel R. Frantz" CC: port-rtf@omg.org Subject: Re: Revision of 1408-9, 1428 References: <008601bdae58$6e26e600$3fc5bdce@idler.beasys.com> Daniel R. Frantz wrote: > > >-----Original Message----- > >From: Bob Kukura [mailto:kukura@iona.com] > >Sent: Sunday, July 12, 1998 10:08 PM > >To: Jon Goldberg > >Cc: port-rtf@omg.org > >Subject: Re: Revision of 1408-9, 1428 > > > > > >I want to vote YES on these, but can't. > > > >I have to vote NO (for Martin) on 140[89] because, unless I am > >completely misinterpreting Jon's last-minute deadlock workaround, > it > >seems to be allowing a POA to call etherealize() on a Servant > >while that > >Servant is still processing a request dispatched by that POA. > ... > >The etherealize() call > >was intended to be used by applications as an indication of when a > >Servant is no longer used by a particular POA - allowing the > application > >to save the Servant's state, free the Servant's storage, or do > whatever > >else the application needed to do when that POA was done with that > >Servant. > > Hmmm... The current spec already does what you don't like. I think > your > understanding of etherealize is incorrect. A POA calls etherealize > only > as the result of taking a single ObjectId from the AOM, not strictly > for > the purpose of deleting servants. That means that etherealize can > indeed > save the Servant's state for this particular object. It doesn't > necessarily mean that the POA is done with the Servant. I apologize for not being clear enough in my message. I do understand that a Servant can be etherealized for one OID while it is still serving requests for other OIDs. My reading of Jon's proposal and his associated discussion was that he wanted to change the rules to allow a Servant to be etherealized for an OID while it is still serving requests for that same OID. It is that change that I object to. Jon, if I am missinterpreting you here, please speak up. > > The current spec makes a distinction between a Servant executing > multiple requests on the same object and a Servant executing > requests on > other objects, so that may be the source of confusion. (see below) > > So, if a POA can already call etherealize when a Servant is still > processing, the proposals for 1408-9 and 1428 don't change > etherealize > at all. They are only trying to clean up some problems regarding > POA::destroy(). If 1408-9,28 aren't good enough for destroy(), we > should > wait for the next round, but if they clean up destroy() without > changing > etherealize, it is useful to put them in now. I think they don't > change > etherealize and they're good enough. I vote YES. > > Now, what does etherealize really mean? Under deactivate, p. 9-35 > > This operation causes the association of the Object Id > specified by the oid parameter and its servant to be > removed from the Active Object Map. If a servant manager > is associated with the POA, ServantLocator::etherealize > will be invoked with the oid and the servant. > . > . > Note or::etherealize > may be invoked multiple times with the same servant when > the other objects are deactivated. It is the > responsibility of the object implementation to refrain > from destroying the servant while it is active with any > Id. > > The very signature of etherealize clearly shows the intent to allow > etherealization while other request processing is still going on. > Etherealize is on pages 9-22/23. > > void etherealize ( > in ObjectId oid, > in POA adapter, > in Servant serv, > in boolean cleanup_in_progress, > in boolean remaining_activations ); > > and then the description of "remaining_activation" says (last > paragraph > in the section). > > In a multi-threaded environment, the POA makes > certain guarantees that allow servant managers > to safely destroy servants. Specifically, the > servant's entry in the Active Object Map > corresponding to the target object is removed > before etherealize() is called. Because calls > to incarnate() and etherealize() are serialized, > this prevents new requests for the target object > from being invoked on the servant during > etherealization. After removing the entry from > the Active Object Map, if the POA determines > before invoking etherealize() that other > requests for the same target object are already > in progress on the servant, it delays the call > to etherealize() until all active methods for > the target object have completed. Therefore, > when etherealize() is called, the servant > manager can safely destroy the servant if it > wants to, unless the remaining_activations > argument is TRUE. This text pretty clearly requires that the POA not call etherealize() for a particular Servant/OID combination until all dispatched invocations on that Servant for that OID have completed. If Jon had intended to change this behaviour, he would have had to change this paragraph as well. Either he missed this, or I am completely misinterpreting his proposal. > > The last two sentence seem pretty clear: the POA will wait till all > processing is done for the single object that was deactivated, but > won't > wait till processing is done on all objects. If it is the case the > processing is done on all objects, then remaining_activation is > FALSE > and the etherealize can delete the Servant, but not otherwise. > > In the case of POA::destroy(), the POA will deactivate each object, > leading to an etherealize call for each object in the > AOM. Eventually > will call etherealize when there are no more objects in the AOM for > that > servant, so that etherealize can indeed delete the servant. If this remains the specified behaviour, I don't understand what Jon means by "the POA pending destruction does *not* wait for all currently outstanding requests to complete" in the discussion and by "POA destruction does not block until all active requests complete execution, as this can cause deadlock" in the proposed text. If this can be explained to me without invalidating the above "remaining_activations" paragraph, and the proposed text can be clarified if necessary (if its not just me), then I'd be happy to vote YES. -Bob > > Dan Return-Path: Date: Mon, 13 Jul 1998 08:30:47 -0700 From: "Jon Goldberg" To: Jonathan Biggar CC: kukura@iona.com, jgoldberg@inprise.com, port-rtf@omg.org Subject: Re: Revision of 1408-9, 1428 References: <35A6CB7C.DCD123B@inprise.com> <35A96C09.C54EBC6@iona.com> On 1428, I think we *should* prevent shutdown(TRUE) from being called in any dispatched thread since we should be preventing even that complex deadlock scenario from being possible. (In case it isn't recorded, Jeff M. has voted YES on both of these proposals). As far as 1408-9, if we change it to only allow etherealization after all active requests have completed, we will allow deadlock in the very simple recursive scenario. I think the RTF is just stuck on this one and I won't bother amending the proposal further since we're out of time. -Jon G. Return-Path: Sender: "George Scott" Date: Tue, 14 Jul 1998 20:37:00 -0700 From: "George M. Scott" Organization: Inprise Corporation To: port-rtf@omg.org CC: gscott@inprise.com Subject: urgent POA issues Unfortunately, I missed all of the exciting debate on the port-rtf list last week, but I did notice a number of the most significant problems (IMHO) did not reach resolution. I think it is fairly important that we reach resolution on these for CORBA 2.3, as they fix a number of potentially serious problems that affect portability of multi-threaded or recursive POA programs. It seemed to me that we were very close to resolution and that if we can get agreement among the members of the POA RTF, we could then try and find a way to get the changes into CORBA 2.3 through some means (It's a shame that July is such a popular vacation month. ;-) ). The issues of concern are 1408, 1409, 1428, and 1627 (which was mistakenly transferred to the C++ RTF IMHO). I intend to send out concrete proposals for these issues in separate messages. If necessary I will reopen 1627 as a new issue, since it is not a language mapping issue, but a general POA issue. First let me summarize the problems and then provide some discussion: 1428 - blocking POA operations. This issue relates to potential deadlock situations in various ORB/POA calls which take a wait_for_completion parameter. It was not clear to me why this did not pass since it looked like there was consensus last Friday, and it should have been included in the RTF report. 1408/1409 - problems with destroy(). The behavior of destroy is not defined explicitly enough to guarantee consistency of programs. 1627 - problems with deactivate_object(). The behavior of deactivate_object is not defined explicitly enough to guarantee consistency of programs. Note the solution to this issue is parallel to the issue of 1408/1409. Also, a very important note, this issue has nothing to do with C++ memory management of servants, which is a separate language mapping specific issue and should be addressed in the C++ RTF (IMHO). I don't think we need much further discussion on 1428, so I mostly want to discuss destroy and deactivate_object which are essentially the same problem. The issue is when can an object be safely deactivated and when can a POA be safely destroyed. More importantly, when does the apparent destruction of the POA occur and when does the apparent deactivation of the object occur? Let's consider deactivate_object first because that is an easy case to illustrate the problems in the current spec. Imagine a server which represents millions of database records by encapsulating them in CORBA Objects. A servant activator is installed to incarnate and etherealize the objects as necessary. The CORBA server essentially serves as a cache for the database. Objects are periodically deactivated to conserve memory which results in etherealize being called and their current state being written back to the database. For improved startup time, a number of objects are pre-activated when the server is started by explicitly calling activate_object rather than wait for the ServantManager to create them. Now let's look at the problems in the current spec and what happens to database consistency, because deactivate_object semantics are not strong enough. Let's say there is an active object, A, which was explicitly activated by calling activate_object. The servant associated with object A is currently processing several requests for A, and the server decides it needs to persist A to the database because resources are running low. A thread in the server will call deactivate_object passing the id for A as an argument. According to the spec, A will immediately be removed from the active object map, but will not be etheralized until all current requests for A have completed. So we now have requests executing in an object which is not in the active object map and has not been etherealized meaning, its state is not in the database. There are many nasty scenarios which can now occur: - A new request arrives for A. The object is not in the active object map, so incarnate() is called on the ServantActivator. (Note, this does not violate the rules for serialization of ServantActivators which state that incarnation may not overlap for objects which were incarnated by a ServantActivator because the original object A was activated explicitly using activate_object.) Because A is still in the server and has not been etherealized, a "stale" version of A will be incarnated from the database, which we will call A'. Eventually the requests executing in A will complete and A will be etherealized to the database. When A' is etherealized it will overwrite A. The database is now most likely completely inconsistent. - Now consider what happens to requests that are executing in A after A has been deactivated. If they call any POA operation which requires the use of the active object map (i.e. the RETAIN policy), they will get the incorrect result. For instance, if they call id_to_servant they may get ObjectNotActive, however if the A' in the above scenario has already been created they may get the Servant for A' which will most likely be different than the Servant for A. The results could be disastorous for the application. There are many variations on the above scenarios which can all occur because the apparent deactivation of an object occured before it should have. With the current model users will have to always be prepared to handle this strange behavior in all user code. This will make development of CORBA components which can be dynamically managed in memory by intelligent application servers very difficult if not impossible. Users will have to modify their code to handle the incosistent behavior and such modification will most likely be dependent on the particular server in which a CORBA component is being deployed. We may as well just give the market to Microsoft now, and not waste our time. All dramatics aside, this is a serious problem which needs to be addressed and very soon in my opinion. Now if it isn't clear already, the POA destroy operation has the exact same problem. If the server were to manage objects by destroying entire POAs instead of individual objects then it is possible to have a POA B and a POA B' in existence at the same time. And not only have a single object inconsistent but an entire set of objects managed by that POA totally inconsistent. I'm sure that will sell a lot of POA implementations.... So here are the requirements as we see them: 1. Etherealize may only be called for an object which has no currently executing requests. 2. Apparent destruction of a POA or deactivation of an object does not occur until all active requests in that POA or object have completed. 3. After destruction or deactivation has commenced no new requests can begin processing until destruction or deactivation has completed. 4. The system should not deadlock even in the presence of recursive calls across multiple servers. Our proposal satisifies all of the above requirements without changing any POA APIs or drastically changing behavior. In all cases the POA behavior is merely clarified and made more explicit so programs may be written in a portable fashion. Let's look at each in more detail: 1. Etheralization may only occur after all requests have completed. This is what the current spec states and we do not intend to change this behavior. (We take back what we said in our earlier proposal) 2. Apparent destruction/deactivation occurs after all requests have completed. Today the spec only states that apparent destruction occurs before etherealization is called. Since etherealization is called after all requests have completed, this change is consistent with the current spec, but adds a stronger requirement that apparent destruction must not occur until all active requests have completed. So this change strengthens the semantics of the spec, but does not change or weaken the current semantics. 3. After destruction or deactivation no new requests can begin. This is also consistent with the current spec, though it does result in a creation which is delayed because the apparent destruction (#2 above) may be delayed due to active requests. 4. Deadlocks should not occur. Now, I will be the first to admit that this is nearly impossible to prove in a complex distributed system. But I would like to eliminate the obvious ones. Our previous proposals could deadlock because two objects in two different processes could have a mutual recursion which could result in deadlock. For example object A could call object B in another server, object B then attempts to call object A again. However, right before object B calls object A another thread in object A's server attempts to deactivate A or destroy A's POA. This will result in deadlock because our previous proposal would act as if the POA's POAManager was in the holding state which will queue requests and hence block, resulting in deadlock. Our new proposal states that the object will behave as if it is in the discarding state which means it will not queue the request but instead throw the TRANSIENT exception. What a client ORB or application does with TRANSIENT is not currently defined in any CORBA spec, but it does say the request should be reissued. So an ORB may simply repeatedly reissue the request resulting in a livelock, or it may be intelligent and realize that after five minutes of receiving transients it may as well give up. This is implementation dependent but the important thing is that the ORB/application is deadlock free and whether or not it will make progress or livelock is an implementation decision that vendors may choose to make. Before anybody screams about this please remember that whether or not applications deadlock in CORBA today (even with the POA) is dependent on the server's thread model. A single threaded ORB will always deadlock in the above scenario because it can't handle distributed mutual recursion. A multi- threaded ORB with a fixed size thread pool will deadlock when it runs out of threads. All we are trying to accomplish is to allow a well written ORB/POA to not deadlock even in the most extreme cases. We believe this is possible, and the spec should allow for such implementations. Now there are a few improvements we can also make to help this new model out a little bit. For example, if an object is being deactivated but it has the policy USE_ACTIVE_OBJECT_MAP_ONLY (the default policy) then it is possible to actual behave as if the POA were in the inactive state and immediately throw OBJECT_NOT_EXIST. Similiarly if a POA is destroyed and its parent does not have an adapter activator then it is also possible to immediately return an OBJECT_NOT_EXIST because the POA will not be automatically created after it is destroyed and OBJECT_NOT_EXIST will be the result. This will handle a lot of the common cases and improve performance because the requests will not need to be reissued. As I said previously I will send out our proposals in separate messages. We would greatly appreciate feedback as soon as possible and would like to build consensus behind a solution to this problem this week be it our solution or any other proposed solution. Thanks. vGeorge Return-Path: Sender: "George Scott" Date: Tue, 14 Jul 1998 20:37:55 -0700 From: "George M. Scott" Organization: Inprise Corporation To: port-rtf@omg.org CC: gscott@inprise.com Subject: Proposal for issues 1408,1409 Fire away.... Issue 1408: POA destroy() is ill-defined Issue 1409: Multiple threads calling destroy() once destroy() has begun Nature: Revision Summary: POA destroy is not defined sufficiently enough to prevent multiple activations of the same POA name in the same process. POA::destroy semantics are not defined for multiple threads calling destroy. Resolution: Accepted for Corba RTF 2.3 Revision: The following text will replace the current describing the destroy operation in Section 9.3.8, Page 9-31. This operation destroys the POA and all descendant POAs. All descendant POAs are destroyed (recursively) before the destruction of the containing POA. The POA so destroyed (that is, the POA with its name) may be re-created later in the same process. (This differs from the POAManager::deactivate operation that does not allow a re-creation of its associated POA in the same process. After a deactivate, re-creation is allowed only if the POA is later destroyed.) When destroy is called the POA will behave as follows: 1. The POA will behave as if its POAManager is in the discarding state if the POA's parent has an AdapterActivator installed. Otherwise, the POA will behave as if its POAManager is in the inactive state. 2. The POA will call destroy on all of its children. 3. The apparent destruction of the POA will occur only after all currently executing requests in the POA and all of its children have completed. After destruction has become apparent the POA may be recreated via either an AdapterActivator or a call to create_POA. 4. If the etherealize_objects parameter is TRUE, the POA has the RETAIN policy, and a servant manager is registered with the POA, the etherealize operation on the servant manager will be called for each active object in the Active Object Map. The apparent destruction of the POA occurs before any calls to etherealize are made. Thus, for example, an etherealize method that attempts to invoke operations on the POA will receive the OBJECT_NOT_EXIST exception. {editorial note - below I have merged text from these two issues and the relevant text from issue 1428 because they change the same text. If one of these resolutions fails, we will need to edit the following text) The wait_for_completion parameter is handled as follows: - If wait_for_completion is TRUE and the current thread is not in an invocation context dispatched from any POA, the destroy operation will return only after all active requests have completed and all invocations of etherealize have completed. - If wait_for_completion is TRUE and the current thread is in an invocation context dispatched from any POA, then the BAD_INV_ORDER exception is thrown and POA destruction does not occur. - If wait_for_completion is FALSE, the destroy operation destroys the POA and its children but does not wait for active requests to complete nor for etherealization to occur. If destroy is called multiple times before destruction is complete (because there are active requests), the etherealize_objects parameter will only apply to the first call of destroy. Subsequent calls with conflicting etherealize_objects settings will use the value of the etherealize_objects from the first call. The wait_for_completion parameter will be handled as defined above for each individual call (some callers may choose to block, while others may not). Return-Path: Sender: jon@floorboard.com Date: Wed, 15 Jul 1998 14:24:29 -0700 From: Jonathan Biggar To: "George M. Scott" CC: port-rtf@omg.org Subject: Re: urgent POA issues References: <35AC23DC.7B2F1ECF@inprise.com> George M. Scott wrote: > > Unfortunately, I missed all of the exciting debate on the port-rtf > list last week, but I did notice a number of the most significant > problems (IMHO) did not reach resolution. > > I think it is fairly important that we reach resolution on these for > CORBA 2.3, as they fix a number of potentially serious problems that > affect portability of multi-threaded or recursive POA programs. It > seemed to me that we were very close to resolution and that if we > can > get agreement among the members of the POA RTF, we could then try > and > find a way to get the changes into CORBA 2.3 through some means > (It's > a shame that July is such a popular vacation month. ;-) ). > > The issues of concern are 1408, 1409, 1428, and 1627 (which was > mistakenly transferred to the C++ RTF IMHO). I intend to send out > concrete proposals for these issues in separate messages. If > necessary I will reopen 1627 as a new issue, since it is not a > language mapping issue, but a general POA issue. > > First let me summarize the problems and then provide some > discussion: > > 1428 - blocking POA operations. This issue relates to potential > deadlock situations in various ORB/POA calls which take a > wait_for_completion parameter. It was not clear to me why this did > not pass since it looked like there was consensus last Friday, and > it > should have been included in the RTF report. > > 1408/1409 - problems with destroy(). The behavior of destroy is not > defined explicitly enough to guarantee consistency of programs. There was not consensus on 1428, 1408/1409 due to the requirement that the POA act as if it were in the discarding state. Many RTF members want to find a solution that is more transparent to clients. > - A new request arrives for A. The object is not in the active object map, > so incarnate() is called on the ServantActivator. (Note, this does not > violate the rules for serialization of ServantActivators which state that > incarnation may not overlap for objects which were incarnated by a > ServantActivator because the original object A was activated explicitly > using activate_object.) Because A is still in the server and has not > been etherealized, a "stale" version of A will be incarnated from the > database, which we will call A'. Eventually the requests executing in > A will complete and A will be etherealized to the database. When A' > is etherealized it will overwrite A. The database is now most > likely completely inconsistent. It would be better to explicitly modify the text to state that when the POA uses a ServantActivator, that explicit calls to activate_object() and deactivate_object() are also serialied in the same way as incarnate() and etherealize() are. > - Now consider what happens to requests that are executing in A after > A has been deactivated. If they call any POA operation which requires > the use of the active object map (i.e. the RETAIN policy), they will > get the incorrect result. For instance, if they call id_to_servant > they may get ObjectNotActive, however if the A' in the above scenario > has already been created they may get the Servant for A' which will > most likely be different than the Servant for A. The results could > be disastorous for the application. No A' should exist due to my above comment. It is questionable whether this is a pracital scenario. Since the request is operating in the context of the servant already, why should the operation implementation need to call id_to_servant()? > There are many variations on the above scenarios which can all occur > because the apparent deactivation of an object occured before it > should have. With the current model users will have to always be > prepared to handle this strange behavior in all user code. This > will > make development of CORBA components which can be dynamically > managed > in memory by intelligent application servers very difficult if not > impossible. Users will have to modify their code to handle the > incosistent behavior and such modification will most likely be > dependent on the particular server in which a CORBA component is > being > deployed. We may as well just give the market to Microsoft now, and > not waste our time. All dramatics aside, this is a serious problem > which > needs to be addressed and very soon in my opinion. A bit heavy on the retoric, don't you think? > Let's look at each in more detail: > > 1. Etheralization may only occur after all requests have completed. > This is > what the current spec states and we do not intend to change this > behavior. > (We take back what we said in our earlier proposal) Fine. > 2. Apparent destruction/deactivation occurs after all requests have completed. > Today the spec only states that apparent destruction occurs before > etherealization is called. Since etherealization is called after all > requests have completed, this change is consistent with the current > spec, but adds a stronger requirement that apparent destruction must not > occur until all active requests have completed. So this change strengthens > the semantics of the spec, but does not change or weaken the current > semantics. Right. > 3. After destruction or deactivation no new requests can begin. This is > also consistent with the current spec, though it does result in a creation > which is delayed because the apparent destruction (#2 above) may be > delayed due to active requests. Right. > 4. Deadlocks should not occur. Now, I will be the first to admit that this > is nearly impossible to prove in a complex distributed system. But I would > like to eliminate the obvious ones. Our previous proposals could deadlock > because two objects in two different processes could have a mutual > recursion which could result in deadlock. For example object A could call > object B in another server, object B then attempts to call object A again. > However, right before object B calls object A another thread in object > A's server attempts to deactivate A or destroy A's POA. This will result > in deadlock because our previous proposal would act as if the POA's > POAManager was in the holding state which will queue requests and hence > block, resulting in deadlock. Our new proposal states that the object > will behave as if it is in the discarding state which means it will not > queue the request but instead throw the TRANSIENT exception. > > What a client ORB or application does with TRANSIENT is not currently > defined in any CORBA spec, but it does say the request should be reissued. > So an ORB may simply repeatedly reissue the request resulting in a > livelock, or it may be intelligent and realize that after five minutes > of receiving transients it may as well give up. True, it is not explicitly stated, but the weight of the evidence suggests that an ORB is supposed to make the TRANSIENT exception visible to client code. A more general question is whether this tightly coupled design (A calls B which calls A) is desireable in the first place. I think most designers would see this as a trouble spot right away, and redesign the system to use an event channel or oneway call to resolve the deadlock. > This is implementation > dependent but the important thing is that the ORB/application is > deadlock > free and whether or not it will make progress or livelock is an > implementation decision that vendors may choose to make. Before > anybody > screams about this please remember that whether or not > applications > deadlock in CORBA today (even with the POA) is dependent on the > server's > thread model. A single threaded ORB will always deadlock in the > above > scenario because it can't handle distributed mutual recursion. This is not necessarily true for single threaded servers. Some can handle recursive dispatch of requests while blocked waiting for a remote invocation. > A multi- > threaded ORB with a fixed size thread pool will deadlock when it > runs out > of threads. All we are trying to accomplish is to allow a well > written > ORB/POA to not deadlock even in the most extreme cases. We > believe this > is possible, and the spec should allow for such implementations. A laudable goal. > Now there are a few improvements we can also make to help this new model > out a little bit. For example, if an object is being deactivated but > it has the policy USE_ACTIVE_OBJECT_MAP_ONLY (the default policy) then > it is possible to actual behave as if the POA were in the inactive > state and immediately throw OBJECT_NOT_EXIST. True, this would be a useful deadlock avoidance technique. > Similiarly if a POA is destroyed and its parent does not have an > adapter activator then it is also possible to immediately return > an > OBJECT_NOT_EXIST because the POA will not be automatically > created after > it is destroyed and OBJECT_NOT_EXIST will be the result. Also valid. > This will handle a lot of the common cases and improve performance because > the requests will not need to be reissued. Again, the weight of the evidence does not suggest that an ORB implementation should be free to intercept TRANSIENT exceptions and reissue them transparently for the client. -- Jon Biggar Floorboard Software jon@floorboard.com jon@biggar.org Return-Path: Date: Wed, 15 Jul 1998 18:54:35 -0400 From: Paul H Kyzivat Organization: NobleNet To: "George M. Scott" CC: port-rtf@omg.org Subject: Re: urgent POA issues References: <35AC23DC.7B2F1ECF@inprise.com> George M. Scott wrote: > Imagine a server which represents millions of database records by > encapsulating them in CORBA Objects. A servant activator is > installed > to incarnate and etherealize the objects as necessary. The CORBA > server essentially serves as a cache for the database. Objects are > periodically deactivated to conserve memory which results in > etherealize being called and their current state being written back > to > the database. For improved startup time, a number of objects are > pre-activated when the server is started by explicitly calling > activate_object rather than wait for the ServantManager to create > them. I presume you are assuming a UNIQUE_ID policy so that a given servant is associated with at most one object. > > Now let's look at the problems in the current spec and what happens > to > database consistency, because deactivate_object semantics are not > strong enough. Let's say there is an active object, A, which was > explicitly activated by calling activate_object. The servant > associated with object A is currently processing several requests > for > A, and the server decides it needs to persist A to the database > because resources are running low. A thread in the server will call > deactivate_object passing the id for A as an argument. According to > the spec, A will immediately be removed from the active object map, > but will not be etheralized until all current requests for A have > completed. So we now have requests executing in an object which is > not in the active object map and has not been etherealized meaning, > its state is not in the database. There are many nasty scenarios > which can now occur: > > - A new request arrives for A. The object is not in the active > object map, > so incarnate() is called on the ServantActivator. (Note, this > does > not > violate the rules for serialization of ServantActivators which > state that > incarnation may not overlap for objects which were incarnated by > a > ServantActivator because the original object A was activated > explicitly > using activate_object.) Because A is still in the server and has > not > been etherealized, a "stale" version of A will be incarnated from > the > database, which we will call A'. Eventually the requests > executing > in > A will complete and A will be etherealized to the database. When > A' > is etherealized it will overwrite A. The database is now most > likely completely inconsistent. OK, I think I see your problem. The spec implied to me (although it isn't explicit) that the serialization of incarnate and etherealize must extent to stalling a subsequent incarnation while an etherealization is pending for the same objectId. This is different than requiring the object to remain active until the etherealization begins - tests for the presence of the object can still fail (because it has been deactivated) and an explict activation is OK. > > - Now consider what happens to requests that are executing in A > after > A has been deactivated. If they call any POA operation which > requires > the use of the active object map (i.e. the RETAIN policy), they > will > get the incorrect result. For instance, if they call > id_to_servant > they may get ObjectNotActive, however if the A' in the above > scenario > has already been created they may get the Servant for A' which > will > most likely be different than the Servant for A. The results > could > be disastorous for the application. Given the interpretation I gave above, there would be no A' while there are requests executing in A. It indeed might be the case that id_to_servant would fail. This has nothing to do with the timing of etherealization - it only has to do with explicit multithreading code written by the developer. Why should it succeed? It is unwise to be be writing code that depends on activations in a POA while at the same time writing concurrent code that removes them. And this isn't a hard thing to avoid in this case since the servant ought to already know who it is, or it can find out from POACurrent. > > There are many variations on the above scenarios which can all occur > because the apparent deactivation of an object occured before it > should have. With the current model users will have to always be > prepared to handle this strange behavior in all user code. This > will > make development of CORBA components which can be dynamically > managed > in memory by intelligent application servers very difficult if not > impossible. There are (at least) three things going on here: - removal of the oid:servant association from the map - etherealization of the oid:servant association - deletion of the servant No matter what we do, all of these need to be dealt with, and these may all occur at (more-or-less) the same time, or at widely spaced times, or not at all, in various combinations. The existing policies make some combinations easier to use than others. You are not going to get agreement about when these things "should" occur because there is no one answer. You seem to think that one particular combination of interest to you is not currently supported (or at least isn't easily used), and are also proposing that another combination that is currently allowed is bad or useless and should be replaced by the one you want. These things should be separate considerations. If some feature is without merit then we should perhaps consider removing it. If what you want is not currently possible AND is important then we should consider adding it. > Users will have to modify their code to handle the > incosistent behavior and such modification will most likely be > dependent on the particular server in which a CORBA component is > being > deployed. We may as well just give the market to Microsoft now, and > not waste our time. All dramatics aside, this is a serious problem > which > needs to be addressed and very soon in my opinion. Inconsistency is yet another issue. I agree that we should be clear about what the user can expect, and that this contract should be sufficient for use. On the other hand, one of the most important (and hardest) things a standard must do is decide what does not need to be standardized. There must not be needless restrictions that prevent ORB implementors from making implementation decisions. It is perfectly acceptable for a standard to say that some aspect of behavior is explicitly undefined. The problems come when some people think a behavior is defined and others think it is not. > > Now if it isn't clear already, the POA destroy operation has the > exact > same > problem. If the server were to manage objects by destroying entire > POAs > instead of individual objects then it is possible to have a POA B > and > a POA > B' in existence at the same time. And not only have a single object > inconsistent but an entire set of objects managed by that POA > totally > inconsistent. I'm sure that will sell a lot of POA > implementations.... The analogy isn't perfect. In the case of POAs, the only thing that is ever explicitly visible is a reference to a POA - there are no visible POA servants (though they may exist in a particular ORB implementation.) Just as a regular object reference remains invariant though it may map to different servants from time to time, a reference to a POA should also remain invariant though it may refer to a different incarnation of that POA from time to time. So it should never appear as if there is both a B and a B'. > > So here are the requirements as we see them: [snip] > Our proposal satisifies all of the above requirements without > changing > any POA APIs or drastically changing behavior. In all cases the POA > behavior is merely clarified and made more explicit so programs may > be > written in a portable fashion. > > Let's look at each in more detail: > > 1. Etheralization may only occur after all requests have completed. > This is > what the current spec states and we do not intend to change this > behavior. > (We take back what we said in our earlier proposal) No problem here. > > 2. Apparent destruction/deactivation occurs after all requests have > completed. > Today the spec only states that apparent destruction occurs > before > etherealization is called. Since etherealization is called after > all > requests have completed, this change is consistent with the > current > spec, but adds a stronger requirement that apparent destruction > must not > occur until all active requests have completed. So this change > strengthens > the semantics of the spec, but does not change or weaken the > current > semantics. I agree for the POA because it is the *implementation* of the POA that mediates requests and that is being destroyed. I disagree for deactivation for reasons stated here and in other replies. > > 3. After destruction or deactivation no new requests can begin. > This > is > also consistent with the current spec, though it does result in a > creation > which is delayed because the apparent destruction (#2 above) may > be > delayed due to active requests. In the case of POA destruction I agree. In the case of deactivation see my comments elsewhere. > > 4. Deadlocks should not occur. Now, I will be the first to admit > that > this > is nearly impossible to prove in a complex distributed system. This is a nice goal, but it is unachievable. Deadlock detection would at best be very expensive. Deadlock prevention in all cases requires imposition of rules that prevent reasonable usage. I think all we can do is ensure that deadlocks are not a direct consequence of reasonable and common usage, and make it relatively easy for a developer to figure out what kinds of behavior might lead to deadlock. In practical cases it is pretty easy to avoid deadlock for the particular circumstances of any specific server. > But I would > like to eliminate the obvious ones. Our previous proposals could > deadlock > because two objects in two different processes could have a > mutual > recursion which could result in deadlock. For example object A > could call > object B in another server, object B then attempts to call object > A > again. > However, right before object B calls object A another thread in > object > A's server attempts to deactivate A or destroy A's POA. This > will > result > in deadlock because our previous proposal would act as if the > POA's > POAManager was in the holding state which will queue requests and > hence > block, resulting in deadlock. Our new proposal states that the > object > will behave as if it is in the discarding state which means it > will > not > queue the request but instead throw the TRANSIENT exception. I don't think deadlocks of this sort are a big issue, because hardly anyone is going to destroy POAs dynamically and to do so in conjunction with mutual recursion would be remarkable. That said, I would rather risk the deadlock than throw away the requests. If somebody wants this kind of behavior they can first set the POAs POAManager to discarding and then destroy the POA. > > What a client ORB or application does with TRANSIENT is not > currently > defined in any CORBA spec, but it does say the request should be > reissued. > So an ORB may simply repeatedly reissue the request resulting in > a > livelock, or it may be intelligent and realize that after five > minutes > of receiving transients it may as well give up. This is I don't think the spec mandates that an ORB should automatically retry a request when a transient exception is received. I don't even think it is a good idea. Unless or until it is clear that this is required behavior we should assume that an ORB client is going to get the exception and have to deal with it explicitly - it is going to bother somebody. > implementation > dependent but the important thing is that the ORB/application is > deadlock > free and whether or not it will make progress or livelock is an > implementation decision that vendors may choose to make. Before > anybody > screams about this please remember that whether or not > applications > deadlock in CORBA today (even with the POA) is dependent on the > server's > thread model. A single threaded ORB will always deadlock in the > above > scenario because it can't handle distributed mutual recursion. A > multi- > threaded ORB with a fixed size thread pool will deadlock when it > runs out > of threads. > All we are trying to accomplish is to allow a well > written > ORB/POA to not deadlock even in the most extreme cases. We > believe > this > is possible, and the spec should allow for such implementations. Explicit deactivation of the POAManager prior to destruction will handle this. > > Now there are a few improvements we can also make to help this > new > model > out a little bit. For example, if an object is being deactivated > but > it has the policy USE_ACTIVE_OBJECT_MAP_ONLY (the default policy) > then > it is possible to actual behave as if the POA were in the > inactive > state and immediately throw OBJECT_NOT_EXIST. > > Similiarly if a POA is destroyed and its parent does not have an > adapter activator then it is also possible to immediately return > an > OBJECT_NOT_EXIST because the POA will not be automatically > created > after > it is destroyed and OBJECT_NOT_EXIST will be the result. > > This will handle a lot of the common cases and improve > performance > because > the requests will not need to be reissued. > > As I said previously I will send out our proposals in separate > messages. > We would greatly appreciate feedback as soon as possible and would > like to > build consensus behind a solution to this problem this week be it > our > solution or any other proposed solution. I will try to reply to the specific proposals, but consider this message as context and partial reply to them. Return-Path: Date: Wed, 15 Jul 1998 19:20:33 -0400 From: Paul H Kyzivat Organization: NobleNet To: "George M. Scott" CC: port-rtf@omg.org Subject: Re: Proposal for issues 1408,1409 References: <35AC2413.B8E8BB4A@inprise.com> George M. Scott wrote: > > Fire away.... > > Issue 1408: POA destroy() is ill-defined > Issue 1409: Multiple threads calling destroy() once destroy() > has begun > Nature: Revision > > Summary: POA destroy is not defined sufficiently enough to > prevent multiple activations of the same POA name in > the same process. POA::destroy semantics are not defined > for multiple threads calling destroy. > > Resolution: Accepted for Corba RTF 2.3 > > Revision: The following text will replace the current describing > the destroy operation in Section 9.3.8, Page 9-31. > > This operation destroys the POA and all descendant POAs. All > descendant POAs are destroyed (recursively) before the destruction > of > the containing POA. The POA so destroyed (that is, the POA with its > name) may be re-created later in the same process. (This differs > from > the POAManager::deactivate operation that does not allow a > re-creation > of its associated POA in the same process. After a deactivate, > re-creation is allowed only if the POA is later destroyed.) > > When destroy is called the POA will behave as follows: > > 1. The POA will behave as if its POAManager is in the discarding > state > if > the POA's parent has an AdapterActivator installed. Otherwise, > the > POA > will behave as if its POAManager is in the inactive state. This second guessing adds complexity to the implementation and just makes it harder for the user to understand. I think the default should be to act as if HOLDING, so that nothing is lost by default. If something else is desired, the user can set the state explicitly prior to destruction. But I would prefer to pick one state, even if it is DISCARDING, rather than have a more complex rule. > > 2. The POA will call destroy on all of its children. > Doing this step after (1) pretty much guarantees that you can't successfuly delete the root POA and get any etherealization. This means either that ORB shutdown never results in etherealization or else it must delete the POAs using some more complex process than just deleting the root POA. I think it is *essential* that shutdown do successful etherealization. If some process for deleting a tree of POAs must exist for this, then I think that same process should be used whenever deleting a POA. The only way I know to do this is to leave the POA in full working order while *waiting* for the deletion of its children to complete (including their etherealization). If the call to destroy is with wait=false, then this means that destroy may return with the POA still in working order. This is unpleasant, but I don't see any other way. Even this isn't a complete solution - nor is there any complete solution. (I spent a lot of time thinking about this one!) It is reasonable to imagine a POA with its servant activator activated in a peer POA; and possible (tho not reasonable) for it to have its servant activator activated in a child POA. Some of these weird cases could be made to work by building a dependency graph among the POAs and sorting it to decide the order in which to do destruction. But this is a lot of work and there is no guarantee that an ordering exists. I think it is sufficient to guarantee that etherealization will be successfully called if a POA activates its servant activator in some POA that lies on the path between it and the root. (Personally I think ServantManagers and AdapterActivators should have been servants rather than objects. That would have eliminated this problem. But too late for that.) > 3. The apparent destruction of the POA will occur only after all > currently > executing requests in the POA and all of its children have > completed. > After destruction has become apparent the POA may be recreated > via > either an AdapterActivator or a call to create_POA. Also can't become apparent until etherealization of children is complete. > > 4. If the etherealize_objects parameter is TRUE, the POA has the > RETAIN > policy, and a servant manager is registered with the POA, the > etherealize operation on the servant manager will be called for > each > active object in the Active Object Map. The apparent destruction > of the > POA occurs before any calls to etherealize are made. Thus, for > example, > an etherealize method that attempts to invoke operations on the > POA > will > receive the OBJECT_NOT_EXIST exception. Subject to the comments above this seems ok. > > {editorial note - below I have merged text from these two issues and > the relevant text from issue 1428 because they change the same text. > If one of these resolutions fails, we will need to edit the > following > text) > > The wait_for_completion parameter is handled as follows: > > - If wait_for_completion is TRUE and the current > thread is not in an invocation context dispatched from any POA, > the destroy operation will return only after all active requests > have completed and all invocations of etherealize have > completed. > > - If wait_for_completion is TRUE and the current thread is in an > invocation context dispatched from any POA, then the > BAD_INV_ORDER > exception is thrown and POA destruction does not occur. > > - If wait_for_completion is FALSE, the destroy operation > destroys the POA and its children but does not wait for active > requests to complete nor for etherealization to occur. This is ok except as I noted above, the POA may not yet appear to have been destroyed at all on return. > > If destroy is called multiple times before destruction is > complete (because there are active requests), the > etherealize_objects parameter will only apply to the first > call of destroy. Subsequent calls with conflicting > etherealize_objects settings will use the value of the > etherealize_objects from the first call. The wait_for_completion > parameter will be handled as defined above for each individual > call (some callers may choose to block, while others may not). This seems ok. Return-Path: Sender: "George Scott" Date: Wed, 15 Jul 1998 21:24:07 -0700 From: "George M. Scott" Organization: Inprise Corporation To: Jonathan Biggar CC: port-rtf@omg.org Subject: Re: urgent POA issues References: <35AC23DC.7B2F1ECF@inprise.com> <35AD1E0D.5D3BF7C@floorboard.com> Jonathan Biggar wrote: > > George M. Scott wrote: > > > > > > 1408/1409 - problems with destroy(). The behavior of destroy is > not > > defined explicitly enough to guarantee consistency of programs. > > There was not consensus on 1428, 1408/1409 due to the requirement > that > the POA act as if it were in the discarding state. Many RTF members > want to find a solution that is more transparent to clients. OK, we have done more thinking on this and believe we can live with the holding state as I originally proposed more than a week ago. However, we would like to keep the escapes I describe in the proposal. That is, if there is no chance of the object being automatically reactivated or the POA being automatically recreated there is no need to hold, and OBJECT_NOT_EXIST can be returned immediately. This will help eliminate some deadlock situations. Also, it is possible for a client ORB to timeout a request if it has not received a reply for some period of time and issue a cancel request. In which case, deadlock would be avoided after some delay for the timeout. So if a vendor were to choose to provide such a feature in their product deadlock can be avoided. Also, we do consider the deadlock described to be a rare occurence and can be documented. > > - A new request arrives for A. The object is not in the active object map, > > so incarnate() is called on the ServantActivator. (Note, this does not > > violate the rules for serialization of ServantActivators which state that > > incarnation may not overlap for objects which were incarnated by a > > ServantActivator because the original object A was activated explicitly > > using activate_object.) Because A is still in the server and has not > > been etherealized, a "stale" version of A will be incarnated from the > > database, which we will call A'. Eventually the requests executing in > > A will complete and A will be etherealized to the database. When A' > > is etherealized it will overwrite A. The database is now most > > likely completely inconsistent. > > It would be better to explicitly modify the text to state that when the > POA uses a ServantActivator, that explicit calls to activate_object() > and deactivate_object() are also serialied in the same way as > incarnate() and etherealize() are. I prefer not to add any additional serialization to such calls as they introduce additional blocking which only adds more potential deadlock situations. > > > - Now consider what happens to requests that are executing in A > after > > A has been deactivated. If they call any POA operation which > requires > > the use of the active object map (i.e. the RETAIN policy), they > will > > get the incorrect result. For instance, if they call > id_to_servant > > they may get ObjectNotActive, however if the A' in the above > scenario > > has already been created they may get the Servant for A' which > will > > most likely be different than the Servant for A. The results > could > > be disastorous for the application. > > No A' should exist due to my above comment. It is questionable > whether > this is a pracital scenario. Since the request is operating in the > context of the servant already, why should the operation > implementation > need to call id_to_servant()? see my previous posting. > > > 4. Deadlocks should not occur. Now, I will be the first to admit > that this > > is nearly impossible to prove in a complex distributed system. > But I would > > like to eliminate the obvious ones. Our previous proposals > could deadlock > > because two objects in two different processes could have a > mutual > > recursion which could result in deadlock. For example object A > could call > > object B in another server, object B then attempts to call > object A again. > > However, right before object B calls object A another thread in > object > > A's server attempts to deactivate A or destroy A's POA. This > will result > > in deadlock because our previous proposal would act as if the > POA's > > POAManager was in the holding state which will queue requests > and hence > > block, resulting in deadlock. Our new proposal states that the > object > > will behave as if it is in the discarding state which means it > will not > > queue the request but instead throw the TRANSIENT exception. > > > > What a client ORB or application does with TRANSIENT is not > currently > > defined in any CORBA spec, but it does say the request should > be reissued. > > So an ORB may simply repeatedly reissue the request resulting > in a > > livelock, or it may be intelligent and realize that after five > minutes > > of receiving transients it may as well give up. > > True, it is not explicitly stated, but the weight of the evidence > suggests that an ORB is supposed to make the TRANSIENT exception > visible > to client code. Ok, since it turns out the TRANSIENT issue is a little harder for people to swallow than I expected. We will go back to the holding state as I disucssed above. > A more general question is whether this tightly coupled design (A calls > B which calls A) is desireable in the first place. I think most > designers would see this as a trouble spot right away, and redesign the > system to use an event channel or oneway call to resolve the deadlock. Yes, this is probably true. Again, as we have already discussed this is a rare deadlock situation and one I think we can live with. George Return-Path: Sender: "George Scott" Date: Wed, 15 Jul 1998 21:41:37 -0700 From: "George M. Scott" Organization: Inprise Corporation To: Paul H Kyzivat CC: port-rtf@omg.org Subject: Re: Proposal for issues 1408,1409 References: <35AC2413.B8E8BB4A@inprise.com> <35AD3940.6642DAF5@noblenet.com> Paul H Kyzivat wrote: > > George M. Scott wrote: > > > > > When destroy is called the POA will behave as follows: > > > > 1. The POA will behave as if its POAManager is in the discarding > state > > if > > the POA's parent has an AdapterActivator installed. Otherwise, > the > > POA > > will behave as if its POAManager is in the inactive state. > > This second guessing adds complexity to the implementation and just > makes it harder for the user to understand. > > I think the default should be to act as if HOLDING, so that nothing > is > lost by default. If something else is desired, the user can set the > state explicitly prior to destruction. But I would prefer to pick > one > state, even if it is DISCARDING, rather than have a more complex > rule. I agree it adds complexity, but it does eliminate some deadlock situations. So I would prefer to keep it in, but I will agree with whatever the consensus is on this one, since it is not worth holding back the entire proposal for this one point. > > > > > 2. The POA will call destroy on all of its children. > > > > Doing this step after (1) pretty much guarantees that you can't > successfuly delete the root POA and get any etherealization. This > means > either that ORB shutdown never results in etherealization or else it > must delete the POAs using some more complex process than just > deleting > the root POA. > > I think it is *essential* that shutdown do successful > etherealization. > If some process for deleting a tree of POAs must exist for this, > then I > think that same process should be used whenever deleting a POA. > > The only way I know to do this is to leave the POA in full working > order > while *waiting* for the deletion of its children to complete > (including > their etherealization). If the call to destroy is with wait=false, > then > this means that destroy may return with the POA still in working > order. > > This is unpleasant, but I don't see any other way. Yes, I see your point. It looks like this may be required, for correct operation. I will ponder this one some more. This is particularly nasty.... > Even this isn't a complete solution - nor is there any complete > solution. (I spent a lot of time thinking about this one!) It is > reasonable to imagine a POA with its servant activator activated in > a > peer POA; and possible (tho not reasonable) for it to have its > servant > activator activated in a child POA. Some of these weird cases could > be > made to work by building a dependency graph among the POAs and > sorting > it to decide the order in which to do destruction. But this is a lot > of > work and there is no guarantee that an ordering exists. > > I think it is sufficient to guarantee that etherealization will be > successfully called if a POA activates its servant activator in some > POA > that lies on the path between it and the root. > > (Personally I think ServantManagers and AdapterActivators should > have > been servants rather than objects. That would have eliminated this > problem. But too late for that.) Yes, making the ServantManager and AdapterActivators "special" would have helped things out tremendously here. But not much we can do about this one. > > 3. The apparent destruction of the POA will occur only after all > > currently > > executing requests in the POA and all of its children have > > completed. > > After destruction has become apparent the POA may be recreated > via > > either an AdapterActivator or a call to create_POA. > > Also can't become apparent until etherealization of children is > complete. v This would contradict what the spec currently states. George Return-Path: Sender: "George Scott" Date: Wed, 15 Jul 1998 22:08:14 -0700 From: "George M. Scott" Organization: Inprise Corporation To: Paul H Kyzivat CC: port-rtf@omg.org Subject: Re: urgent POA issues References: <35AC23DC.7B2F1ECF@inprise.com> <35AD332B.F8D71F57@noblenet.com> Paul H Kyzivat wrote: > > George M. Scott wrote: > > > Imagine a server which represents millions of database records by > > encapsulating them in CORBA Objects. A servant activator is > installed > > to incarnate and etherealize the objects as necessary. The CORBA > > server essentially serves as a cache for the database. Objects are > > periodically deactivated to conserve memory which results in > > etherealize being called and their current state being written > back to > > the database. For improved startup time, a number of objects are > > pre-activated when the server is started by explicitly calling > > activate_object rather than wait for the ServantManager to create > > them. > > I presume you are assuming a UNIQUE_ID policy so that a given > servant is > associated with at most one object. Yes. > > > > - A new request arrives for A. The object is not in the active > > object map, > > so incarnate() is called on the ServantActivator. (Note, this > does > > not > > violate the rules for serialization of ServantActivators which > > state that > > incarnation may not overlap for objects which were incarnated > by a > > ServantActivator because the original object A was activated > > explicitly > > using activate_object.) Because A is still in the server and > has > > not > > been etherealized, a "stale" version of A will be incarnated > from > > the > > database, which we will call A'. Eventually the requests > executing > > in > > A will complete and A will be etherealized to the database. > When > > A' > > is etherealized it will overwrite A. The database is now most > > likely completely inconsistent. > > OK, I think I see your problem. The spec implied to me (although it > isn't explicit) that the serialization of incarnate and etherealize > must > extent to stalling a subsequent incarnation while an etherealization > is > pending for the same objectId. > > This is different than requiring the object to remain active until > the > etherealization begins - tests for the presence of the object can > still > fail (because it has been deactivated) and an explict activation is > OK. No, because a user can also call activate_object and not use incarnate and this will result in two copies of an object being created. > > > > > - Now consider what happens to requests that are executing in A > after > > A has been deactivated. If they call any POA operation which > > requires > > the use of the active object map (i.e. the RETAIN policy), they > > will > > get the incorrect result. For instance, if they call > id_to_servant > > they may get ObjectNotActive, however if the A' in the above > > scenario > > has already been created they may get the Servant for A' which > will > > most likely be different than the Servant for A. The results > could > > be disastorous for the application. > > Given the interpretation I gave above, there would be no A' while > there > are requests executing in A. No, because even outside of incarnate a user can call activate_object after deactivate_object and have multiple copies running. > It indeed might be the case that id_to_servant would fail. This has > nothing to do with the timing of etherealization - it only has to do > with explicit multithreading code written by the developer. Why > should > it succeed? It is unwise to be be writing code that depends on > activations in a POA while at the same time writing concurrent code > that > removes them. And this isn't a hard thing to avoid in this case > since > the servant ought to already know who it is, or it can find out from > POACurrent. Several members of the OMG are currently working on a proposal for CORBA components. As part of this proposal users will write components (servants) which implement some business logic or other functions. These components will be deployed in a container (POA + extra code) which will activate/deactivate objects as necessary. So the user will actually have little control over the concurrency of calls executing while an object is being deactivated because they are not writing the entire server, only a single servant which is running inside of a sophisticated component server. It is not safe to assume that a single person wrote all code which runs inside of a server process. > > There are many variations on the above scenarios which can all occur > > because the apparent deactivation of an object occured before it > > should have. With the current model users will have to always be > > prepared to handle this strange behavior in all user code. This will > > make development of CORBA components which can be dynamically managed > > in memory by intelligent application servers very difficult if not > > impossible. > > There are (at least) three things going on here: > - removal of the oid:servant association from the map > - etherealization of the oid:servant association > - deletion of the servant deletion of the servant is a language mapping issue which can be handled by garbage collection or proper reference counting. > No matter what we do, all of these need to be dealt with, > and these may all occur at (more-or-less) the same time, or > at widely spaced times, or not at all, in various combinations. > The existing policies make some combinations easier to use than > others. > > You are not going to get agreement about when these things "should" > occur because there is no one answer. > > You seem to think that one particular combination of interest to you > is > not currently supported (or at least isn't easily used), and are > also > proposing that another combination that is currently allowed is bad > or > useless and should be replaced by the one you want. There is only one combination and that is the one you have listed. It is only the timing I question. And I only question the timing if there are requests executing. The current spec already states that etherealization cannot occur until there are no requests executing, so the timing of etherealization and servant destruction are already fixed in the spec. So I'm trying to change the timing of only one element. A farily minor change and not as ominous as you make it sound. > These things should be separate considerations. If some feature is > without merit then we should perhaps consider removing it. If what > you > want is not currently possible AND is important then we should > consider > adding it. Yes, what I want is consistency, and it is not currently possible, so I would hope we would consider adding it. > > Users will have to modify their code to handle the > > incosistent behavior and such modification will most likely be > > dependent on the particular server in which a CORBA component is > being > > deployed. We may as well just give the market to Microsoft now, > and > > not waste our time. All dramatics aside, this is a serious > problem > > which > > needs to be addressed and very soon in my opinion. > > Inconsistency is yet another issue. I agree that we should be clear > about what the user can expect, and that this contract should be > sufficient for use. On the other hand, one of the most important > (and > hardest) things a standard must do is decide what does not need to > be > standardized. There must not be needless restrictions that prevent > ORB > implementors from making implementation decisions. It is perfectly > acceptable for a standard to say that some aspect of behavior is > explicitly undefined. Yes, but undefined behavior was previously spelled BOA. We are now trying to define behavior so programs can be written in a portable fashion. I agree there are certain things that can remain undefined and I don't think anything I have proposed is forcing any vendor to use a particular implementation. It is merely adding stronger semantics to the spec, which a vendor must implement, but there are still many possible implementations. > The problems come when some people think a behavior is defined and > others think it is not. Agreed. > > Now if it isn't clear already, the POA destroy operation has the exact > > same > > problem. If the server were to manage objects by destroying entire > > POAs > > instead of individual objects then it is possible to have a POA B and > > a POA > > B' in existence at the same time. And not only have a single object > > inconsistent but an entire set of objects managed by that POA totally > > inconsistent. I'm sure that will sell a lot of POA > > implementations.... > > The analogy isn't perfect. In the case of POAs, the only thing that is > ever explicitly visible is a reference to a POA - there are no visible > POA servants (though they may exist in a particular ORB implementation.) > > Just as a regular object reference remains invariant though it may map > to different servants from time to time, a reference to a POA should > also remain invariant though it may refer to a different incarnation of > that POA from time to time. So it should never appear as if there is > both a B and a B'. I'm confused by your statement. Are you saying that a reference to a POA that is destroyed and then is later recreated is still a valid reference? I don't think this is the case. And if it is the case, then this should be very explicitly stated in the spec. For example: Imagine the rootPOA is available and has an adapter activator installed which can create a POA named "foo". Executing the following code will activate the POA, then destroy it. // Java POA a = rootPOA.find_POA("foo", true); a.destroy(false, false); Now are you stating that if I now invoke an operation on "a", that it will recreate the POA? For example: a.the_name(); will actually reactivate the POA? Or will it throw OBJECT_NOT_EXIST. It was my understanding that a destroyed POA will always throw OBJECT_NOT_EXIST when invoked, and recreating the same POA will not result in the old reference suddenly becoming valid again. > > > But I would > > like to eliminate the obvious ones. Our previous proposals > could > > deadlock > > because two objects in two different processes could have a > mutual > > recursion which could result in deadlock. For example object A > > could call > > object B in another server, object B then attempts to call > object A > > again. > > However, right before object B calls object A another thread in > > object > > A's server attempts to deactivate A or destroy A's POA. This > will > > result > > in deadlock because our previous proposal would act as if the > POA's > > POAManager was in the holding state which will queue requests > and > > hence > > block, resulting in deadlock. Our new proposal states that the > > object > > will behave as if it is in the discarding state which means it > will > > not > > queue the request but instead throw the TRANSIENT exception. > > I don't think deadlocks of this sort are a big issue, because hardly > anyone is going to destroy POAs dynamically and to do so in > conjunction > with mutual recursion would be remarkable. > > That said, I would rather risk the deadlock than throw away the > requests. If somebody wants this kind of behavior they can first set > the > POAs POAManager to discarding and then destroy the POA. Yes, I think we all agree this is probably a rare case. Though, I know one day some end user of CORBA will hit this problem and we'll hear about it. > > What a client ORB or application does with TRANSIENT is not > > currently > > defined in any CORBA spec, but it does say the request should > be > > reissued. > > So an ORB may simply repeatedly reissue the request resulting > in a > > livelock, or it may be intelligent and realize that after five > > minutes > > of receiving transients it may as well give up. This is > > I don't think the spec mandates that an ORB should automatically > retry a > request when a transient exception is received. I don't even think > it is > a good idea. Unless or until it is clear that this is required > behavior > we should assume that an ORB client is going to get the exception > and > have to deal with it explicitly - it is going to bother somebody. Yes, I have received quite a bit of feedback on what people think TRANSIENT means, so I'm going to propose using the holding state instead. George Return-Path: Sender: jon@floorboard.com Date: Wed, 15 Jul 1998 22:44:50 -0700 From: Jonathan Biggar To: "George M. Scott" CC: Paul H Kyzivat , port-rtf@omg.org Subject: Re: Proposal for issues 1408,1409 References: <35AC2413.B8E8BB4A@inprise.com> <35AD3940.6642DAF5@noblenet.com> <35AD8481.2B30E4F8@inprise.com> George M. Scott wrote: > > (Personally I think ServantManagers and AdapterActivators should > have > > been servants rather than objects. That would have eliminated this > > problem. But too late for that.) > > Yes, making the ServantManager and AdapterActivators "special" would > have helped things out tremendously here. But not much we can do > about this one. Well, we could still do this. Just add text that says that the POA extracts the servant of the ServantManager and AdapterActivator references it receives and interacts directly with the servant, rather than being mediated by the POA associated with those references. This would require of course that things like POA::Current would not work in incarnate() (and other) calls, but that might not be too much of a price to pay. Hand in hand with solving the servant memory management issue, this would reduce the number of deadlock and premature death scenarios considerably. Given the other number of deadlocks involved with having these references mediated by a POA whose POAManager is currently in the holding or discarding state anyway, I think this is a good idea. -- Jon Biggar Floorboard Software vjon@floorboard.com jon@biggar.org Return-Path: Sender: jon@floorboard.com Date: Wed, 15 Jul 1998 22:52:04 -0700 From: Jonathan Biggar To: "George M. Scott" CC: Paul H Kyzivat , port-rtf@omg.org Subject: Re: urgent POA issues References: <35AC23DC.7B2F1ECF@inprise.com> <35AD332B.F8D71F57@noblenet.com> <35AD8ABE.D8982CAB@inprise.com> George M. Scott wrote: > I'm confused by your statement. Are you saying that a reference to a > POA that is destroyed and then is later recreated is still a valid > reference? I don't think this is the case. And if it is the case, then > this should be very explicitly stated in the spec. For example: > > Imagine the rootPOA is available and has an adapter activator installed > which can create a POA named "foo". Executing the following code will > activate the POA, then destroy it. > > // Java > POA a = rootPOA.find_POA("foo", true); > a.destroy(false, false); > > Now are you stating that if I now invoke an operation on "a", that it > will recreate the POA? For example: > > a.the_name(); > > will actually reactivate the POA? Or will it throw OBJECT_NOT_EXIST. > It was my understanding that a destroyed POA will always throw > OBJECT_NOT_EXIST when invoked, and recreating the same POA will not > result in the old reference suddenly becoming valid again. I agree with George. A reference to a POA that is destroyed should never work again, even when a POA with the same name is recreated. Practically, if the ORB implements POAs as objects mediated by the root POA, which has the TRANSIENT and SYSTEM_ID policies, this is the case anyway. -- Jon Biggar Floorboard Software jon@floorboard.com jon@biggar.org Return-Path: Date: Mon, 20 Jul 1998 11:57:48 -0400 From: Paul H Kyzivat Organization: NobleNet To: "George M. Scott" CC: port-rtf@omg.org Subject: Re: Proposal for issues 1408,1409 References: <35AC2413.B8E8BB4A@inprise.com> <35AD3940.6642DAF5@noblenet.com> <35AD8481.2B30E4F8@inprise.com> George M. Scott wrote: > > Paul H Kyzivat wrote: > > > 3. The apparent destruction of the POA will occur only after all > > > currently > > > executing requests in the POA and all of its children have > > > completed. > > > After destruction has become apparent the POA may be > recreated > via > > > either an AdapterActivator or a call to create_POA. > > > > Also can't become apparent until etherealization of children is > > complete. > > This would contradict what the spec currently states. Perhaps, but if this is necessary to make things work then we must consider making the change anyway. Which is worse - making this change or having etherealization always fail at shutdown??? Return-Path: Date: Mon, 20 Jul 1998 14:40:52 -0400 From: Paul H Kyzivat Organization: NobleNet To: Jonathan Biggar CC: "George M. Scott" , port-rtf@omg.org Subject: Re: Proposal for issues 1408,1409 References: <35AC2413.B8E8BB4A@inprise.com> <35AD3940.6642DAF5@noblenet.com> <35AD8481.2B30E4F8@inprise.com> <35AD9352.148601EE@floorboard.com> Jonathan Biggar wrote: > > George M. Scott wrote: > > > (Personally I think ServantManagers and AdapterActivators should > have > > > been servants rather than objects. That would have eliminated > this > > > problem. But too late for that.) > > > > Yes, making the ServantManager and AdapterActivators "special" > would > > have helped things out tremendously here. But not much we can do > > about this one. > > Well, we could still do this. Just add text that says that the POA > extracts the servant of the ServantManager and AdapterActivator > references it receives and interacts directly with the servant, > rather > than being mediated by the POA associated with those references. > This > would require of course that things like POA::Current would not work > in > incarnate() (and other) calls, but that might not be too much of a > price > to pay. > > Hand in hand with solving the servant memory management issue, this > would reduce the number of deadlock and premature death scenarios > considerably. > > Given the other number of deadlocks involved with having these > references mediated by a POA whose POAManager is currently in the > holding or discarding state anyway, I think this is a good idea. While I think this should have been done in the first place, I would not like to see a change like this rushed. The right way to do this would be to change the argument/return type for set_servant_manager, get_servant_manager and attribute the_activator to Servant. Just leaving the interface the way it is and grabbing the servant will have lots of complicated implications, such as: - What if the reference is to an object in a POA with a servant locator? Do you grab the servant and then call postinvoke? v - What if the reference is to an object in a POA with a default servant? - For the user, this is the worst of both worlds. It is still necessary to find some place to activate the object in order to pass a reference, but then a bunch of new rules come into play. Probably the normal recommended behavior would be to pass in serv->_this() and then immediately deactivate the object. This would take several obscure lines of code to achieve. Lets not take a bad situation and make it worse by doing half of a fix. Return-Path: Sender: "George Scott" Date: Thu, 23 Jul 1998 20:19:16 -0700 From: "George M. Scott" Organization: Inprise Corporation To: port-rtf@omg.org CC: orb_revision@omg.org, goldberg@inprise.com Subject: POA issues 1408,1409,1428, and 1627 I am sending in the current proposals for issues 1408, 1409, 1428 and 1627. It is critical that we reach consensus on these issues by the end of the day Friday (7/24) in order for these resolutions to be included in CORBA 2.3. Voting members of the previous Portability RTF please indicate if you are in favor or against the proposals by the 5pm PDT Friday. Jishnu is willing to move these issues to the Core RTF if we have consensus. The proposals for issues 1408/1409 and 1627 contain two sub-proposals. Please indicate which sub-proposal you favor in your response. Inprise will vote as follows: 1408/1409 - Yes for b. 1428 - Yes. 1627 - Yes for b. However, if there is consensus for the "a" variants of our proposal we are willing to change our vote to get resolution to these issues. Thanks. George P.S. I will be away from the office all day Friday, so comments on the proposal should be sent to Jon Goldberg (goldberg@inprise.com). Return-Path: Date: Thu, 23 Jul 1998 22:22:10 -0700 From: George Scott To: port-rtf@omg.org CC: orb_revision@omg.org Subject: Issue 1408/1409 - POA::destroy Here is our current proposal for issue 1408/1409 it incorporates feedback from our previous proposal. In particular all child POAs are completely destroyed before parent destruction begins to work around any issues of deactivating servant managers that may be required for etherealization of servants in child POAs. There are two sub-proposals (a and b). The (a) proposal is similiar to what we have proposed before in that the POA behaves as if it were in the holding state during destruction. This can potentially deadlock as we have previously discussed, so we have an alternative proposal (b) which does not deadlock. In proposal (b) the POA will not be destroyed until all requests have completed execution, it does not prevent new requests from executing, so there is no chance of deadlock. The POA will only be destroyed when there are no active requests. If a user wants different behavior, similar to (a) they may use the POAManager to change the state to holding or some other state before invoking destroy. Either proposal is straight forward to implement. We prefer (b) over (a) but will go with the consensus on this issue to reach resolution. We believe (b) provides the most flexibility to the user and is deadlock free. If a user wants behavior similar to (a) they may use the POAManager as we have discussed. George ---------------------------------------------------------------- Issue 1408: POA destroy() is ill-defined Issue 1409: Multiple threads calling destroy() once destroy() has begun Nature: Revision Summary: POA destroy is not defined sufficiently enough to prevent multiple activations of the same POA name in the same process. POA::destroy semantics are not defined for multiple threads calling destroy. Resolution: Accepted for Corba RTF 2.3 Revision: The following text will replace the current describing the destroy operation in Section 9.3.8, Page 9-31. This operation destroys the POA and all descendant POAs. All descendant POAs are destroyed (recursively) before the destruction of the containing POA. The POA so destroyed (that is, the POA with its name) may be re-created later in the same process. (This differs from the POAManager::deactivate operation that does not allow a re-creation of its associated POA in the same process. After a deactivate, re-creation is allowed only if the POA is later destroyed.) When destroy is called the POA will behave as follows: 1. The POA will call destroy on all of its immediate descendants. (Editorial note: Choose one behavior of 2a or 2b below): 2a.After all descendant POAs have been destroyed and their servants etherealized, the POA will behave as if its POAManager were in the holding state. The apparent destruction of the POA will occur only after all currently executing requests in the POA have completed. After destruction has become apparent, the POA may be recreated via either an AdapterActivator or a call to create_POA. 2b.After all descendant POAs have been destroyed and their servants etherealized, the POA will continue to process requests until there are no requests executing in the POA. The apparent destruction of the POA will occur only after all executing requests in the POA have completed. After destruction has become apparent, the POA may be recreated via either an AdapterActivator or a call to create_POA. 3. If the etherealize_objects parameter is TRUE, the POA has the RETAIN policy, and a servant manager is registered with the POA, the etherealize operation on the servant manager will be called for each active object in the Active Object Map. The apparent destruction of the POA occurs before any calls to etherealize are made. Thus, for example, an etherealize method that attempts to invoke operations on the POA will receive the OBJECT_NOT_EXIST exception. {editorial note - below I have merged text from these two issues and the relevant text from issue 1428 because they change the same text. If one of these resolutions fails, we will need to edit the following text) The wait_for_completion parameter is handled as follows: - If wait_for_completion is TRUE and the current thread is not in an invocation context dispatched from any POA, the destroy operation will return only after all active requests have completed and all invocations of etherealize have completed. - If wait_for_completion is TRUE and the current thread is in an invocation context dispatched from any POA, then the BAD_INV_ORDER exception is thrown and POA destruction does not occur. - If wait_for_completion is FALSE, the destroy operation destroys the POA and its children but does not wait for active requests to complete nor for etherealization to occur. If destroy is called multiple times before destruction is complete (because there are active requests), the etherealize_objects parameter will only apply to the first call of destroy. Subsequent calls with conflicting etherealize_objects settings will use the value of the etherealize_objects from the first call. The wait_for_completion parameter will be handled as defined above for each individual call (some callers may choose to block, while others may not). Return-Path: Sender: jon@floorboard.com Date: Thu, 23 Jul 1998 23:22:28 -0700 From: Jonathan Biggar To: "George M. Scott" CC: port-rtf@omg.org, orb_revision@omg.org, goldberg@inprise.com Subject: Re: POA issues 1408,1409,1428, and 1627 References: <35B7FD34.FCF9AC38@inprise.com> George M. Scott wrote: > > I am sending in the current proposals for issues 1408, 1409, 1428 > and > 1627. It is critical that we reach consensus on these issues by the > end of the day Friday (7/24) in order for these resolutions to be > included > in CORBA 2.3. Voting members of the previous Portability RTF please > indicate if you are in favor or against the proposals by the 5pm PDT > Friday. Jishnu is willing to move these issues to the Core RTF if > we have consensus. > > The proposals for issues 1408/1409 and 1627 contain two > sub-proposals. > Please indicate which sub-proposal you favor in your response. > > Inprise will vote as follows: > > 1408/1409 - Yes for b. > 1428 - Yes. > 1627 - Yes for b. > > However, if there is consensus for the "a" variants of our proposal > we > are willing to change our vote to get resolution to these issues. I also vote the same way: 1408/1409 - yes for b 1428 yes 1627 yes for b I like the new proposals. The only possible trouble left would be a case of livelock, where requests keep coming in for the object (or POA) continuously which prevent the deactivation or destruction from occurring. However, I expect this to be much less problematical than the previous deadlocking cases, and not too likely to occur anyway. -- Jon Biggar Floorboard Software jon@floorboard.com jon@biggar.org Return-Path: Date: Fri, 24 Jul 1998 17:37:28 +0100 From: jhierro@jam.tid.es (Juan Jose Hierro Sureda) To: port-rtf@omg.org, gscott@inprise.com Subject: Re: POA issues 1408,1409,1428, and 1627 Cc: orb_revision@omg.org, goldberg@inprise.com X-Sun-Charset: US-ASCII > > The proposals for issues 1408/1409 and 1627 contain two > sub-proposals. > Please indicate which sub-proposal you favor in your response. > > Inprise will vote as follows: > > 1408/1409 - Yes for b. > 1428 - Yes. > 1627 - Yes for b. > In respect to POA issues 1408/1409, Telefonica I+D votes yes for a In respect to POA issue 1627, Telefonica I+D votes yes for a I agree with Jon Biggar in that a possible trouble left for options b in proposals for 1408/1409 and 1627 would be a case of livelock, where requests keep coming in for the object (or POA) continuously which prevent the deactivation or destruction from occurring. However, I disagree with him and believe it would represent a serious problem, more problematic than the previous deadlocking cases which, I believe, can be avoided by the programmer if proposal a. is adopted on these issues. Regards, -- Juanjo Return-Path: Date: Fri, 24 Jul 1998 12:31:22 -0400 From: Bob Kukura Organization: IONA Technologies To: George Scott CC: port-rtf@omg.org, orb_revision@omg.org Subject: Re: Issue 1408/1409 - POA::destroy References: <35B81A02.138F6FC8@inprise.com> We prefer sub-proposal (b). But we can't vote YES until a couple of issues are resolved. See below. George Scott wrote: > > Here is our current proposal for issue 1408/1409 it incorporates > feedback from our previous proposal. In particular all child > POAs are completely destroyed before parent destruction begins > to work around any issues of deactivating servant managers that > may be required for etherealization of servants in child POAs. > > There are two sub-proposals (a and b). The (a) proposal is similiar > to what we have proposed before in that the POA behaves as if it > were in the holding state during destruction. This can potentially > deadlock as we have previously discussed, so we have an alternative > proposal (b) which does not deadlock. > > In proposal (b) the POA will not be destroyed until all requests > have completed execution, it does not prevent new requests from > executing, so there is no chance of deadlock. The POA will only > be destroyed when there are no active requests. If a user wants > different behavior, similar to (a) they may use the POAManager to > change the state to holding or some other state before invoking > destroy. > > Either proposal is straight forward to implement. We prefer (b) > over (a) but will go with the consensus on this issue to reach > resolution. We believe (b) provides the most flexibility to the > user and is deadlock free. If a user wants behavior similar to > (a) they may use the POAManager as we have discussed. > > George > > ---------------------------------------------------------------- > Issue 1408: POA destroy() is ill-defined > Issue 1409: Multiple threads calling destroy() once destroy() > has begun > Nature: Revision > > Summary: POA destroy is not defined sufficiently enough to > prevent multiple activations of the same POA name in > the same process. POA::destroy semantics are not defined > for multiple threads calling destroy. > > Resolution: Accepted for Corba RTF 2.3 > > Revision: The following text will replace the current describing > the destroy operation in Section 9.3.8, Page 9-31. > > This operation destroys the POA and all descendant POAs. All > descendant POAs are destroyed (recursively) before the destruction > of > the containing POA. The POA so destroyed (that is, the POA with its > name) may be re-created later in the same process. (This differs > from > the POAManager::deactivate operation that does not allow a > re-creation > of its associated POA in the same process. After a deactivate, > re-creation is allowed only if the POA is later destroyed.) > > When destroy is called the POA will behave as follows: > > 1. The POA will call destroy on all of its immediate descendants. > > (Editorial note: Choose one behavior of 2a or 2b below): > > 2a.After all descendant POAs have been destroyed and their servants > etherealized, the POA will behave as if its POAManager were in > the > holding state. The apparent destruction of the POA will occur > only > after all currently executing requests in the POA have completed. > After destruction has become apparent, the POA may be recreated > via > either an AdapterActivator or a call to create_POA. > > 2b.After all descendant POAs have been destroyed and their servants > etherealized, the POA will continue to process requests until > there > are no requests executing in the POA. The apparent destruction > of > the POA will occur only after all executing requests in the > POA have completed. After destruction has become apparent, the > POA > may be recreated via either an AdapterActivator or a call to > create_POA. > > 3. If the etherealize_objects parameter is TRUE, the POA has the > RETAIN > policy, and a servant manager is registered with the POA, the > etherealize operation on the servant manager will be called for > each > active object in the Active Object Map. The apparent destruction > of > the > POA occurs before any calls to etherealize are made. Thus, for > example, > an etherealize method that attempts to invoke operations on the > POA > will > receive the OBJECT_NOT_EXIST exception. Does this mean that a create_POA() of the same name can succeed after "apparent destruction", but during etherealization? I don't think that should be allowed. This may have been discussed before, but I don't remember if there was a conclusion. If state is saved by an application during etherealization, it would be disasterous to recreate a new POA instance for the same set of objects before the old state was saved. We need to decide whether attempting to re-create the same POA name during etherealization will block or raise an exception. > > {editorial note - below I have merged text from these two issues and > the relevant text from issue 1428 because they change the same text. > If one of these resolutions fails, we will need to edit the > following > text) > > The wait_for_completion parameter is handled as follows: The cross ORB instance dependency issue needs to be addressed as I described in my response to the issue 1428 email. > > - If wait_for_completion is TRUE and the current > thread is not in an invocation context dispatched from any POA, > the destroy operation will return only after all active requests > have completed and all invocations of etherealize have > completed. > > - If wait_for_completion is TRUE and the current thread is in an > invocation context dispatched from any POA, then the > BAD_INV_ORDER > exception is thrown and POA destruction does not occur. > > - If wait_for_completion is FALSE, the destroy operation > destroys the POA and its children but does not wait for active > requests to complete nor for etherealization to occur. > > If destroy is called multiple times before destruction is > complete (because there are active requests), the > etherealize_objects parameter will only apply to the first > call of destroy. Subsequent calls with conflicting > etherealize_objects settings will use the value of the > etherealize_objects from the first call. The wait_for_completion > parameter will be handled as defined above for each individual > call (some callers may choose to block, while others may not). -Bob Return-Path: Sender: jon@floorboard.com Date: Fri, 24 Jul 1998 10:14:53 -0700 From: Jonathan Biggar To: Bob Kukura CC: George Scott , port-rtf@omg.org, orb_revision@omg.org Subject: Re: Issue 1408/1409 - POA::destroy References: <35B81A02.138F6FC8@inprise.com> <35B8B6DA.254383A0@iona.com> Bob Kukura wrote: > > We prefer sub-proposal (b). > > But we can't vote YES until a couple of issues are resolved. See > below. > > George Scott wrote: > > 3. If the etherealize_objects parameter is TRUE, the POA has the > RETAIN > > policy, and a servant manager is registered with the POA, the > > etherealize operation on the servant manager will be called for > each > > active object in the Active Object Map. The apparent > destruction of > > the > > POA occurs before any calls to etherealize are made. Thus, for > > example, > > an etherealize method that attempts to invoke operations on the > POA > > will > > receive the OBJECT_NOT_EXIST exception. > > Does this mean that a create_POA() of the same name can succeed > after > "apparent destruction", but during etherealization? I don't think > that > should be allowed. This may have been discussed before, but I don't > remember if there was a conclusion. If state is saved by an > application > during etherealization, it would be disasterous to recreate a new > POA > instance for the same set of objects before the old state was > saved. We > need to decide whether attempting to re-create the same POA name > during > etherealization will block or raise an exception. I would favor blocking, since the attempt to recreate the POA may be from an incoming request calling to a POA activator object. If an exception was raised, this would break client transparency. -- Jon Biggar Floorboard Software jon@floorboard.com jon@biggar.org Return-Path: Sender: jon@floorboard.com Date: Fri, 24 Jul 1998 10:54:07 -0700 From: Jonathan Biggar To: Juan Jose Hierro Sureda CC: port-rtf@omg.org, gscott@inprise.com, orb_revision@omg.org, goldberg@inprise.com Subject: Re: POA issues 1408,1409,1428, and 1627 References: <9807241831.AA07082@jam.tid.es> Juan Jose Hierro Sureda wrote: > The POAManagers associated to the ORB would be deactivated as a result > of invoking shutdown ... That means that no other request may invoke > the operation where shutdown was invoked ... Of course, the thread > where shutdown was invoked would not be able to invoke CORBA operations > after shutdown has returned (an exception would be returned) but would > be able to perform some work. > > With this behaviour, I don't see the deadlock ... Here is the scenario: request A invokes request B which calls shutdown in blocking mode. If request A is invoked in the context of an object local to the same orb as B, then you will get deadlock. Shutdown will block waiting for request A to complete, which never will, since it is waiting for request B to complete, which is blocked on shutdown. -- Jon Biggar Floorboard Software jon@floorboard.com jon@biggar.org Return-Path: Date: Fri, 24 Jul 1998 18:57:46 +0100 From: jhierro@jam.tid.es (Juan Jose Hierro Sureda) To: jhierro@jam.tid.es, jon@floorboard.com Subject: Re: POA issues 1408,1409,1428, and 1627 Cc: port-rtf@omg.org, gscott@inprise.com, orb_revision@omg.org, goldberg@inprise.com X-Sun-Charset: US-ASCII > > Did you forget your rationale? > > > > 1428: Blocking POA operations > > > Proposal: Clarification 7/8 > > > Yes: Mischkinsky, Mukerji, Giddings > > > No: Kukura > > > > > > > I vote NO on this proposal. My rationale is that it would > prevent > > usage of CORBA objects to admin servers (e.g., usage of an Admin > CORBA object > > at each server that exports a shutdown operation which effectively > > shuts down the server where it is executing > > > > module Server { > > interface Admin { > > ... > > void shutdown (); > > }; > > > > ). > > > > The proposal currently being discussed doesn't allow a simple > implementation > > of the Admin::shutdown operation based on invoking ORB::shutdown: > the > > invocation of ORB::shutdown inside the method that implements > operation > > Admin::shutdown would get a BAD_INV_ORDER exception. > > > > This short of administrative service is used in some products > that HP > > and Telefonica I+D have developed together and may have already > been > > shipped. > > This proposal does not prevent your Admin object from working right! > It > still allows you to call shutdown, but only the non-blocking > version. > Using the blocking version in your implementation of > Server::Admin::shutdown() is problematical anyway, since there is no > guarantee that a response can be sent once the ORB is shutdown. > I may use the non-blocking version of ORB::shutdown to implement Server::Admin::shutdown but that wouldn't support the kind of semantics I would like to see in all cases. I would be more in favour of a proposal that states that: - shutdown would block until all request being processed by POAManagers associated to that ORB have completed except for processing of the request where shutdown was called (if called from a request dispatch thread) - the ORB effectively shutdowns (i.e., ORB::run returns) when all ORB processing is completed (this means that the ORB would effectively shut down after the thread that invoked ORB::shutdown actually finishes, if that thread was trigered to dispatch a request) Anyway, i still don't like the proposed solution so I maintain the NO vote ... I prefer to resolve this issue in the next RTF since it's important and I believe that, if we change it, we should change it forever ... Of course, i may be the only one who votes no :-) -- Juanjo Return-Path: Date: Fri, 24 Jul 1998 14:00:05 -0400 From: Bob Kukura Organization: IONA Technologies To: Jonathan Biggar CC: George Scott , port-rtf@omg.org, orb_revision@omg.org Subject: Re: Issue 1408/1409 - POA::destroy References: <35B81A02.138F6FC8@inprise.com> <35B8B6DA.254383A0@iona.com> <35B8C10D.A3673341@floorboard.com> Jonathan Biggar wrote: > > Bob Kukura wrote: > > > > We prefer sub-proposal (b). > > > > But we can't vote YES until a couple of issues are resolved. See > below. > > > > George Scott wrote: > > > 3. If the etherealize_objects parameter is TRUE, the POA has the > RETAIN > > > policy, and a servant manager is registered with the POA, the > > > etherealize operation on the servant manager will be called > for each > > > active object in the Active Object Map. The apparent > destruction of > > > the > > > POA occurs before any calls to etherealize are made. Thus, > for > > > example, > > > an etherealize method that attempts to invoke operations on > the POA > > > will > > > receive the OBJECT_NOT_EXIST exception. > > > > Does this mean that a create_POA() of the same name can succeed > after > > "apparent destruction", but during etherealization? I don't think > that > > should be allowed. This may have been discussed before, but I > don't > > remember if there was a conclusion. If state is saved by an > application > > during etherealization, it would be disasterous to recreate a new > POA > > instance for the same set of objects before the old state was > saved. We > > need to decide whether attempting to re-create the same POA name > during > > etherealization will block or raise an exception. > > I would favor blocking, since the attempt to recreate the POA may be > from an incoming request calling to a POA activator object. If an > exception was raised, this would break client transparency. We agree. -Bob > > -- > Jon Biggar > Floorboard Software > jon@floorboard.com > jon@biggar.org Return-Path: Date: Fri, 24 Jul 1998 19:31:18 +0100 From: jhierro@jam.tid.es (Juan Jose Hierro Sureda) To: jhierro@jam.tid.es, jon@floorboard.com Subject: Re: POA issues 1408,1409,1428, and 1627 Cc: port-rtf@omg.org, gscott@inprise.com, orb_revision@omg.org, goldberg@inprise.com X-Sun-Charset: US-ASCII > > Juan Jose Hierro Sureda wrote: > > I would be more in favour of a proposal that states that: > > > > - shutdown would block until all request being processed by > POAManagers > > associated to that ORB have completed except for processing of > the > > request where shutdown was called (if called from a request > dispatch > > thread) > > > > - the ORB effectively shutdowns (i.e., ORB::run returns) when > > all ORB processing is completed (this means that the ORB would > > effectively shut down after the thread that invoked > ORB::shutdown > > actually finishes, if that thread was trigered to dispatch a > > request) > > This can still deadlock if the request that calls shutdown is in > turn > invoked by the implementation of another request in the same ORB. > The POAManagers associated to the ORB would be deactivated as a result of invoking shutdown ... That means that no other request may invoke the operation where shutdown was invoked ... Of course, the thread where shutdown was invoked would not be able to invoke CORBA operations after shutdown has returned (an exception would be returned) but would be able to perform some work. With this behaviour, I don't see the deadlock ... > > Anyway, i still don't like the proposed solution so I maintain > > the NO vote ... I prefer to resolve this issue in the next RTF > > since it's important and I believe that, if we change it, we > > should change it forever ... > > Nothing is forever... :-) Of course I don't want to change > something > and discover a problem in the new version that is as bad or worse > than > what we had before. Ultimately, however, it just has to come down > to > our best judgement at the time. > That's the point: I don't want to change something until I'm sure (or believe I'm sure :-) that I will not discover a problem in the new version that is as bad or worse than what we had before. -- Juanjo Return-Path: Date: Fri, 24 Jul 1998 12:00:15 -0700 From: "Jon Goldberg" Reply-To: goldberg@inprise.com To: Juan Jose Hierro Sureda CC: jon@floorboard.com, port-rtf@omg.org, gscott@inprise.com, orb_revision@omg.org Subject: Re: POA issues 1408,1409,1428, and 1627 References: <9807241757.AA07022@jam.tid.es> Juan Jose Hierro Sureda wrote: > > > > > Did you forget your rationale? > > > > > > 1428: Blocking POA operations > > > > Proposal: Clarification 7/8 > > > > Yes: Mischkinsky, Mukerji, Giddings > > > > No: Kukura > > > > > > > > > > I vote NO on this proposal. My rationale is that it would > prevent > > > usage of CORBA objects to admin servers (e.g., usage of an Admin > CORBA object > > > at each server that exports a shutdown operation which > effectively > > > shuts down the server where it is executing > > > > > > module Server { > > > interface Admin { > > > ... > > > void shutdown (); > > > }; > > > > > > ). > > > > > > The proposal currently being discussed doesn't allow a simple > implementation > > > of the Admin::shutdown operation based on invoking > ORB::shutdown: the > > > invocation of ORB::shutdown inside the method that implements > operation > > > Admin::shutdown would get a BAD_INV_ORDER exception. > > > > > > This short of administrative service is used in some products > that HP > > > and Telefonica I+D have developed together and may have already > been > > > shipped. > > > > This proposal does not prevent your Admin object from working > right! It > > still allows you to call shutdown, but only the non-blocking > version. > > Using the blocking version in your implementation of > > Server::Admin::shutdown() is problematical anyway, since there is > no > > guarantee that a response can be sent once the ORB is shutdown. > > > > I may use the non-blocking version of ORB::shutdown to implement > Server::Admin::shutdown but that wouldn't support the kind of > semantics I would like to see in all cases. > > I would be more in favour of a proposal that states that: > > - shutdown would block until all request being processed by > POAManagers > associated to that ORB have completed except for processing of > the > request where shutdown was called (if called from a request > dispatch > thread) I just want to point out that you are always free to have your application explicitly put POAManagers in any state you'd like (holding/deactivated) if you want explicit behavior with respect to other requests. You can then invoke shutdown. This proposal only clears up rules for the implicit case and does a good job of eliminating deadlock for those cases. We cannot deal with every case implicitly, so I'm strongly in favor of making deadlocks as rare as possible when things are done implicitly, while making sure that applications that know a lot about what they are doing still have the necessary hooks to get any other reasonable behavior by explicitly using those hooks. George's proposal satisfies both of those goals. take care, Jon Return-Path: Date: Fri, 24 Jul 1998 12:08:48 -0700 From: "Jon Goldberg" Reply-To: goldberg@inprise.com To: Bob Kukura CC: gscott@inprise.com, orb_revision@omg.org, port-rtf@omg.org Subject: Re: Issue 1408/1409 - POA::destroy References: <35B81A02.138F6FC8@inprise.com> Bob Kukura wrote: > > We prefer sub-proposal (b). > > But we can't vote YES until a couple of issues are resolved. See > below. > > George Scott wrote: > > > > 2b.After all descendant POAs have been destroyed and their > servants > > etherealized, the POA will continue to process requests until > there > > are no requests executing in the POA. The apparent destruction > of > > the POA will occur only after all executing requests in the > > POA have completed. After destruction has become apparent, the > POA > > may be recreated via either an AdapterActivator or a call to > > create_POA. > > > > 3. If the etherealize_objects parameter is TRUE, the POA has the > RETAIN > > policy, and a servant manager is registered with the POA, the > > etherealize operation on the servant manager will be called for > each > > active object in the Active Object Map. The apparent > destruction of > > the > > POA occurs before any calls to etherealize are made. Thus, for > > example, > > an etherealize method that attempts to invoke operations on the > POA > > will > > receive the OBJECT_NOT_EXIST exception. > > Does this mean that a create_POA() of the same name can succeed > after > "apparent destruction", but during etherealization? I don't think > that > should be allowed. This may have been discussed before, but I don't > remember if there was a conclusion. If state is saved by an > application > during etherealization, it would be disasterous to recreate a new > POA > instance for the same set of objects before the old state was > saved. We > need to decide whether attempting to re-create the same POA name > during > etherealization will block or raise an exception. good catch. I agree that it should block. Basically, apparent destruction doesn't occur until all executing requests have completed. Once destruction is apparent, the POA behaves as if its POAManager is in the holding state. This still leaves deadlock as possible if etherealize explicitly creates the POA or makes a request on an object activated by that POA, but this is acceptable as a corner case to us. > > > > {editorial note - below I have merged text from these two issues > and > > the relevant text from issue 1428 because they change the same > text. > > If one of these resolutions fails, we will need to edit the > following > > text) > > > > The wait_for_completion parameter is handled as follows: > > The cross ORB instance dependency issue needs to be addressed as I > described in my response to the issue 1428 email. Please see my note in response to that message. I think the wording needs to stay the same to avoid a deadlock case, and I don't think there is actually any dependency issue between the ORBs (unless I've misunderstood your meaning of "dependency"). -Jon Return-Path: Date: Fri, 24 Jul 1998 23:11:47 -0400 From: Paul H Kyzivat Organization: NobleNet To: Juan Jose Hierro Sureda CC: jon@floorboard.com, port-rtf@omg.org, gscott@inprise.com, orb_revision@omg.org, goldberg@inprise.com Subject: Re: POA issues 1408,1409,1428, and 1627 References: <9807241831.AA07082@jam.tid.es> Juan Jose Hierro Sureda wrote: > The POAManagers associated to the ORB would be deactivated as a > result > of invoking shutdown ... That means that no other request may > invoke > the operation where shutdown was invoked ... Of course, the thread > where shutdown was invoked would not be able to invoke CORBA > operations > after shutdown has returned (an exception would be returned) but > would > be able to perform some work. The ORB shutdown cannot complete with an operation outstanding, for many reasons. Here is one: the POA containing the object-that-is-in-an-invocation must be destroyed before shutdown is complete. This may require etherealization of that very object. But it cannot be etherealized until the operation is complete. You seem to be asking someone to commit suicide and report back that he has successfully done so, buried himself, and planted grass and a proper headstone on the grave. This isn't really possible. Return-Path: Date: Sat, 25 Jul 1998 16:13:26 -0700 From: "Jon Goldberg" Reply-To: goldberg@inprise.com To: orb_revision@omg.org CC: port-rtf@omg.org Subject: Issue 1409/1409 - POA::destroy Hi Folks- Here is the revised text for resolving 1408-9. It is George's proposal with item 2b selected over 2a, and an additional clarifying statement suggested by Bob Kukura for item 3 under POA behavior. This represents the informal consensus of the voting members of the portability RTF. By informal, I mean all the vocal members agreed to this text. No formal vote was taken. Feel free to disagree with my assessment of the situation. Given the "gentlemen's agreement" between the core and portability RTFs, I think these can now be voted on by the core RTF. Jishnu's deadline is probably pretty close (sometime on Monday?) so please vote quickly. If Jishnu could send out the voting list one more time, I'd appreciate it. thanks, Jon ---------------------------------------------------------------- Issue 1408: POA destroy() is ill-defined Issue 1409: Multiple threads calling destroy() once destroy() has begun Nature: Revision Summary: POA destroy is not defined sufficiently enough to prevent multiple activations of the same POA name in the same process. POA::destroy semantics are not defined for multiple threads calling destroy. Resolution: Accepted for Corba RTF 2.3 Revision: The following text will replace the current describing the destroy operation in Section 9.3.8, Page 9-31. This operation destroys the POA and all descendant POAs. All descendant POAs are destroyed (recursively) before the destruction of the containing POA. The POA so destroyed (that is, the POA with its name) may be re-created later in the same process. (This differs from the POAManager::deactivate operation that does not allow a re-creation of its associated POA in the same process. After a deactivate, re-creation is allowed only if the POA is later destroyed.) When destroy is called the POA will behave as follows: 1. The POA will call destroy on all of its immediate descendants. 2. After all descendant POAs have been destroyed and their servants etherealized, the POA will continue to process requests until there are no requests executing in the POA. The apparent destruction of the POA will occur only after all executing requests in the POA have completed. After destruction has become apparent, the POA may be recreated via either an AdapterActivator or a call to create_POA. 3. If the etherealize_objects parameter is TRUE, the POA has the RETAIN policy, and a servant manager is registered with the POA, the etherealize operation on the servant manager will be called for each active object in the Active Object Map. The apparent destruction of the POA occurs before any calls to etherealize are made. Thus, for example, an etherealize method that attempts to invoke operations on the POA will receive the OBJECT_NOT_EXIST exception. Once apparent destruction has occurred, the POA behaves as if its POAManager is in the holding state until destruction is complete. Thus, for example, an invocation of create_POA() with the same name will block until POA destruction has finished. {editorial note - below I have merged text from these two issues and the relevant text from issue 1428 because they change the same text. If one of these resolutions fails, we will need to edit the following text) The wait_for_completion parameter is handled as follows: - If wait_for_completion is TRUE and the current thread is not in an invocation context dispatched from any POA, the destroy operation will return only after all active requests have completed and all invocations of etherealize have completed. - If wait_for_completion is TRUE and the current thread is in an invocation context dispatched from any POA, then the BAD_INV_ORDER exception is thrown and POA destruction does not occur. - If wait_for_completion is FALSE, the destroy operation destroys the POA and its children but does not wait for active requests to complete nor for etherealization to occur. If destroy is called multiple times before destruction is complete (because there are active requests), the etherealize_objects parameter will only apply to the first call of destroy. Subsequent calls with conflicting etherealize_objects settings will use the value of the etherealize_objects from the first call. The wait_for_completion parameter will be handled as defined above for each individual call (some callers may choose to block, while others may not). Return-Path: Date: Mon, 27 Jul 1998 10:40:52 -0400 From: Bob Kukura Organization: IONA Technologies To: goldberg@inprise.com CC: orb_revision@omg.org, port-rtf@omg.org Subject: Re: Issue 1409/1409 - POA::destroy References: <35BA6696.FBE5F36@inprise.com> Jon Goldberg wrote: > [...] > {editorial note - below I have merged text from these two issues and > the relevant text from issue 1428 because they change the same text. > If one of these resolutions fails, we will need to edit the > following > text) > > The wait_for_completion parameter is handled as follows: Hi Jon, Doesn't the phrase "from any POA" which appears twice below need to read "from any POA belonging to the same ORB as this POA" to be consistent with the revised resolution for 1428? -Bob > > - If wait_for_completion is TRUE and the current > thread is not in an invocation context dispatched from any POA, > the destroy operation will return only after all active requests > have completed and all invocations of etherealize have > completed. > > - If wait_for_completion is TRUE and the current thread is in an > invocation context dispatched from any POA, then the > BAD_INV_ORDER > exception is thrown and POA destruction does not occur. > > - If wait_for_completion is FALSE, the destroy operation > destroys the POA and its children but does not wait for active > requests to complete nor for etherealization to occur. > > If destroy is called multiple times before destruction is > complete (because there are active requests), the > etherealize_objects parameter will only apply to the first > call of destroy. Subsequent calls with conflicting > etherealize_objects settings will use the value of the > etherealize_objects from the first call. The wait_for_completion > parameter will be handled as defined above for each individual > call (some callers may choose to block, while others may not). Return-Path: Date: Mon, 27 Jul 1998 07:57:58 -0700 From: "Jon Goldberg" Reply-To: goldberg@inprise.com To: Bob Kukura CC: orb_revision@omg.org, port-rtf@omg.org Subject: Re: Issue 1409/1409 - POA::destroy References: <35BA6696.FBE5F36@inprise.com> <35BC9174.6B6FBD65@iona.com> oops. I forgot the overlap between the issues. Please consider those two phrases updated as Bob suggests. -Jon Bob Kukura wrote: > > Jon Goldberg wrote: > > > > [...] > > > {editorial note - below I have merged text from these two issues > and > > the relevant text from issue 1428 because they change the same > text. > > If one of these resolutions fails, we will need to edit the > following > > text) > > > > The wait_for_completion parameter is handled as follows: > > Hi Jon, > > Doesn't the phrase "from any POA" which appears twice below need to > read > "from any POA belonging to the same ORB as this POA" to be > consistent > with the revised resolution for 1428? > > -Bob > > > > > - If wait_for_completion is TRUE and the current > > thread is not in an invocation context dispatched from any POA, > > the destroy operation will return only after all active > requests > > have completed and all invocations of etherealize have > > completed. > > > > - If wait_for_completion is TRUE and the current thread is in an > > invocation context dispatched from any POA, then the > BAD_INV_ORDER > > exception is thrown and POA destruction does not occur. > > > > - If wait_for_completion is FALSE, the destroy operation > > destroys the POA and its children but does not wait for active > > requests to complete nor for etherealization to occur. > > > > If destroy is called multiple times before destruction is > > complete (because there are active requests), the > > etherealize_objects parameter will only apply to the first > > call of destroy. Subsequent calls with conflicting > > etherealize_objects settings will use the value of the > > etherealize_objects from the first call. The wait_for_completion > > parameter will be handled as defined above for each individual > > call (some callers may choose to block, while others may not).