Issues for UML testing Profile Revision Task Force

List of issues (green=resolved, yellow=pending Board vote, red=unresolved)

List options: All ; Open Issues only; or Closed Issues only

Jira Issues

Issue 7195: UML 2.0 Alignment Jira Issue UTP-27
Issue 9797: Section: 6.3.2 Test Behavior Jira Issue UTP11-33
Issue 9798: Section: 6.3.2 Test Behavior, FinishAction Jira Issue UTP11-34
Issue 9799: Section: 6.3.3 Test Data, CodingRule Jira Issue UTP11-35
Issue 15644: Scheduler seems to be a tool concept Jira Issue UTP11-41
Issue 15648: Scheduler reference should be made optional in utp::TestContext Jira Issue UTP11-42
Issue 15649: Clarification of Arbiter Description Jira Issue UTP11-43
Issue 15650: utp::InteractionOperator::determAlt not applicable in CombinedFragment Jira Issue UTP11-44
Issue 15651: Who is the receiver of a LogAction? Jira Issue UTP11-45
Issue 15652: �TestLog� should be renamed to avoid confusion Jira Issue UTP11-46
Issue 15653: UML::Action metaclass is too ambiguous for �FinishAction� Jira Issue UTP11-47
Issue 15654: utp::TimeOut should extend UML::TimeEvent Jira Issue UTP11-48
Issue 15655: TimeOutMessage, TimeOutAction and TimeOut are semantically equivalent Jira Issue UTP11-49
Issue 15656: UTP should have an official logo Jira Issue UTP11-50
Issue 15657: Missing numbering of chapters leads to imprecise references Jira Issue UTP11-51
Issue 15658: UML Testing Profile should be officiallly acronymed by UTP Jira Issue UTP11-52
Issue 15770: Restructuring of chapters Jira Issue UTP11-53
Issue 15771: Continuation information of DefaultApplication should be formalized as model concepts Jira Issue UTP11-54
Issue 15772: Merging distributed semantical descriptions of elements Jira Issue UTP11-55
Issue 15773: Contents and location of chapter 4 must be clarified Jira Issue UTP11-56
Issue 15813: Figure 6.19 need clarification Jira Issue UTP11-57
Issue 15907: Errornous constraint description of TestLogApplication Jira Issue UTP11-58
Issue 15908: "Testing Profile" word phrases should be changed Jira Issue UTP11-59
Issue 15909: Stereotypes should be decorated with angle brackets �� Jira Issue UTP11-60
Issue 15910: Test Context must be a UML::BehavioredClassifier, too Jira Issue UTP11-61
Issue 15913: Test management should be supported Jira Issue UTP11-62
Issue 15915: Test log should contain additional information Jira Issue UTP11-63
Issue 15916: Reviews and audits should be supported by UTP Jira Issue UTP11-64
Issue 15921: Spell out all acronyms at first use with the document, e.g. see first page, UML, MOF, and XMI Jira Issue UTP11-36
Issue 15926: input - SUT issue Jira Issue UTP11-37
Issue 15928: Verify Relationship not in UTP Jira Issue UTP11-25
Issue 15929: testcase or TestCase Jira Issue UTP11-38
Issue 15931: Test Objective issue Jira Issue UTP11-26
Issue 15932: General definitions question Jira Issue UTP11-39
Issue 15939: UTP & SysML Jira Issue UTP11-27
Issue 15941: Complex Data Jira Issue UTP11-28
Issue 15942: Who is "self"? Jira Issue UTP11-40
Issue 16000: TestRequirements shall be defined Jira Issue UTP11-29
Issue 16105: Missing semantics section Jira Issue UTP11-65
Issue 16163: Rework of TestArchitecture section Jira Issue UTP11-66
Issue 16346: UTP profile Jira Issue UTP11-30
Issue 16878: Integration of TTCN-3 template modification Jira Issue UTP12-25
Issue 16898: Tag definitions of test management concepts should be of type ValueSpecification Jira Issue UTP12-26
Issue 16900: Correction for issue #15652 Jira Issue UTP12-27
Issue 16902: TestLog should have detailed information as TestLogEntry Jira Issue UTP12-28
Issue 16906: Base class of TestComponents must be instances of BehavioredClassifier, too Jira Issue UTP12-29
Issue 17222: LogAction should be FinishAction Jira Issue UTP12-30
Issue 17223: Modernization of introductory chapters Jira Issue UTP12-31
Issue 17224: UTP should constitute a new conceptual package structure Jira Issue UTP12-32
Issue 17229: Typos, style glitches and figures Jira Issue UTP12-33
Issue 17231: ValidationAction should extends CallOperationAction Jira Issue UTP12-34
Issue 17292: Rename predefined type Time to Timepoint Jira Issue UTP12-35
Issue 17462: ValidatonAction should provide a reason why the verdict has been assigned Jira Issue UTP11-31
Issue 17539: Semantics of DefaultApplication::repetition not clear Jira Issue UTP11-32

Issue 7195: UML 2.0 Alignment (uml-testing-profile-rtf)

Summary: U2TP has to be aligned with the finalized UML 2.0

Default chapter, Notation section (p.28): �a default is behavior and has no special notation�. DefaultApplication chapter, Description section (p.28): �a default application is a dependency used to apply a default behavior to a unit of test component�. However, in Notation section of DefaultApplication chapter there is written that default should be represented as comment. The same notation is used in examples on Figures 29 and 31. Since information in Default and DefaultApplication chapters contradicts each other we need clarification what notation should be used for Default and DefaultApplication. Also it is unclear how repeat and continue parameters should be represented. According to UML specification the detailed semantics of behavior is determined by it subtypes. Each subtype (Interaction, Activity, StateMachine) has it own notation. Since Default can also be described on Interaction, Activity or StateMachine the most convenient notation for Default would be behavior subtype notation with <<Default>> stereotype and for DefaultApplication � dependency with stereotype <<DefaultApplication>>.   

This issue is a subset of the more comprehensive issues 15771.  Disposition:	See issue 15771 for disposition  

<<FinishAction>> stereotype extends Action metaclass. However FinishAction chapter, Notation section specifies how FinishAction should be represented on Activity, Interaction and StateMachine diagrams. In that case we have conflict with UML 2.0 specification: according to UML specification actions cannot be used on Interaction and StateMachine diagrams. Actually for interactions UML 2.0 defines ActionExecutionSpecification that specifies execution of action within the Lifeline. However, then <<FinishAction>> stereotype should extend ActionExecutionSpecification metaclass.   

This issue is a subset of the more comprehensive issues 15653.  Disposition:	See issue 15653 for disposition  

UML Testing Profile defines that �the notation for coding rule is identical to a comment in UML�. Can coding rule be displayed as stereotype attribute (tagged value) using notation given by UML 2.0 specification?  

Rational: Scheduler is a predefined interface defining operations used for controlling the tests and the test components. None of the operations of the Scheduler are available to the UML specifier.      Issue: The usage of the scheduler is not clear. Scheduler seems to be a pure tool concept. Section 6.3.1 Test Architecture, Subsection Scheduler says the following:       Scheduler is a predefined interface defining operations used for controlling the tests and the test components. None of the operations of the Scheduler are available to the UML specifier. The implementation of the predefined interface will determine the detailed semantics. The implementation must make sure that the scheduler has enough information to keep track of the existence and participation of the test components in every test case. The test context itself will ensure the creation of a scheduler.    Hence, the scheduler cannot be used within the test model as it is not visible to the UML elements. The specification should separate between model elements and interfaces to the test execution system, which can be seen as technical issues.

Scheduler is not part of the profile (of any test specification model) and confuses reader. It is a dedicated tool concept and will be moved into the Annex.  Disposition:	See issue 16163 for disposition  

Rational: Scheduler is a predefined interface defining operations used for controlling the tests and the test components. None of the operations of the Scheduler are available to the UML specifier.      Issue: Section 6.3.1 Test Architecture, Subsection TestContext, Subsubsection Constraints states the following:   ---  A test context must contain exactly one property realizing the Scheduler interface  ---  A scheduler represents a pure tool concept and is not visible to any UML element in the test model. Implementing this interface in context of test model is not possible.

Scheduler is not part of the profile (of any test specification model) and confuses reader. It is a dedicated tool concept and will be moved into the Annex. As a consequence the property of TestContext must be removed.  Disposition:	See issue 16163 for disposition  

Rational: Arbiter is a predefined interface defining operations used for arbitration of tests. Test cases, test contexts, and the runtime system can use realizations of this interface to assign verdicts of tests and to retrieve the current verdict of a test (verdicts are discussed in Section 6.3.2, �Test Behavior,� on page 14)       Issue: Section 6.3.2 Test Behavior, Subsection Verdict mentions the following:  ---  The final verdict of a test case is determined by an arbiter. Every test context has an arbiter and the tool vendor will provide a default arbiter if the test context does not explicitly specify one [� ]  ---  Section 6.3.1 "TestArchitecture" (page 10) says:  ---  Every test context must have an implementation of the arbiter interface, and the tool vendor constructing tools based on  the Testing Profile will provide a default arbiter to be used if one is not explicitly defined in the test context.  ---      It seems to be a logical contradiction. Since the multiplictity of a test context's arbiter reference (see Fig. 6.1 TestArchitecture) is exactly one (so it has to be set in any case), why is it necessary to have a default arbiter in addition? It can never be used by a test context. In the case, a test context wants to use a kind of default arbiter, it has to define it anyway.  In order to avoid confusion on the readers, this must be clarified and probably simplified.

This issue will be clarified by the merge of scattered information, addressed by resolution to issue 15772. The solution itself is easy. The section to be clarified will be removed by the referred resolution.  Disposition:	See issue 15772 for disposition  

Rational: Section 6.3.2 Test Behavior, Subsection InteractionOperator says the following:   ---  The deterministic alternative is a shorthand for an Alternative where the operands are evaluated in sequence such that it is deterministic which operand is chosen given the value of the guards, regardless of the fact that the guard for more than one operand may be true  ---      Issue: utp::InteractionOperator::determAlt is embedded in an UTP enumeration and therefore not applicable to UML models. UML::CombinedFragment refers to UML::InteractionOperator instead, so that it seems as utp::InteractionOperator is an extension to UM::InteractionOperator. Extending metaclasses is restricted by the UML Profile design requirements (UML::Superstructure, Section 18.1.2, Item 1):  ---  8. [�] UML Profiles should form a metamodel extension mechanism that imposes certain restrictions on how the UML metamodel can be modified. The reference metamodel is considered as a �read only� model, that is extended without changes by profiles. It is therefore forbidden to insert new metaclasses in the UML metaclass hierarchy (i.e., new super-classes for standard UML metaclasses) or to modify the standard UML metaclass definitions (e.g., by adding meta-associations). Such restrictions do not apply in a MOF context where in principle any metamodel can be reworked in any direction.  ---            UTP::InteractionOperator::determAlt must be completely redesigned in order to be applicable to UML models.

see pge 45 for figure

Rational: Section 6.3.2 Test Behavior, Subsection LogAction says the following:  ---   The LogAction defines the tester�s desire to log certain aspects during the execution of a test case.  ---      Issue: Since �LogAction� stereotype enhances UML::SendObjectAction, the receiver of the object to be logged must be explicitly mentioned. It is not clear who receives the sent object.     Commonly, a log is sent to the execution environment which accepts and persists the entry. This seems to be a tool concepts, too, comparable to Scheduler semantics. It might make sense to introduce another additional technical interface for logging at test execution system level. �LogAction� might extend uml::OpaqueAction, so that there is no explicit receiver to be specified on model level. Vendors must ensure that logs are transported properly to the log interface.

Rational: A �TestLog � represents the actual trace of an executed test case. It contains the messages and stimuli sent to the SUT as well as the result of the SUT. Additionally, the final verdict of each test component and each explicit log statement is present.    Issue: The name �TestLog� is confusing in a way that only the �LogAction� results seem to be included in the resulting behavior. In fact, it is a detailed and exact trace of all events executed in the test case.

Rational: Section 6.3.2 Test Behavior, Subsection FinishAction (Stereotyp description) says the following:   ---  A FinishAction is an action that determines that the test component will finish its execution of the test case immediately. This does not mean that the test component is terminated.  ---    Issue: Extending the abstract metaclass UML::Action (the root of any defined Action within UML superstructure), a tester has to select a concrete subclass of Action of his flavor. Extending a metaclass in terms of a stereotype is not comparable to the object-oriented concepts of extension (i.e. specialization). As a result, �FinishAction� could be assigned to each concrete action in a model, what is too ambiguous. With respect to test model interchange a more precise semantics and metaclass should be defined. It would be better, even for the understanding, to extend a concrete subclass of UML::Action (e.g. UML::OpaqueAction or similar).

Rational: UTP specification 1.0, Section 6.3.4 Time Concepts, Subsection TimOut says the following:   ---  A �TimeOut� is generated by a timer when it expires and may trigger an associated behavior (e.g., a transition in a state machine).  ---    Issue: It is pretty obvious that this is an error resulting from the fact that UTP 1.0 was finalized before UML 2.0 was finalized. UML::TimeTrigger is not a legal, existing metaclass of the UML specification, but there is a metaclass called UML::TimeEvent that can be used instead.

see figure  on page 55 of ptc/2011-07-18

Rational:  UTP specification, Section 6.3.4, Timer Concepts, Subsections TimeOut, TimeOutMessages, TimeOut Action says the following:  ---  TimeOut: A TimeOut is generated by a timer when it expires and may trigger an associated behavior. The TimeOut is placed in the input pool of the object owning the timer.      TimeOutMessage: A TimeOutMessage is generated by a timer when it expires. The timeout message is sent to the active class   that owns the timer.      A timeout is enabled when the timer expires. It may trigger an associated activity. The TimeOutAction occurs, when all input conditions for that activity are satisfied (including the TimeOutAction).  

Since the behavioral descriptions in UML are very different, having dedicated concepts for all kinds of behavior is very pragmatic and useful. Otherwise, one combined stereotyped would need to have additional tagged values in order to be applicable to all behavior kinds. However, this would lead to complex constraints, depending on the behavior kind the stereotype is used in. Therefore, we suggest to close this issue with no change. As a side-note, the resolution for issue 15654 already changed the base class of TimeOut from TimeTrigger to TimeEvent.  Disposition:	Closed  

Issue: For better a recognition and placement inside the UML 2 specification family, a UTP logo should be created.

    Issue: UTP specification 1.0 does not have an ongoing numbering of subsection within the main chapters.  This leads to imprecise references of particular subsections. Other specifications of OMG exhibit such an ongoing numbering.     For example, Section 6.3.1 Test Architecture of the UTP spec contains two subsections "Arbiter" on the same hierarchy level, so that it is not clear how references to either of them should be distinguished.

This issue is directly related to the new outline of the specification (see Issue #15770), and therefore merged with it.   Disposition:	See issue 15770 for disposition  

Issue: There is no clarity concerning the official acronym of the profile. In the namespace of the XMI definition file, u2tp is used as shortcuut. However, since UML 2 is established in both research and industry, upcoming profiles are considered to be profiles for UML 2. UTP as typical 3-characters acronym should be sufficient and contemporary.

The structure of the specification is erroneous in a way, that is not clear why some parts, belonging to the same topic, have been extracted into an own subsection:      Issues with section 6.4 and 6.5:  Section 6.4 �MOF-based Metamodel� consists of the subsection 6.4.1 �Test Architecture and Behavior� solely. Except that using a single, numbered subsection is not a good style, the sibling section 6.5 �Test Data� must belong to section 6.4, henceforth referred as subsection 6.4.2 �Test Data�.   Additionally, the subsection 6.5.1 �Time Concepts� is rather a sibling subsection to 6.4.1 and 6.4.2, resulting in 6.4.3 �Time Concepts�, so that the logically more correct structure is like:  6.4 �MOF-based Metamodel�  -6.4.1 Test Architecture and Test Behavior  -6.4.2 Test Data  -6.4.3 Timer Concepts      Issues with section 6.6 and 6.7:  Section 6.6 �Examples� describes a comprehensive usage example of UTP. Since chapter 6 should be dedicated to the definition of the profile�s concepts solely, section 6.6 should be extracted either to a new chapter 7 or to the appendix (as it is often done in other MOF-related specifications).  Section 6.7 �Mapping� should remain as section 6.6, since it definitely belongs to the specification of the profile.

UTP specification, section 6.4.2 Test Behavior, subsection DefaultApplication (p.19) say the following:  "A default application is a dependency used to apply a default behavior to a unit of testing on a test component. That unit of test behavior must be one of the following: Package, Classifier, Behavior, InteractionFragment, State, Region, or Activity."      Additionally, subsubsection Notation says the following:  "The notation for a default is identical to a Comment (i.e., a rectangle with a bent corner). The text in the comment symbol has the following syntax:          default default-identifier [continue | repeat [repetition-count] ]  If nothing is given following the default identifier, continue is assumed. If no repetition-count is given, infinity is assumed."      Description  ##########################################  The information, given in the notation section, determine how to proceed if a applied default is executed. Continue means that the process flow will continue after the behavior of the default was executed once. In contrast, repeat indicates the number of re-invocations of the default, in a case, when a previous execution has not matched.  Those information are crucial parts of �DefaultApplication�, hence, introducing them just as a remark in the notation section is not enough. Since the continuation information are currently just defined as �textual syntax�, meaning, there are no metamodel concepts for expressing the continuation behavior, it is not clear where (to what property of the extended metaclass �Dependency�) to add these information. Additionally, a parser is required to obtain the information from the text.  To easy both the applicability and understandability those information should be part of �DefaultApplication� as property. By doing so, access to the information is easy to get and to evaluate.

The subsection 6.3.1 � 6.3.4 define the profiles concepts, semantics and descriptions. Those descriptions are the main section of the whole specification, though it must be ideally free from ambiguities, redundancies and impreciseness.   Before introducing the profile�s elements, an introduction section is given.       Unfortunately, those introductions have been used to describe also detailed descriptions of an element�s objective and semantic. Even if an introduction into the following problem space is needful and helpful for the reader, the way how it was done in UTP is confusing, since the semantical descriptions are scattered over several subsections and sometimes even redundant.       This affects the following elements and their descriptions:  - 6.3.1: Arbiter, Scheduler,  SUT  - 6.3.2: Verdict, Default, FinishAction, LogAction, determAlt, TestLog  - 6.3.3: Data Pool, Data Partition, Data Selector, Coding Rules  - 6.3.4: Timer, TimeOut, Timezone    In order to keep the specification clean and easy to read and maintain, it should be precisely identified, what parts of the description should rather be positioned in the element�s subsections and what could remain in the introduction.

see figure on page 68 of ptc/2011-07-18

Chapter 4 �Terms and Definitions� copes with defining a common terminology for the further specification. Its subsections reflects the logical packages of the UTP, hence it should be rather a subsection of chapter 6.       Chapter 6 �The UML Testing Profile� deals with the profiles and its elements exclusively, so adding a subsection for defining a common terminology and providing an overall overview of the subsequently discussed profile and MOF-based metamodel (which are structured by the same logical packages) fits into the intention of chapter 6. It also concentrates the domain-specific knowledge of the profile�s concepts in the core chapter, what may help reader to find and retrieve information on a particular element.    Additionally, chapter 4 must be thoroughly revised regarding its content. Currently, there is no agreement of what is included in the sections. For example, section 4.1 �Test Architecture� contains all profile element�s (and additional, logical concepts being no part of the profile) whereas subsection 4.4 �Timer Concepts� just discuss a subset of the profile element�s.

In section 6.6.1 Money Example, Figure 6.19 shows a generalization relationship between IMoney and Money/MoneyBag.      The prefix "I" supposes the IMoney classifier to be an interface. If so, the generalization relationship is wrongly used. Instead, this should rather be a InterfaceRealization (dashed arrow).    But if so IMoney classifier is supposed to be a class, the prefix "I" should removed. This would require the class to be renamed, to avoid name clashes with the specialized class "Money".

The second constraint of TestLogApplication seems to be wrong. Currently, it says:      "[2] The supplier of a test log must be a named element with a test log stereotype applied"      This must rather be:  "[2] The supplier of a test log application must be a named element with a test log stereotype applied"                                             Rational: A dependency going from a �TestLog� to a �TestCase� or a �TestContext�.    Discussion: The direction of <<TestLogApplication>> currently points from a <<TestCase>> to a <<TestLog>>. Given the UML semantics of UML::Dependency, this would be interpreted as the test case is depending on the test log. Actually, this must be switched, since a test case specification is complete and not depending on the test log elements. Rather, a test log is depending on a test case/test context, expressing that this test log was created due to the test case specification it points to.  [1] The client of a test log application must be a named element with a test case or test context stereotype applied.  [2] The supplier of a test log application must be a named element with a test log stereotype applied    

The specification is ambiguous regarding the usage of its correct name within the descriptional sections. Sometimes, "UML Testing Profile" is mentioned, sometines "Testing Profile" is mentioned.    In order to increase editorial consistency and cohasion this should be consequently changed to either "UML Testing Profile" or "UTP"

Within the descriptional sections, the specification often refers to stereotype names, without showing the typical stereotype brackets (��).   For example:      "A data partition stereotyped classifier..." may be changed to  "A �DataPartition� stereotyped classifier.."      or      "The notation for data partition is a classifier with stereotype data partition."  may be changed to  "The notation for data partition is a classifier with stereotype �DataPartition�."    This should be consequently done along the descriptional sections to increase redeability and consistency, also regarding other UML profile specifications.

Rational:  UTP spec, 1.0, section 6.3.1 TestArchitecture, subsection TestContext, subsubsection Description mentiones the following:   "A structured classifier acting as a grouping mechanism for a set of test cases. The composite structure of a test context is   referred to as test configuration. The classifier behavior of a test context is used for test control."      Issue:  Since a test context extends UML::StructuredClassifier solely, a test context will never have a classifier behavior or operations, representing the test cases of that test context. StructuredClassifiers do not care about behavior but about structure. In order to be able to add a classifier behavior to a test context, it is required, that test context stereotypes only those classifiers that are subclasses of UML::StructuredClassifier AND UML::BehavioredClassifier (context TestContext inv: self.base_StructuredClassifier.oclIsKindOf(StructuredClassifier) and self.base_StructuredClassifier.oclIsKindOf(BehavioredClassifier).    Even if in the current UML specification (haven't checked this thoroughly) all concrete subclasses of StructuredClassifier are subclasses of BehavioredClassifers, too, UTP should formally define this constraint since it is a crucial aspect of test context semantics

Resolution is to include the metaclass BehavioredClassifier as well into the normative description and semantics of TestContext.  Disposition:	See issue 16163 for disposition  

There is missing support of test management issues within the current version of UTP in contradiction to the fact, that test frameworks usually are supported by a tool based or manual test management process.  One of the major tasks of test management is to decide, which test cases must be executed. In safety-critical environments, good rationale shall be given if test cases are ommited. To establish corresponding process interfaces from model based test design to test management, the test models have to be enriched with test management features.  Currently, UTP test cases are supposed to be independent of the specific configuration of the SUT. If product variants are tested, specific test cases may be required that only apply to one or several variants (but not all).   This is very important to fulfill basic regulatory affairs, e.g. IEEE 826 defining the content of test plan and test design.  Example 1: A test case attribute "priority" in the model can be used, to include the design of prioritized test sets within the model used for stakeholder review and later for test suite management in the test management tool used.  Example 2: Explicite tags for requirements as model attributes to test cases enable a traceable and efficient requirements based test design with UTP models and can be used as a basis for tool interfaces / synchronization tools between requirements engineering and test design.

see figure on page 105 of ptc/2011-07-18

Especially for manual tests and in regulatory domains like Medical IT or Automotive additional test execution information has to be logged (e.g. tester's identification).   A standard is necessary for potential process support coming out of an UTP test model.

This issue is a subset of issue 15913 and will be merged with it.  Disposition:	See issue 15913 for disposition  

A standard for attributes supporting review processes out of / with UTP based test design (e.g. author, change history for the TestCase element) is missing in the current version of UTP.

Will be closed with no change. Adding information for reviews and audits is usefull, but not a dedicated test-specific concept.   An issue will be raised at the UML that helpful informtations for audits and reviews should taken into account, since this is a general purpose requirement.  Disposition:	Closed  

Spell out all acronyms at first use with the document, e.g. see first page, UML, MOF, and XMI    

Change �SUT�   The word system is too overloaded and causes us to think of an entire system. Much of the testing is not done on the entire system but on parts of the system as subsystems, components or units before they are integrated. In fact, there can be multiple �SUT� items defined in a single test context.       Resolution:     Suggest changing it to �IUT� �Item Under Test�.    

The resolution of this issue is clear subset of the resolution of issue 16105 and therefore, will be merged with it.  Disposition:	See issue 16105 for disposition  

Source: Lockheed Martin, Jon Hagar    Specification:   UML Testing Profile  Section:         tbd section    Nature: Issue  Severity: med  Summary:     Verify Relationship  The verify relationship, a �verify� stereotype dependency, is defined in SysML (section 16.3.2.7) and is used from a test case to a requirement. It can also be used between a test context and a requirement.  There is nothing specified in the Testing Profile.       Resolution:          The UTP should state that it uses the �verify� relationship from the SysML specification, and have sections/diagram created to show this.    

Show an additional example of how UTP might use stereotypes from SysML to define test specifications and maintain requirements traceability

Source: Lockheed Martin, Jon Hagar    Specification:   UML Testing Profile  Section:         all    Nature: Issue  Severity: med  Summary:     testcase or TestCase  In the UML testing profile the stereotype is �TestCase� in the SysML profile it defines �testCase�. If both profiles include it, which one is should be used? In the same model, you do not want one team using one and another using the other.       Resolution:     Coordinate the profiles so there is one Test Case stereotype for both, the SysML profile and UTP, i.e., update UTP to match SysML profile.      Revised Text:   Actions taken:    

There was a long discussion between UTP and SysML RTF. The outcome of this is that the UML profiling mechanism is not powerful enough. Since this is not an issue related to UTP, we will deal the following way with the issue.  Close the issue for UTP  Submit an issue to UML regarding deficiencies of the profiling mechanism   Submit an issue to SysML to synchronize the UTP related concepts with the latest UTP specification  Disposition:	Closed  

Test Objective  1.       In the OMG Testing Profile (OTP) specification a Test Objective is defined both as a stereotype on a dependency, �TestObjective�, and as an artifact elsewhere.  The �TestObjective� stereotype does not apply to this artifact, just to the dependency.     2.       The Test Objective dependency relationship is intended to show the dependency from a Test Context or a Test Case to a model element that contains the textual string stating the objective of the test, the test objective artifact.          Resolution:     Test Objective as a dependency:     a.       Change Suggestion 1 - No longer use the stereotype �TestObjective� for this dependency relationship. A dependency relationship stereotype name is typically a verb, like verify, satisfy, refine, and copy. Using a noun phrase like �TestObjectiveon� to name a dependency does not provide the declaration that is needed, i.e. subject verb object, e.g. �BlockA satisfies requirement 1�,  or �Use Case A refines requirement 1�.    b.      Change Suggestion 2 � Use the verb �Validate� as the name of the stereotype that is associated with a dependency.  This stereotyped dependency can be used from a Test Case or Test Context to a Test Objective but, can also be used between a test case and model elements that state objectives/goals/capabilities of the item being validated.     c.       Why use validate?  � Others were considered such as satisfy, realize and implement.                                                                    i.      The validation definition used by test is �Validation is the set of activities ensuring and gaining confidence that a system is able to accomplish its intended use, goals and objectives (i.e. meet stakeholder requirements) in the intended operational environment� , or �Did we build the right thing?�  These system uses, goals and objectives are represented in the model in the form of use cases, MOEs, TPMs, etc.                                                                   ii.      There is a clear distinction between verification and validation. Based on this, there will be unique Test Contexts for verification and validation.                                                                 iii.      Just as we need to show what test cases will be verifying what requirements we will also need to show what test cases will be validating objectives and goals of the system (or the thing being validated). Therefore, this relationship will ultimately be needed just as verify is used today to verify requirements.                                                                iv.      This relationship can also be used to show what test cases are validating the objectives stated for a Test Context. It is the same relationship pattern.           Test Objective part 2  1.       A Test Objective artifact is a textual statement stating a goal or objective of a test context. The OTP specification doesn�t define the type of model element to use for this artifact.     Resolution:          1.       Test Objective artifact suggestion �     a.       We have selected a comment stereotyped �TestObjective� to capture the Test Objective artifact for a Test Context. However, other model elements could be used, such as an attribute of the Test Context. The advantage of a comment is that it can be reused across test contexts  

  Deferred due to time restrictions

Source: Lockheed Martin, Jon Hagar  Specification:   UML Testing Profile  Section:         sec 4  Nature: Issue  Severity: med  Summary:     Section 4 defines a series of definition and terms unique to the UTP, but the UTP uses other testing terms within the document, e.g. test, verification, validation, test plan, and other.  These do not always have comment usage in industry.     Resolution:          We  suggest a common list of test terms be provided either directly or by reference to an ISO or IEEE type of standard  

The merge of contents, addressed by issue 15773, will solve this issue.  Disposition:	See issue 15773 for disposition  

SysML also defines a stereotype �TestCase� on "Operation" and "Behavior" as well as a stereotype �Verify� between requirements and test cases. This should be reflected in the UTP specification as well.

This issue will be merged with issue 15928 for resolution

UTP 1.0 provides little support specification of large datasets that are of complex structure. Such data set are typical in test cases for information systems that process complex business objects (e.g. contracts). Furthermore the concepts "DataPool", "DataPartition", and "DataSelector" are based on a programmer's procedural paradigm and are currently not specified proper stereotypes.

The current data package must doubtlessly be thoroughly revised. However, data partitions are quite useful to express equivalence classes for a given base type (currently done with generalizations, but this does not scale). The issue rather points to the inflexible extension or specification capabilities of instance specifications in the UML. Currently, instance specifications cannot participate in a generalization dependency, preventing reusage of partially defines instance specification in order to avoid redundancy. This seems to be an issue for UML, however, since instance specification are more vital for testing pruposes than for architecture design purposes, UTP should address a refinement mechanism for instance specification in the next minor revisions.  Necessary concepts:  Extension of already defined instance specification  Refinement of slots  Overriding of slots  Disposition:	Deferred  

What is the type/class of the instance specification called "self" in figure 6.25?

The UTP contains a stereotype <<TestObjective>>, connects either a test context of test case to any model element, expressing the purpose/objective of the test case or the test context. However, most standards related to testing define standalone specifications of such a test case's target in forms of test purposes (ETSI) or test requirements (IEEE/ISQTB).      It would be good to have a dedicated stereotype <<TestRequirement>> to express the test case descriptions, similar to system requirements (for example stereotype <<Requirement>> in SysML).     It should have atleast an ID, a name, a description and a priority

Deferred due to time restrictions

The following profile elements do not provide a semantics section:      - SUT  - TestContext  - TestLog  - TestLogApplication  - TestObjective    For a precise and unambiguous interpretation, a description of the semantics is important and should be supplemented.

A master issue for the rework of test architecture package. Several issues will be merged into this issue to improve readability and understandability of the resolutions

see figure on page 75 of ptc/2011-07-18

IN both changebarred / no-changebar documents for UTP (i.e., 11-06-10 and 11-06-09) do the following:      Keep:      � [OCL] Object Constraint Language (OCL), version 2.2, formal/2010-02-01, 2010       Replace:      [UMLi] Unified Modeling Language (UML) Specification: Infrastructure, version 2.3, formal/2010-05-03, 2010   � [UMLs] Unified Modeling Language (UML) Specification: Superstructure, version 2.3, formal/2010-05-05, 2010   � [MOF] Meta Object Facility (MOF) Core Specification, version 2.0, formal/2006-01-01, 2010       with corresponding names for:      https://www.omg.org/spec/UML/2.4.1  https://www.omg.org/spec/MOF/2.4.1  https://www.omg.org/spec/XMI/2.4.1        I have to produce the SysML1.3 normative / SMSC-kosher files in XMI anyway, I'll do the same for your profile.  I'll send you XMI2.4.1 profiles for UTP normative & informative files that extend UML2.4.1.      Change the inventory such that the following files are: informative machine-readable artifacts:      https://www.omg.org/cgi-bin/doc?ptc/11-05-15  https://www.omg.org/cgi-bin/doc?ptc/11-05-14      In the inventory, add:      https://www.omg.org/cgi-bin/doc?ptc/xx-yy-zz (the XMI2.4.1 / UML2.4.1 version of 11-05-14 -- normative -- without test case management  https://www.omg.org/cgi-bin/doc?ptc/xx-yy-zz+1 (the XMI2.4.1 / UML2.4.1 version of 11-05-15 -- informative -- with test case management      Get the numbers from Juergen. I don't know if you want new numbers for the whole series of UTP or if Juergen can get you 11-05-XX/XX+1 numbers for the two additional artifacts & the new versions of the UTP documents  that say UTP1.1 extends the 2.4.1 series of UML/XMI/MOF.      Finally, add another resolution to explain the changes in your updated report (i.e., 11-06-08)    

This issue is probably coupled with (at least related with issue #15941 https://www.omg.org/issues/uml-testing-profile-rtf.open.html#Issue15941) and targets a more convenient handling of UML instance specifications for test modeling to avoid redundancy.      TTCN-3 offer quite efficient and proven concepts to exploit already defined/created instance specifications (called template in TTCN-3). The concept is called "modifies" and allows the derivation of new templates by reusing existing templates.    See Clause 15.5 of TTCN-3 core specification (ETSI ES 201 873-1 V4.2.1) for further information.

In most situation in the UML superstructure, the abstract metaclass ValueSpecification is used when flexibility should be granted to the user in how to model concrete values for an attribute/tag definition.      Using ValueSpecification allows a clear way for tools to deal with different ways of how user model values.      In the test management extension chapter, the tag definitions are typed as semantic-free string types, which makes it hard to calculate them automatically and to interchange the models.      Going from String to ValueSpecification would allow user to define their UML_compliant way of value modeling and still not restrict the user to a certain modeling method.      Example:       TestCase{  priority : ValueSpecification [0..1]  }      A user could therefore provide an enumeration for priorities he uses in his test process like      MyPriorityKind{  low  medium  high  }      This allows the user to fill the priority tag definition of test case with a user-specific value of priority.      instance:TestCase{     slot{        definingFeature = priority        value=InstanceValue{           type = MyPriorityKInd           instance = low        }     }  }      Possible solution:   Change tag definition types in the test management extension chapter from String to ValueSpecifcation

This is a correction for issue #15652:      In the resolution for issue #15652 we restricted the number of <<TestComponent>> lifelines that are covered by a <<DetermAlt>> to one.      This is too restrictive. There can be of course several test component lifelines being covered by the very same <<DetermAlt>>.      Possible resolution:  Remove the first paragraph in the semantics section      Change the first sentence of the current second paragraph from      If a deterministic alternative is reached during the execution of an interaction, the involved test component waits until it receives an input      to      If a deterministic alternative is reached during the execution of an interaction, the involved test component lifeline waits until it receives an input    - Remove constraint [2]

TestLog describe the actual execution of a test case against the SUT it was stimulating. A TestLog is described as concrete subclass of Behavior.      For particular behavioral descriptions (Activity, Interaction) the tets log allows the integration of more details for the exeuction of a single test step, e.g. a timestamp when this test step (sending of a message, performing a validation action, ...) has been carried out by the test execution environment.      Therefore, there should be at least for test log's represented as Interaction and Activity a concept like the following:      TestLogEntry extends OccurrenceSpecification/Action{      timestamp : UML::TimeExpression [0..1]  ...  }

Since TestComponents can define test-relevant behavior, it must be ensured by definition that each base class of a TestComponent is also a BehavioredClassifier.    Behaviors for test cases can be expressed as operations in the test component, for example. To make this formally and by definition possible (and if we do not change the base class of TestComponent to Property - see previously submitted issue by me), we should also extend BehavioredClassifier and make clear that a base class, to which <<TestComponent>> is applied must be both a StructuredClassifier and BehavioredClassifier.

The term 'LogAction' actually refers to the element FinishAction. So it should be renamed to 'FinishAction' and a new term for 'LogAction' should be included

UTP is a specification for the creation of model-based test specififcation, thus, it has an important and meaningful role in the realm of model-based testing.      Unfortunately, the introductory chapters do not mentione, nor define, model-based testing, test model or any relevant concept related to MBT techniques. As the first and the most sophisticated standardization approach of MBT concepts, the UTP specification should be much clearer on its purpose and relationship to MBT.    In fact, being incepted and mostly written almost 10 years ago, UTP needs a modernization polishing.

Currently, UTP consists of four conceptual (not technical) packages: test architecture, test behavior, test data and timer concepts.      The concepts which are described within those packages (which are represented as different subsections in the normative section of the specification) can be further separated.      For example, timing (all concepts of timer concepts package) is mainly associated with test behavior. A timer can only be started if a behavior is executed. It might be more consistent to move the concepts from timer concepts into test behavoir as well, since it is clearly dedicated to test behavior.      Another example are some concepts which clearly belong to the test management area like TestObjective and/or TestLog. Those test management concepts are part of UTP since its adoption by OMG, so it would just be consequent to establish a new package exclusively for test management concepts.      A more consistent conceptual package structure could look like this:      1. Test Architecture  2. Test Behavior  2.1 Test Case and Defaults  2.2 Test Actions  2.3 Test Timer  3 Test Data  3.1 Wildcards  3.2 Test Data Structure  4. Test Management

During UTP 1.1 revision, the entire document structure has been modernized, cleaned-up and restructured. However, there are several typos and style glitches in the current specification, which mostly originated from copy and paste errors.    In addition, some of the figures need polishing as well (e.g. Figure 7.8, Figure B.14, Figure B.23, Figure B.26, Figure B.28)

There was an error introduced into the textual definition of ValidationAction during the sophisticated restructuring of UTP 1.0 to UTP 1.1.    So actually, ValidationAction should extend CallOperationAction instead of Dependency (as it is correctly depicted in the abstract syntax)

In UTP 1.1 there are two predefined time-related types: Time and Duration. Time is used for starting a timer (Timer::start(expre:Time)), so it represent rather a Duration. In addition, time is ambiguous, because it is not clear what time actually represents, a particular point in time or a duration of time units.      Since UTO already defines a Duration type to express a duration of time unit, Time should be renamed to Timepoint to cope with particular points in time. Concrete timepoint formats are not given by UTP.     Furthermore, Timer should offer a second start() method with the signature Timer::start(duration:Duration). The user would be able to decide whether a time should timeout at a certain point in time or after a given duration.

Most test execution systems allow the definition of a reason why a particular test cases has failed or passed.      In TTCN-3, for example, the user can attach a string message to the setverdict() operation indicating the reason that describes verdict from a logical and human-readable perspective.      In JUnit the same can be done within the Assert.XXX methods.       Such a human-readable message guides the analysis commonly performed by testers and developers, by providing a logical explanation (logical in the sense of what does the verdict state with regard to the intention of the test case) to the technical verdict.      A message "The system was expected to respond y and when X was submitted" is more obvious and can be understood even by people outside of the testing/development process than a simple FAIL, where the logical reason behind that fail has to be derived from the stimuli and corresponding reponse within the test case execution trace.      To support this, ValidationAction should have an additional attribute 'reason : String [0..1]'    This would just be consequent, since a TestLog in (the extended version) UTP is able to expose the reason why the final verdict has beedn assigned to the test log.

The repetition attribute has not a clear semantics with respect to the statements given in the Default section. Repetition actually represents how often the control flow will jump back to the unit of testing that actually applies the default.