This is a hand-created mockup of a test catalog, created to help myself visualize how things should fit together.
Revisions: 2021-01-25 : CMSMcQ : began making the file.
It is inspired partly by examination of the XSD and QT3 test suites, and partly by introspection. The XSD test suite has firm ideas about what belongs in a document together and what must be in a separate document; in particular, it requires test cases and the catalogs for test sets (the constituent parts of a test suite) to be in separate documents. The QT3 test suite is explicitly more relaxed about that, at least as regards actual test cases. Having the test-case input internal to the catalog makes some tasks easier, and avoids filling a directory with many many very small files.
So as a design principle I would like to say that both test
cases and test sets can occur either inline or externally.
So there will be test-set and test-set-ref
elements, test-case and test-case-ref
elements. And similarly for other things I discover I want to
have either inline or external.
This is a very small grammar, with a very few test cases. Everything is so small it fits nicely inline.
The grammar is one of the smallest that illustrates center embedding.
This collection of tests illustrates one important pattern for test sets: inputs are a grammar in ixml notation and some strings which do or do not belong to the language defined by the grammar. Expected results include the XML form of the ixml grammar (that is, the result of parsing it against the standard ixml grammar for ixml) and the XML form of the strings.
Other patterns may exist, but for now, I think this works.
This set of test cases is constructed using 'arc-final' coverage: for each arc in the FSA used to generate the test cases, a test case is constructed that begins in the start state and ends by traversing that arc. If the arc's target state is final, the test is a positive test (and not included here); otherwise it's a negative test and is included here.
This test case illustrates the general principle
that everywhere in the test catalog, namespaced
attributes in other namespaces are allowed. The
test case was generated by Gingersnap, which uses the
gt:* attributes to provide some metadata about the
test: the gt:trace attribute identifies the states
and arcs visited by this test, in the finite state
automaton from which the tests were generated.
The gt:polarity attribute specifies that the
input string is not a sentence. (It's redundant,
since in this catalog sketch that information is
conveyed by the result element
containing assert-not-a-sentence.)
But I have kept it around to illustrate the point.
This set of test cases is constructed using 'negative arc-internal' coverage: for each arc in the FSA used to generate the test cases, one or more test cases are constructed that begin in the start state, traverse that arc, and continue, ending in a non-final state.
This test set illustrates cases in which the input string and/or the expected output are external.
It also illustrates the provision of other information by embedding it in app-info.