Natural language regression test names

regression_tests

#1

I’m working on adding my development (illustrative) and eval (held-out) languages to the regression test suite, and noticed that we don’t seem to have a convention for naming them.

I’d like to propose something along the lines of <library-prefix>-<iso>. (For example, my Lakota test suite for valence change would be valchg-lkt.) My rationale is:

  1. It maintains the ability to run the library-specific group of tests by using the prefix, and
  2. It makes it at least somewhat explicit that the test grammar, although based on a human language, is designed as a test for a specific purpose and not as a representative grammar/fragment of the language.

The message should be, I think, “this isn’t guaranteed to be a coherent subset of Lakota; it’s not even guaranteed to have the same analyses I’d do for a full grammar.”

Does this sound reasonable?


#2

Sounds like a good idea to me; I would probably do the same.


#3

I think a convention is needed so I’m glad you bring this up. That sounds good to me.