Error adding regression tests

regression_tests

#1

I’m trying to add a test to the regression tests, and I keep getting this error:

$ ./matrix.py regression-test-add evidentials-auxiliary-choose-choices evidentials-auxiliary-choose-test_sentences
Enter a short comment describing this regression test: testing evidential auxiliary verbs with selected evidential term inventory
FAIL!
Traceback (most recent call last):
  File "./matrix.py", line 945, in <module>
    main()
  File "./matrix.py", line 195, in main
    lg = gmcs.regression_tests.add_regression_test.add(choices, txtsuite)
  File "/home/mhaeger/delphin/matrix/branches/mhaeger-evidentials/gmcs/regression_tests/add_regression_test.py", line 106, in add
    shutil.copy(os.path.join(rt_root, "home", "gold", lg_name, "item"), os.path.join(rt_root, "skeletons", lg_name))
  File "/usr/lib/python2.7/shutil.py", line 119, in copy
    copyfile(src, dst)
  File "/usr/lib/python2.7/shutil.py", line 82, in copyfile
    with open(src, 'rb') as fsrc:
IOError: [Errno 2] No such file or directory: '/home/mhaeger/delphin/matrix/branches/mhaeger-evidentials/gmcs/regression_tests/home/gold/evidentials-auxiliary-choose/item'

Any ideas on what’s causing it, and how to fix it? In my VM with the LKB running on Ubuntu, these test suites parse fine in [incr tsdb()], so I’m thinking it’s my regression testing set-up, not the tests themselves.

Thanks!


#2

Did you check that both CUSTOMIZATIONROOT and ACEROOT are set up correctly? In particular, check that CUSTOMIZATIONROOT is set to the branch that you are working on rather than the trunk.


#3

You know, I was certain I had the root variables correct because I can run the regression tests just fine, but I had ACEROOT pointing to ~/ace/ when all the ace stuff was in ~/ace/ace. Updated ACEROOT, and it works fine now. Thanks for making me take another look.


#4

Great! If you like, you can check the “solution” box; it’s hidden under the “…”, to the left of “Reply”. This way whoever reads this post will see immediately what worked.


#5

I’m seeing this error even though my CUSTOMIZATIONROOT and ACEROOT are fine. As it turns out, it looks like a bug (or at least undocumented behavior) in the interaction between customization.py and add_regression_test.sh.

Here’s what I figured out:

  1. Customizing a choices file to a non-existing directory creates that directory and builds the customized grammar in it.
  2. Customizing a choices file to an existing directory creates a new directory to hold the customized grammar; that new directory is named using the iso-code from the choices file if set, otherwise using the language from the choices file.
  3. add_regression_test.sh ensures the directory gmcs/regression_tests/scratch/grammars/ exists.
  4. add_regression_test.sh calls customize with `gmcs/scratch/grammars/ as the output directory.

The result is that if you set an iso-code in your choices file, then the generated grammar will be named with it. Unfortunately, everything else in the add_regression_test process is using the language from the choices, so ace will not be able to find the generated grammar.

The simple workaround is not to set an iso-code for regression tests based on languages that have them; there’s also a one-line change to add_regression_test.sh that probably fixes it. This is also captured in trac; I’ll fix it when I have a moment again.