Comparing HCONS and MRS, example of a regression

After I changed everything in my branch of the customization system to append-lists, I am getting some regression test failures related to the order (?) in which things are added to the MRS, for example:


  1. Is this a bad change (I am assuming it is), i.e. do we want all that info about the noun on the noun, not on its exist predication?

  2. The h5-h6-h7 indices situations look equivalent to me, i.e. in gold h6=h4 and h6 is exists_q’s RESTR; in current, h5=h7 and h5 is exist_q’s RESTR. h4 is the noun’s LBL in gold and so is h7 in current. So, no problem here?

Hi Olga,

Which EP the properties appear on makes no difference as long as they’re on the correct variable. They could even show up on the ICONS variables, for instance, but generally they appear on the first occurrence of a variable in the MRS.

Speaking of ICONS, I notice that the top MRS has none while the bottom one has:

ICONS: < e2 non-focus x13 >

I think this is the source of the failure, assuming you’re using PyDelphin (or to do the comparison. I don’t recall if Woodley’s mrs-compare or [incr tsdb()]'s compare-in-detail look at ICONS.

OK, so the stuff I asked about is a non-diff.

The ICONS diff, I think we’ve seen it when we were first testing rtest. Did you arrive at a conclusion, is it a good diff, should I just be updating the gold in such cases?

I think so. ICONS info is important, too, otherwise Sanghoun’s regression tests are mostly meaningless.

Hmm I wonder though how I am going to figure out what the correct MRS is, in each case… I suppose when ICONS is missing entirely from gold (but, btw, why would it be missing? because it looks like it literally is missing from the gold MRS) and is present in current, then perhaps we can assume that some ICONS is better than no ICONS. But other than that I am not sure I can go through hundreds of regression tests and figure out what is going on…

…but, that’s another topic, I think: InfoStr regression tests: running them with rtest

My question was about HCONS and that was answered (that it is not a diff).

Yes, as mentioned in the thread you linked, PyDelphin only prints non-empty ICONS lists. ACE seems to always print them IF the grammar enables them (probably in ace/config.tdl). If this is important there might be something I can do about keeping empty ICONS lists in the MRS.

If the gold version of your profile did not have ICONS and the current one does, that means that some change in the grammar introduced an individual constraint. It’s up to you as grammar developer to decide if those ICONS are intended or spurious. Updating gold analyses is just part of the process :frowning:

There’s two ways I can think of that these examples could go from not having ICONS to having ICONS:

(1) There’s a change in the structure (which rules are firing) because of some changes you made. This can be investigated by looking at the derivations (and not just MRSs) in the two profiles.

(2) Somewhere along the line, the ICONS got lost for some non-grammar reason but someone (illegitimately) updated the gold profile. To investigate this, I’d look at the commit history on those specific profiles. When were they changed and were there any comments?