What does mrs-compare actually compare?


#1

I’ve got some regression tests failing after my latest edits in my branch because of MRS differences. After staring at the gold and current MRSs for a while, the only differences I can see are the order of the things on the RELS list and the exact variable names (e.g. x2 vs x7). All the same things seems to be coindexed in both cases, and the same rels are present. It seems like ordering in the RELS list and variable names shouldn’t cause mrs-compare to declare these MRSs different, but I can’t see what it’s actually doing because it’s a binary file whose documentation I can’t locate. Anyone know if these differences would actually be considered diffs? Or is it more likely I missed spotting some other difference?


#2

Also, if there’s a way to take the unadorned text MRSs and generate the pretty, interactive MRSs, I’d be very interested to know about it :slight_smile:


#3

I haven’t checked the code, but I seem to recall Woodley (the author of mrs-compare) saying that he does a string comparison of the two MRSs after doing a deterministic relabeling of the variables. If he doesn’t reorder the MRSs, that could cause differences. I could be wrong about this, because this seems overly brittle.

— edit: I was misremembering a comment in the mrs-compare.c code. It’s actually compared by doing mrs_subsumes(x, y) && mrs_subsumes(y, x), which I think is the same as calculating a variable bijection, but I’m not sure of the details. :end edit—

We’ve talked about replacing it with PyDelphin’s delphin.mrs.compare.isomorphic() function (see https://github.com/delph-in/pydelphin/wiki/delphin.mrs.compare#isomorphic), which calculates a bijection of variables (if one exists, they are isomorphic), but I don’t think such a change has been committed yet.


#4

As for browser-based MRS visualization, see here: https://github.com/delph-in/delphin-viz

There may be ways to send a stored MRS to LUI, if you want the visualization like in the LKB or ACE.

And if you just want syntax highlighting at the commandline, try PyDelphin’s convert subcommand (of the delphin or delphin.sh command): https://github.com/delph-in/pydelphin/wiki/Command-line-Tutorial

For example:

$ ace -g grm.dat -Tq <<< "test sentence" | delphin convert --pretty-print

Note that it looks best if your terminal is in “dark” mode (otherwise the yellow is hard to read).


#5

Thanks for the pointer to all these super useful tools! I’m going to make a lot of use of them. I ended up finding the difference via the time-tested “stare at it a very long time” method – some of the features on one of the arguments were different. So I’m guessing the order of things on the RELS list isn’t an issue, as you’d expect.