Test grammar

Hey all,

When I was working on making the homebrew tests for ACE, I ran into the question of what is an actual good test for confirming ACE works end to end? For better or worse, I made the decision to attempt to create the smallest possible grammar that would still generate a parse by hacking away at matrix.tdl.

The result is https://github.com/dantiston/delphin-tiniest-grammar. As noted in the repo, it parses one string ("n1 iv") and it generates the following MRS:

SENT: n1 iv
[ LTOP: h0
INDEX: event2
RELS: < [ "_n1_n_rel"<-1:-1> LBL: handle3 ARG0: ref-ind4 ]
 [ "_iv_v_rel"<-1:-1> LBL: handle1 ARG0: event2 ARG1: ref-ind5 ] >
HCONS: < h0 qeq handle1 > ]

I have some questions.

  1. I don’t remember all the details about my problem, but I struggled to get the bare-np phrase to work, and subsequently the MRS does not have a quantifier relation. I am under the impression that means this is an invalid MRS, is that correct? If so, is it worth it to get the quantifier relation in, either forcibly or through the bare-np phrase?
  2. The grammar doesn’t have any lexical rules; should it to be useful?
  3. I tried and failed to get generation to work. I’m not sure if it didn’t work because I cut something important out (something from semi.vpm?) or some other reason. I suppose testing generation is just as important as parsing?
  4. Does anyone else think this would be useful for their projects (or for pedagogy)? If so, should we move it to the main delphin repo?

Also, I haven’t tested it with the LKB yet :slight_smile:

That seems like a fun exercise :slight_smile:

Re (1), I think you’re correct about the MRS. It’s generally not considered well-formed if nominal things don’t have quantifiers. That said, there’s some wiggle room. I think we used to not quantify pronouns, and there’s been discussion of just dropping default quantifiers like udef_q. So it’s up to you, I guess?

Re (2), depends on what you mean by “useful”. Do you want a minimal grammar that touches all the parts of our parsing machinery? Or merely something that compiles and parses? If the former, there are other things, too, like morphosemantic properties (and appropriate VPM rules for them), trigger rules for generation, and CFROM/CTO values (token mapping maybe?).

Re (3), the lack of a VPM may be why generation isn’t working. I would add one to map those variables from grammar-internal names (handle, event, ref-ind) to external names (h, e, x). For some reason the top h0 is not handle0. Regarding importance, see my response to (2) above.

Re (4), the Copestake book on TFS grammars has a minimal-ish grammar, IIRC. But that book is outdated in some areas, and you might offer your grammar as a refresh of that idea.

1 Like

I was thinking to suggest the same, but your comment about the book being outdated in some areas worries me! How can I identify these areas? Where can I find more up-to-date references? Maybe Emily’s courses material?

If you want to teach grammar engineering, I think Emily’s or Francis’s course materials are probably more approachable.

If you want to build tools around DELPH-IN technologies (my use case), some specifics are out-of-date and it, understandably, doesn’t include any developments since 2002. In particular, I used it as a reference for TDL syntax (which is only a minor part of the book), but the syntax has since changed a bit, and http://moin.delph-in.net/TdlRfc is a more complete and current document now.

I think a more compelling use case for it is to learn a bit more deeply about the implementation and theoretical side of our grammar-engineering paradigm. I strengthened and expanded upon what I learned in Emily’s HPSG and grammar engineering courses regarding type hierarchies and unification, as well as some more obscure ideas like default unification.

1 Like