This is mainly a note for @sweaglesw and @johnca, but others may be interested.
@Dan and @bond contacted me about a few TDL variations in NorSource that PyDelphin wasn’t able to parse. I think ACE and the LKB were able to parse them and I don’t think any changes are needed there, but I want to be up-front with the new behavior in PyDelphin to ensure we’re in sync.
Comments after letter-set and wild-card
This one is trivial. PyDelphin did not allow comments to appear on the same line as letter-set and wild-card declarations:
%(letter-set (!a abc)) ; some comment
The EBNF of TDL syntax on the TdlRFC wiki did not allow for these comments. However, I see no reason to forbid comments in that position (as opposed to within the declaration), so I’ve changed my parser to allow them and I’ve updated the EBNF:
- letterset : "%(letter-set" SPACE? letterset_def SPACE? ")"
- wildcard : "%(wild-card" SPACE? wildcard_def SPACE? ")"
+ letterset : "%(letter-set" SPACE? letterset_def SPACE? ")" SPACING
+ wildcard : "%(wild-card" SPACE? wildcard_def SPACE? ")" SPACING
Do let me know if you foresee any problems here.
Implicit AVMs and Conjunctions
These two require no change to the grammar, but a change in how I was modeling the TDL objects. In a conventional type definition, you’d see a feature path followed by a conjunction of a type and an AVM, like this:
my-type := supertype & [
FEATURE some-type & [ SUBFEATURE constraint ] ].
But NorSource had two variations involving repeated feature paths that I hadn’t anticipated. First, an implicit AVM on the inner type via dot-notation:
my-type := supertype & [
FEATURE some-type,
FEATURE.SUBFEATURE constraint ].
… and, second, an implicit conjunction:
my-type := supertype & [
FEATURE some-type,
FEATURE [ SUBFEATURE constraint ] ].
For the first, PyDelphin raised an error because it failed to subscript SUBFEATURE on a non-existent AVM on the value of FEATURE (dot-notation already created implicit AVMs in PyDelphin, but not if there was already a value on the feature path). For the second, the second value of FEATURE replaced (i.e., reassigned) the first. Neither of these were intended behaviors and, thus, both were bugs.
I have changed PyDelphin to allow for these variations by creating implicit AVMs and conjunctions as necessary so the constraints don’t get lost. One limitation is that they will not be re-serialized with the repeated features and instead will use the explicit conjunctions and AVMs as in the first example above.
I think these problems were due to an incomplete modeling of TDL objects in PyDelphin and so ACE and the LKB are probably not affected, but I’d be interested in hearing if you have different interpretations here.