Updating old SRG [incr tsdb()] profiles

[incr tsdb()]
#1

I have some old SRG [incr tsdb()] profiles, and when I load them and try to use the logon version of the SRG with them, I get this error message when trying to click on a sentence to parse it: “out-of-date profile `mrs’” (or any other profile name; it’s not just the MRS one). Can I do something about this? The grammar generally works, i.e. if I enter the same sentence in the parse window, it will be parsed.

Screen Shot 2022-10-03 at 12.22.00 PM

The profiles look normal to me, but I don’t know; what should I check about them?.. Or what further information I could provide here for someone to give me advice?

Reparsing and updating a treebank keeping previous decisions
#2

The [incr tsdb()] database schema has changed over time, and I think this message means that the profiles are using an older version of the schema. I wouldn’t be surprised if there were a conversion tool that can update profiles in pydelpin, but I wouldn’t know how to invoke it if so.

1 Like
#3

You can see what has changed in the schema by comparing the `relations’ file in that old profile with the one that appears when you create a new profile. Sometimes the number of fields in one of the tables has been increased by one or two, and it is possible to use a simple script to add those empty fields to each line of the relevant table in your old profile. (Of course it is safer to reparse that profile and update any annotations, but that can be more work.)

1 Like
#4

@Dan’s advice to check the relations file before refreshing the profile is good, in case there are any important changes requiring manual steps. If that’s not the case, see below…

Correct. You just need to use delphin mkprof with the new relations file. You can either make a new profile or overwrite the old one with the following commands:

  • delphin mkprof new-profile --relations new-relations --source old-profile
  • delphin mkprof old-profile --relations new-relations --refresh

Example

Imagine a relations file with some changes (below, one field is removed and one added to the item file, and the item-set file is removed entirely):

$ diff tmp/relations new-relations 
7d6
<   i-category :string
15a15
>   i-foo :string
60,64d59
< 
< item-set:
<   i-id :integer :key :partial
<   s-id :integer :key
<   polarity :integer

Using the --source option (without --refresh) will keep the original:

$ delphin mkprof new-tmp --relations new-relations --source tmp/
    9659 bytes	relations
     111 bytes	item
       0 bytes	analysis
       0 bytes	phenomenon
       0 bytes	parameter
       0 bytes	set
       0 bytes	item-phenomenon
       0 bytes	run
       0 bytes	parse
       0 bytes	result
       0 bytes	rule
       0 bytes	output
       0 bytes	edge
       0 bytes	tree
       0 bytes	decision
       0 bytes	preference
       0 bytes	update
       0 bytes	fold
       0 bytes	score

Note the item-set file was not created. Also see that the field changes are effected in the new profile:

$ delphin select i-category new-tmp/
undefined column: i-category
$ delphin select i-foo new-tmp/ | wc -l
3

Other Notes

  • [incr tsdb()] does not really work with arbitrary changes as described above. It has a hard-coded notion of what a relations file contains. It has some support to import from old profiles but won’t write to profiles with relations files not matching what it currently has. So when you refresh you probably want to use the official relations file (I think I use http://svn.delph-in.net/trunk/lingo/lkb/src/tsdb/skeletons/english/Relations for this).
  • You can add --gzip to the command to compress non-empty files. The original files may have been compressed, and running the command above without --gzip will decompress them.
1 Like
#5

I appreciate all the responses, thanks, especially @Dan and @goodmami !

I am still confused though, because I think I am still missing the principle understanding of how this is supposed to work.

In principle, is the same (up-to-date) Relations file supposed to work for any test suite?.. I guess not, I guess there is something unique to each one, and for that reason, it should be automatically created each time, using some robust tools?.. I thought, however, that it was the former, but copying a Relations file from a profile that does work currently leads to error “arity mismatch for item (X)”.

@goodmami I appreciate the detailed instructions. I have not tried them yet, but only because I want to first better understand what is it that I would be doing and why :).

#6

As I understand it, the Relations file lays out the structure of the database — gives an explanation of the fields in each of the other files. When the structure is updated, not only does the Relations file change, but one or more of the other files ends up with more/different/fewer fields. So, the scripts take existing profiles (matching an old Relations file) and make the required adjustments.

2 Likes
#7

@ebender is exactly right. The relations file is the database schema. Just as you cannot simply swap out the schema in a SQLite (MySQL, Postgresql, etc.) databases and have it work, you cannot just swap out the relations file, but rather the database needs to be rebuilt following the new relations file. This is what the delphin mkprof command shown above does.

You may also find these documentation pages informative:

1 Like
#8

Thanks, @goodmami ! I’ve read that documentation but it presupposes some knowledge about databases, e.g. some intuition about what a “database schema” is (which is not that difficult to obtain, I guess, but one still needs to know what to look for first…)

I’ll try to figure all this out now and will report back :).

#9

OK, I suppose there are important changes that need to be done manually, but I am still not sure what I should be doing.

Trying to run delphin mkprof I get the error:

number of columns (12) != number of fields (15)

When I diff the relations file with the one from a profile which works, I see the following diffs:

69a70
>   protocol :integer                     # [incr tsdb()] protocol version
108c109
<   aedges :integer                       # active items in chart (PAGE)
---
>   aedges :integer                       # active items in chart
112,113c113,119
<   unifications :integer                 # number of (node) unifications
<   copies :integer                       # number of (node) copy operations
---
>   tedges :integer                       # type-0 entries in (visible) forest
>   eedges :integer                       # type-1 entries in (visible) forest
>   ledges :integer                       # type-2 entries in (visible) forest
>   sedges :integer                       # type-3 entries in (visible) forest
>   redges :integer                       # type-4 entries in (visible) forest
>   unifications :integer                 # number of (top-level) unifications
>   copies :integer                       # number of (top-level) copy operations
152c158
<   o-application :string                 # applicable appliaction(s)
---
>   o-application :string                 # applicable application(s)
168,170c174,176
<   e-name :string                        # edge label (as in `derivation')
<   e-status :integer                     # 0 : passive; 1 : active
<   e-result :integer                     # 0 : nope; 1 : yup, result
---
>   e-label :string                       # edge label (as in ‘derivation’)
>   e-type :integer                       # item type for edge (0 -- 4)
>   e-status :integer                     # bit-coded item flags
173,174c179,181
<   e-daughters :string                   # (Common-Lisp) list of daughters
<   e-parents :string                     # (Common-Lisp) list of parents
---
>   e-score :string                       # platform-specific figure of merit
>   e-daughters :string                   # space-separated list of daughters
>   e-parents :string                     # space-separated list of parents
238a246,249
>   score-start :integer
>   score-end :integer
>   score-id :integer
>   learner :string

Is the error and/or the diffs informative for someone here? Thanks a lot!

#10

@olzama that error occurs when the relations file inside a profile doesn’t match the data that’s there. It seems like maybe a newer relations file was copied into the profile without updating it. You should try to put the old relations file back and try again.

1 Like
#11

Alright! In the end, I got to the result I wanted (being able to load an old profile in [incr tsdb()] and running the grammar over the items that it contains) as follows:

Thanks, both!

1 Like