Setting up SRG with LKB

[olzama]

How do I set up SRG with Freeling and with LKB-FOS? I know it is working for people who already have an older version of Freeling installed along with the logon tree. But I think I (a) don’t have that old version installed; and (b) ultimately, I do not want the old version, I want the new version, and I want to update the grammar rules to work with the new version.

So suppose I have a version of Freeling installed somewhere.

How do I update the SRG/LKB script file (or something else) in order for it to work?

In the logon version of the SRG, there is a file called logon.cfg which is a Freeling config file, but I do not see it mentioned anywhere, this file (in any of the LKB configuration files?..) So I am not sure what to do with it and how to use it. It uses a variable called $FREELINGSHARE, which I found defined in a file called fl_initialize which lives in my /bin directory:

##--- Get FreeLing installation from the location of this script
FREELING=$(dirname $(cd $(dirname $0) && echo $PWD))

##--- define FREELINGSHARE (if not already defined), which is used in default config files
if [ "x$FREELINGSHARE" = "x" ]; then
  export FREELINGSHARE=$FREELING/share/freeling
fi

#--- set paths to libraries required to execute FreeLing
OS=`uname -s`
if [ $OS == "Linux" ]; then 
   export LD_LIBRARY_PATH=$FREELING/lib:$LD_LIBRARY_PATH
else
   # it is a Mac
   export DYLD_LIBRARY_PATH=$FREELING/lib:$DYLD_LIBRARY_PATH
fi

#--- Pass all parameters to the executable, fixing the config file path if necessary
i=1;  eval arg=\${$i}
param=""
while [[ $i -le $# ]]; do

   param="$param $arg"

   if [ "$arg" == "-f" ]; then
     let i=$i+1; eval arg=\${$i}

     if [ -e $arg ]; then
       param="$param $arg"
     else 
       param="$param $FREELING/share/freeling/config/$arg"
     fi
   fi

   let i=$i+1;  eval arg=\${$i}
done

This script, in turn, is run by the analyze script, which is the name of the command that I have to execute in order to run freeling from command line:

## Execute FreeLing environment initialization
. $(cd $(dirname $0) && echo $PWD)/fl_initialize $@

If I run Freeling from command line, by calling analyze, it works. But if I load even the logon distribution of the SRG into LKB, it loads but does not work:

Perhaps the magic is in the final line in the LKB/script:

(initialize-sppp)

There is a Freeling-related file called sppp.dat in the logon distribution of the SRG but I don’t know what to do with it. It does not look like an executable of any sort.

Does anyone know how this is supposed to work?

[johnca]

In short, yes.

At present, to run the SRG in the LKB (and in LKB-FOS) requires a special version of Freeling with an SPPP interface. The binary is called FreelingSPPP. It only runs in Linux. See LkbSppp · delph-in/docs Wiki · GitHub for info about SPPP (which is analogous to YY mode).

Calling the function initialize-sppp starts up FreelingSPPP, which then listens for sentences input to the LKB parser and sends morphological structures back to the parser (i.e. Freeling is running as a kind of server). It’s done this way since Freeling takes quite a while to start up.

FreelingSPPP appears to be based on Freeling 3.1. (Unfortunately, I can’t verify this, and I don’t know where the source code is). Papers about the SRG say that it disambiguates using an HMM tagger. However, it won’t contain any of the more modern statistical processing models introduced in FreeLing 4.x.

To come back to your questions, you need to arrange for initialize-sppp to run upc/bin/sppp in the LOGON distribution. (This sppp shell script then invokes the FreeLingSPPP binary with the appropriate options). To do this, find where *sppp-application* is set in the SRG lkb/globals.lsp file, and replace with something like this:

(setf *sppp-application* 
  (let ((root (getenv "LOGONROOT")))
    (format nil "exec ~a/upc/bin/sppp" root)))

Also, make sure there is a call (initialize-sppp) near the end of the script file.

You should not have to do anything to sppp.dat or login.cfg. The former seems to contain post-processing rules to fix up minor mismatches between Freeling output and SRG expections. The latter is a Freeling configuration file specifying internal parameter settings and data file names.

I suggest you try the LOGON LKB first. It works fine for me as long as I’m in a Linux that runs 32 bit applications (FreelingSPPP is a 32 bit binary). Load the SRG script file and try

(do-parse-tty "El gato corre.")
(do-parse-tty "Un lugar que no me gusta es mi antiguo instituto.")

The current binary release of LKB-FOS doesn’t quite work due to bit rot in a call to an external XML library. I’ll email you a patches file that fixes this issue.

[olzama]

Thanks a lot, @johnca !

I am afraid I am not in a possession of the sppp binary. I don’t believe it is in the logon repository?

[johnca]

In my local LOGON repository, I’ve got the files upc/bin/sppp (a unix shell script) and upc/bin/freelingSPPP (a 32-bit Linux binary). These are the two files that get executed - the former executes the latter. There are also a few data files that get loaded, which are in upc/lib and upc/share/freeling.

I’ve just verified that they’re still all there in the LOGON svn tree at http://svn.emmtee.net/trunk/

[olzama]

Thanks, @johnca ! Indeed, I have the binary under upc/bin.

I have this in globals.lsp:

(setf *sppp-application* 
  (let ((root (getenv "LOGONROOT")))
    (format nil "exec ~a/upc/bin/sppp" root)))

But, attempting to parse el perro duerme., I get the error for which I posted the screenshot above.

I have Ubuntu VERSION=“20.04.1 LTS (Focal Fossa)”.

Looks like perhaps it cannot run 32-bit applications.

I am wondering what the best course of action is then. Who produced that binary, was it Lluis Padro? I suppose I could ask him about a possibility to produce an update?..

[johnca]

Yes, @olzama , that error message is consistent with trying to run a 32-bit application in a 64-bit-only Linux.

You could certainly try to make a new version of freelingSPPP. The main source code file is in upc/src/freelingSPPP.cc . The README file in that directory says that it requires an installation of FreeLing-3.0. I suggest you install that and then follow the instructions to create a 64-bit binary.

If that works, then you could try upgrading to the latest version of FreeLing and making a new binary. However, it’s possible that relevant parts of the FreeLing API have changed, and in that case freelingSPPP.cc would have to be modified. Lluis Padro might be able to advise.

[olzama]

To use the old, logon version of the SRG which relies on Freeling 3.0, apparently all that’s needed is to follow these steps:

How to Run 32-bit App on 64-bit Linux - Fedingo

…in case you have a 64-bit ubuntu, like I did. Then everything works out of the box if you use the logon tree.

Coming up with an updated version is more tricky as the API has indeed changed, and so that idea would require someone dedicating resources to it (in the form of a C++ programmer). I will have to shelve this idea for the time being.