@arademaker, my mistake, it should have been openai.com (fixed above). I’ve made much more progress on testing it last time and can give you what I’ve got so far.
Easiest way to get started is to set up an account on openai.com and use the “Playground”. It is trivial (5 lines of code?) to get it working in your language of choice once it is doing nearly what you want, they have good docs for that.
My scenario is getting GPT3 to transform what I call the “computereze” language that people throw at my game (mostly just missing articles and using verb/object syntax like “take lamp”) into valid English that the ERG will parse well.
So far, I’ve used what OpenAI calls “text completion” to do this, which basically means literally writing down instructions, followed by examples, and then giving it what you want to be transformed and hoping it will follow the pattern.
I have to post filter what it gives me to detect when it goes off the rails. So far it’s just two rules:
- if it has more than one line it is bogus
- if it isn’t surrounded by quotes it is bogus
And it gives very few bogus answers with what I’m using now. Note that you also should run its free “content” filter over its results in case it goes really crazy and sends something you will regret back. The filter will flag rasist, sexist, etc etc stuff. Mostly. No guarantees. It is an experiment for me but it does feel scary to use this in production…
To try out what I’ve done, go to the playground, and use the following settings:
Model: text-davinci-002 (the richest model, I've had mixed success so far on others)
Temperature: 0 (we want no creativity or risks, just a consistent answer)
Leave the rest of the settings at their defaults
Below is the exact text I have been using for my purposes and has worked very well testing against over 1200 phrases (both ones that shouldn’t be corrected and ones that should). You literally paste every single line of it into the playground window, and then fix the last line to be the text you want corrected. So turn the last line from:
"<the text you want to test>" should be
into (for example):
"who was governor of minnesota when ankahee was released?" should be
And hit submit. The playground will “complete” the phrase with the correction.
As always, this stuff is an art, and I’ve noticed that my model no longer adds the “the” into your text, even though it works well for my purposes. I am not even really a beginner using these systems, but here’s what I did to fix the model when I hit a case like this: Add the phrase that didn’t work into the training set and try again with more data from my “treebank”. I kept doing that until I started getting consistent results that I wanted.
Here is my raw completion text (and this shows what it took to get it working like I wanted (so far)). All the phrases where the original and correction are basically the same are places where it screwed up its suggestion and I had to add the phrase in to get it right:
Turn short phrases into full English sentences but don't remove any important words. For example:
"Open door" should be "open the door"
"eat apple" should be "eat the apple"
"put apple in barrel" should be "put the apple in the barrel"
"pacifier in bed" should be "put the pacifier in the bed"
"get safe" should be "get the safe"
"give buttercup" should be "give the buttercup"
"drop backpack" should be "drop the backpack"
"put boot table" should be "put the boot on the table"
"where the diaper bag" should be "where is the diaper bag"
"frog green?" should be "The frog is green?"
"a diamond is blue" should be "A diamond is blue"
"the pen is in the diamond cave" should be "The pen is in the diamond cave"
"there is a pen" should be "There is a pen"
"a bottom is on the slug" should be "A bottom is on the slug"
"describe the rocks" should be "Describe the rocks"
"there is blue paint" should be "There is blue paint"
"blue paint is on the table" should be "Blue paint is on the table"
"a roof is wet" should be "a roof is wet"
"go home" should be "go home"
"restart" should be "restart"
"help" should be "help"
"is a book in the entrance?" should be "Is a book in the entrance?"
"put the diamond in Plage" should be "put the diamond in Plage"
"get the rock on the floor" should be "get the rock on the floor"
"put the crystal on the table where the safe is" should be "put the crystal on the table where the safe is"
"where is the diamond at?" should be "Where is the diamond at?"
"are you still in a cave?" should be "Are you still in a cave?"
"get a hand" should be "get a hand"
"read page 1" should be "read page 1"
"read page 2" should be "read page 2"
"turn page 1" should be "Turn page 1"
"look around" should be "look around"
"paint is on the table" should be "paint is on the table"
"go to a cave" should be "go to a cave"
"is a rock in the cave?" should be "Is a rock in the cave?"
"is a girl in the doorway?" should be "Is a girl in the doorway?"
"what is the keyhole on?" should be "what is the keyhole on?"
"get Plage." should be "get Plage"
"go through the safe" should be "go through the safe"
"leave cave" should be "leave the cave"
"there is a front on a safe" should be "there is a front on a safe"
"drop a rock" should be "drop a rock"
"go into the 1st cave" should be "go into the 1st cave"
"where is a living room" should be "where is a living room"
"where is my grand children's house" should be "where is my grandchildren's house"
"<the text you want to test>" should be