Can't generate synthetic sentences #157

Limtle · 2020-04-16T08:16:46Z

I follow [README]( the https://github.com/stanford-oval/genie-toolkit/blob/master/README.md.)
The environment: linux . Nodejs: 10.4.1
The command I used
yarn global add genie-toolkit
pip install genienlp
cd Our_Almond
git clone https://github.com/stanford-oval/genie-toolkit.git
cd genie-toolkit
npm install
genie download-dataset -o dataset.tt
genie download-snapshot -o thingpedia.tt --entities entities.json

genie generate --locale en --template /home/user/Our_Almond/genie-toolkit/languages/thingtalk/en/thingtalk.genie --thingpedia thingpedia.tt --entities entities.json --dataset dataset.tt -o synthesized.tsv

The error I faced:

stats: size(charts[0][out_param_Array__String]) = 4
stats: size(charts[0][both_prefix]) = 2
Error expanding rule NT[preposition_filter] -> here
{ AssertionError [ERR_ASSERTION]: The expression evaluated to a falsy value:

  assert(pname instanceof Ast.Value.VarRef)

    at Object.makeFilter (/home/user/Our_Almond/genie-toolkit/languages/thingtalk/utils.js:40:5)
    at Object.makeFilter (/home/user/Our_Almond/genie-toolkit/languages/thingtalk/ast_manip.js:160:18)
    at $grammar.addRule.$runtime.simpleCombine (eval at parse (/home/user/.config/yarn/global/node_modules/genie-toolkit/lib/genie-compiler/index.js:122:25), <anonymous>:428:85)
    at Function.combine (/home/user/.config/yarn/global/node_modules/genie-toolkit/lib/sentence-generator/runtime.js:209:29)
    at /home/user/.config/yarn/global/node_modules/genie-toolkit/lib/sentence-generator/runtime.js:255:35
    at expandRule (/home/user/.config/yarn/global/node_modules/genie-toolkit/lib/sentence-generator/runtime.js:985:18)
    at Grammar.generate (/home/user/.config/yarn/global/node_modules/genie-toolkit/lib/sentence-generator/runtime.js:806:25)
    at SentenceGenerator.generate (/home/user/.config/yarn/global/node_modules/genie-toolkit/lib/sentence-generator/index.js:47:23)
    at _initialization.then (/home/user/.config/yarn/global/node_modules/genie-toolkit/lib/sentence-generator/index.js:183:29)
  generatedMessage: true,
  name: 'AssertionError [ERR_ASSERTION]',
  code: 'ERR_ASSERTION',
  actual: false,
  expected: true,
  operator: '==' }
/home/user/.config/yarn/global/node_modules/genie-toolkit/tool/genie.js:13
process.on('unhandledRejection', (up) => { throw up; });
                                           ^

AssertionError [ERR_ASSERTION]: The expression evaluated to a falsy value:

  assert(pname instanceof Ast.Value.VarRef)

    at Object.makeFilter (/home/user/Our_Almond/genie-toolkit/languages/thingtalk/utils.js:40:5)
    at Object.makeFilter (/home/user/Our_Almond/genie-toolkit/languages/thingtalk/ast_manip.js:160:18)
    at $grammar.addRule.$runtime.simpleCombine (eval at parse (/home/user/.config/yarn/global/node_modules/genie-toolkit/lib/genie-compiler/index.js:122:25), <anonymous>:428:85)
    at Function.combine (/home/user/.config/yarn/global/node_modules/genie-toolkit/lib/sentence-generator/runtime.js:209:29)
    at /home/user/.config/yarn/global/node_modules/genie-toolkit/lib/sentence-generator/runtime.js:255:35
    at expandRule (/home/user/.config/yarn/global/node_modules/genie-toolkit/lib/sentence-generator/runtime.js:985:18)
    at Grammar.generate (/home/user/.config/yarn/global/node_modules/genie-toolkit/lib/sentence-generator/runtime.js:806:25)
    at SentenceGenerator.generate (/home/user/.config/yarn/global/node_modules/genie-toolkit/lib/sentence-generator/index.js:47:23)
    at _initialization.then (/home/user/.config/yarn/global/node_modules/genie-toolkit/lib/sentence-generator/index.js:183:29)
Emitted 'error' event at:
    at _initialization.then.catch (/home/user/.config/yarn/global/node_modules/genie-toolkit/lib/sentence-generator/index.js:188:18)

I would appreciate it if you could take a moment to explain something to me

The text was updated successfully, but these errors were encountered:

gcampax · 2020-04-16T19:16:39Z

Hi Limtle. You should make sure that only one version of Genie is in use. If you mix yarn global and git clone, you might end up with two versions of Genie, installed in different directories, which can lead to these kinds of bugs.

The best way to avoid it is, after you run yarn global add genie-toolkit, to go into the genie-toolkit clone and do yarn link, which will replace the global package with the git clone.

Let me know if it fixes this issue, and we'll update our docs accordingly!

Limtle · 2020-04-17T02:17:47Z

The command I used

yarn global add genie-toolkit
pip install genienlp
cd Our_Almond
git clone https://github.com/stanford-oval/genie-toolkit.git
cd genie-toolkit
yarn link

The warning I faced:

warning There's already a package called "genie-toolkit" registered. This command has had no effect. If this command was run in another folder with the same name, the other folder is still linked. Please run yarn unlink in the other folder if you want to register this folder.
Done in 0.04s.

So after this warning
The command I used

cd /home/user/.config/yarn/global/node_modules/genie-toolkit
yarn unlink
cd /home/user/Our_Almond/genie-toolkit
yarn link
cd ..
genie generate --locale en --template /home/user/Our_Almond/genie-toolkit/languages/thingtalk/en/thingtalk.genie --thingpedia thingpedia.tt --entities entities.json --dataset dataset.tt -o synthesized.tsv

The error I faced:

AssertionError [ERR_ASSERTION]: The expression evaluated to a falsy value:

  assert(pname instanceof Ast.Value.VarRef)

    at Object.makeFilter (/home/user/Our_Almond/genie-toolkit/languages/thingtalk/utils.js:40:5)
    at Object.makeFilter (/home/user/Our_Almond/genie-toolkit/languages/thingtalk/ast_manip.js:160:18)
    at eval (eval at parse (/home/user/.config/yarn/global/node_modules/genie-toolkit/lib/genie-compiler/index.js:122:25), <anonymous>:426:85)
    at Function.combine (/home/user/.config/yarn/global/node_modules/genie-toolkit/lib/sentence-generator/runtime.js:209:29)
    at /home/user/.config/yarn/global/node_modules/genie-toolkit/lib/sentence-generator/runtime.js:255:35
    at expandRule (/home/user/.config/yarn/global/node_modules/genie-toolkit/lib/sentence-generator/runtime.js:985:18)
    at Grammar.generate (/home/user/.config/yarn/global/node_modules/genie-toolkit/lib/sentence-generator/runtime.js:806:25)
    at SentenceGenerator.generate (/home/user/.config/yarn/global/node_modules/genie-toolkit/lib/sentence-generator/index.js:47:23)
    at /home/user/.config/yarn/global/node_modules/genie-toolkit/lib/sentence-generator/index.js:183:29 {
  generatedMessage: true,
  code: 'ERR_ASSERTION',
  actual: false,
  expected: true,
  operator: '=='
}
/home/user/.config/yarn/global/node_modules/genie-toolkit/tool/genie.js:13
process.on('unhandledRejection', (up) => { throw up; });
                                           ^

AssertionError [ERR_ASSERTION]: The expression evaluated to a falsy value:

  assert(pname instanceof Ast.Value.VarRef)

    at Object.makeFilter (/home/user/Our_Almond/genie-toolkit/languages/thingtalk/utils.js:40:5)
    at Object.makeFilter (/home/user/Our_Almond/genie-toolkit/languages/thingtalk/ast_manip.js:160:18)
    at eval (eval at parse (/home/user/.config/yarn/global/node_modules/genie-toolkit/lib/genie-compiler/index.js:122:25), <anonymous>:426:85)
    at Function.combine (/home/user/.config/yarn/global/node_modules/genie-toolkit/lib/sentence-generator/runtime.js:209:29)
    at /home/user/.config/yarn/global/node_modules/genie-toolkit/lib/sentence-generator/runtime.js:255:35
    at expandRule (/home/user/.config/yarn/global/node_modules/genie-toolkit/lib/sentence-generator/runtime.js:985:18)
    at Grammar.generate (/home/user/.config/yarn/global/node_modules/genie-toolkit/lib/sentence-generator/runtime.js:806:25)
    at SentenceGenerator.generate (/home/user/.config/yarn/global/node_modules/genie-toolkit/lib/sentence-generator/index.js:47:23)
    at /home/user/.config/yarn/global/node_modules/genie-toolkit/lib/sentence-generator/index.js:183:29
Emitted 'error' event on BasicSentenceGenerator instance at:
    at /home/user/.config/yarn/global/node_modules/genie-toolkit/lib/sentence-generator/index.js:188:18 {
  generatedMessage: true,
  code: 'ERR_ASSERTION',
  actual: false,
  expected: true,
  operator: '=='
}

I would appreciate it if you could take a moment to explain something to me

Kljon · 2020-04-20T03:35:38Z

I have the same problem.
After following the https://github.com/stanford-oval/genie-toolkit/blob/master/INSTALL.md

genie download-dataset -o dataset.tt
genie download-snapshot -o thingpedia.tt --entities entities.json
genie generate --locale en-US --template ~/.yarn/bin/global/node_modules/genie-toolkit/languages/thingtalk/en/thingtalk.genie --thingpedia thingpedia.tt --entities entities.json --dataset dataset.tt -o synthesized.tsv

Error expanding rule NT[where_question] -> where is NT[complete_table]
ReferenceError: type is not defined
    at $grammar.addRule.$runtime.simpleCombine (eval at parse (/home/Kljon/.config/yarn/global/node_modules/genie-toolkit/lib/genie-compiler/index.js:122:25), <anonymous>:97:173)
    at Function.combine (/home/Kljon/.config/yarn/global/node_modules/genie-toolkit/lib/sentence-generator/runtime.js:241:21)
    at /home/Kljon/.config/yarn/global/node_modules/genie-toolkit/lib/sentence-generator/runtime.js:255:35
    at recursiveHelper (/home/Kljon/.config/yarn/global/node_modules/genie-toolkit/lib/sentence-generator/runtime.js:1073:29)
    at recursiveHelper (/home/Kljon/.config/yarn/global/node_modules/genie-toolkit/lib/sentence-generator/runtime.js:1101:25)
    at recursiveHelper (/home/Kljon/.config/yarn/global/node_modules/genie-toolkit/lib/sentence-generator/runtime.js:1129:17)
    at expandRule (/home/Kljon/.config/yarn/global/node_modules/genie-toolkit/lib/sentence-generator/runtime.js:1131:11)
    at Grammar.generate (/home/Kljon/.config/yarn/global/node_modules/genie-toolkit/lib/sentence-generator/runtime.js:806:25)
    at SentenceGenerator.generate (/home/Kljon/.config/yarn/global/node_modules/genie-toolkit/lib/sentence-generator/index.js:47:23)
    at _initialization.then (/home/Kljon/.config/yarn/global/node_modules/genie-toolkit/lib/sentence-generator/index.js:183:29)
/home/Kljon/.config/yarn/global/node_modules/genie-toolkit/tool/genie.js:13
process.on('unhandledRejection', (up) => { throw up; });
                                           ^

ReferenceError: type is not defined
    at $grammar.addRule.$runtime.simpleCombine (eval at parse (/home/Kljon/.config/yarn/global/node_modules/genie-toolkit/lib/genie-compiler/index.js:122:25), <anonymous>:97:173)
    at Function.combine (/home/Kljon/.config/yarn/global/node_modules/genie-toolkit/lib/sentence-generator/runtime.js:241:21)
    at /home/Kljon/.config/yarn/global/node_modules/genie-toolkit/lib/sentence-generator/runtime.js:255:35
    at recursiveHelper (/home/Kljon/.config/yarn/global/node_modules/genie-toolkit/lib/sentence-generator/runtime.js:1073:29)
    at recursiveHelper (/home/Kljon/.config/yarn/global/node_modules/genie-toolkit/lib/sentence-generator/runtime.js:1101:25)
    at recursiveHelper (/home/Kljon/.config/yarn/global/node_modules/genie-toolkit/lib/sentence-generator/runtime.js:1129:17)
    at expandRule (/home/Kljon/.config/yarn/global/node_modules/genie-toolkit/lib/sentence-generator/runtime.js:1131:11)
    at Grammar.generate (/home/Kljon/.config/yarn/global/node_modules/genie-toolkit/lib/sentence-generator/runtime.js:806:25)
    at SentenceGenerator.generate (/home/Kljon/.config/yarn/global/node_modules/genie-toolkit/lib/sentence-generator/index.js:47:23)
    at _initialization.then (/home/Kljon/.config/yarn/global/node_modules/genie-toolkit/lib/sentence-generator/index.js:183:29)
Emitted 'error' event at:
    at _initialization.then.catch (/home/Kljon/.config/yarn/global/node_modules/genie-toolkit/lib/sentence-generator/index.js:188:18)

gcampax · 2020-04-20T03:40:17Z

Those seem to be two separate issues.

@Limtle, the warning indicates that yarn link did not have any effect, hence that's why you're still seeing the original problem. Can try yarn global remove genie-toolkit and then yarn link again? You should not see the warning from yarn link, otherwise it won't help.

@Kljon it looks like you uncovered an actual, separate bugs in the templates. I'll open a PR.

Kljon · 2020-04-20T09:12:12Z

Thanks for your Help.

node --max_old_space_size=80000 ~/bin/genie generate --locale en --template /home/Kljon/.config/yarn/global/node_modules/genie-toolkit/languages/thingtalk/en/thingtalk.genie --thingpedia thingpedia.tt --entities entities.json --dataset dataset.tt -o /work/Kljon/synthesized.tsv

But following error I faced.

--- DEPTH 4
terminate called after throwing an instance of 'std::bad_alloc'
  what():  std::bad_alloc

Limtle · 2020-04-21T06:26:47Z

Thanks. After run yarn global remove genie-toolkit . I did not see the warning from yarn link now, but yarn link doesn't link binaries globally . Maybe have this problem .

I follow @Kljon use ~/.yarn/bin/global/node_modules/genie-toolkit/languages/thingtalk/en/thingtalk.genie as template. It is working. But Got std::bad_alloc too

gcampax · 2020-04-21T07:06:16Z

@Limtle, as indicated in the yarn issue you linked, yarn link does link binaries globally, but only in recent versions of yarn.

As for the memory error, @Kljon, you're passing an invalid (too large) value to --max_old_space_size. This essentially disables GC and causes your process to run out memory.
You should pass something like (max available memory - a couple GBs free for the OS and anything else you're running on)

Kljon · 2020-04-22T02:59:41Z

@gcampax Thank you for your help. it works!
I have another question for Step 3. Paraphrasing. Can I use synthesized.tsv directly genie mturk-make-paraphrase-hits -o paraphrasing-hits.csv < synthesized.tsv without Step 2. Choose The Sentences To Paraphrases.
Because I have some question in Choose The Sentences To Paraphrases. What is the goal of Paraphrases.
Why we need to choose the Sentences to paraphrases instead of using all data to paraphrases.

gcampax · 2020-04-22T03:10:00Z

Paraphrasing is a way to make the sentences in the dataset more natural, and improve the quality of the semantic parser trained from those sentences. You don't paraphrase the whole dataset just as a matter of cost: paraphrasing requires human effort (through a crowdsourcing platform like Amazon MTurk), and a synthetic dataset can easily have one million or more sentences so it would be too expensive to paraphrase in its entirety.
The sampling step also changes the format of the synthesized dataset slightly: it replaces constants which are masked in the synthesized datasets (numbers, dates, times, etc.) with concrete values. This is necessary so the sentences are readable to a human paraphraser. Hence, sampling is not optional.

Kljon · 2020-04-22T03:44:00Z

Thank you for your help and patience.
This is my command

genie sample synthesized.tsv --constants /home/Kljon/.config/yarn/global/node_modules/genie-toolkit/data/en-US/constants.tsv --sampling-strategy bySignature --sampling-control /home/Kljon/.config/yarn/global/node_modules/genie-toolkit/data/easy-hard-functions.tsv -o mturk-input.tsv --thingpedia thingpedia.tt

error I faced

which are the get my Google contacts ?
/home/Kljon/.config/yarn/global/node_modules/genie-toolkit/tool/genie.js:13
process.on('unhandledRejection', (up) => { throw up; });
                                          ^

Error: Missing entity Google (present in the sentence, not in the code)
   at SentenceProcessor.process (/home/Kljon/.config/yarn/global/node_modules/genie-toolkit/lib/sampler.js:246:27)
   at SentenceSampler._run (/home/Kljon/.config/yarn/global/node_modules/genie-toolkit/lib/sampler.js:485:40)
   at SentenceSampler._transform (/home/Kljon/.config/yarn/global/node_modules/genie-toolkit/lib/sampler.js:493:14)
   at SentenceSampler.Transform._read (_stream_transform.js:190:10)
   at SentenceSampler.Transform._write (_stream_transform.js:178:12)
   at doWrite (_stream_writable.js:410:12)
   at clearBuffer (_stream_writable.js:540:7)
   at onwrite (_stream_writable.js:465:7)
   at SentenceSampler.afterTransform (_stream_transform.js:94:3)
   at _run.then (/home/Kljon/.config/yarn/global/node_modules/genie-toolkit/lib/sampler.js:493:37)
Emitted 'error' event at:
   at SentenceSampler.onerror (_stream_readable.js:690:12)
   at SentenceSampler.emit (events.js:182:13)
   at onwriteError (_stream_writable.js:431:12)
   at onwrite (_stream_writable.js:456:5)
   at SentenceSampler.afterTransform (_stream_transform.js:94:3)
   at process._tickCallback (internal/process/next_tick.js:68:7)

Kljon · 2020-05-04T08:46:42Z

Hi @gcampax . I found the genie-toolkit has updated. So I reinstall genie-toolkit but get the same error.
I found variable _assignedEntities in ./genie-toolkit/lib/sampler.js is {}.
And Do you have an example for --sampling-strategy.
Thanks.

gcampax · 2020-07-06T04:08:43Z

We have just merged new documentation, and new starter code that should help with setting up Genie and getting it to generate sentences. If you're still interested, would you be able to try that?

gcampax · 2021-04-04T00:26:33Z

Closing this. Please reopen if you still experience the issue with the latest version of genie-toolkit.

gcampax mentioned this issue Apr 20, 2020

Fix where questions if the "projection_with_filter" flag is not set #159

Merged

gcampax closed this as completed Apr 4, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Can't generate synthetic sentences #157

Can't generate synthetic sentences #157

Limtle commented Apr 16, 2020

gcampax commented Apr 16, 2020

Limtle commented Apr 17, 2020 •

edited

Kljon commented Apr 20, 2020

gcampax commented Apr 20, 2020

Kljon commented Apr 20, 2020 •

edited

Limtle commented Apr 21, 2020

gcampax commented Apr 21, 2020

Kljon commented Apr 22, 2020 •

edited

gcampax commented Apr 22, 2020

Kljon commented Apr 22, 2020

Kljon commented May 4, 2020 •

edited

gcampax commented Jul 6, 2020

gcampax commented Apr 4, 2021

Can't generate synthetic sentences #157

Can't generate synthetic sentences #157

Comments

Limtle commented Apr 16, 2020

gcampax commented Apr 16, 2020

Limtle commented Apr 17, 2020 • edited

Kljon commented Apr 20, 2020

gcampax commented Apr 20, 2020

Kljon commented Apr 20, 2020 • edited

Limtle commented Apr 21, 2020

gcampax commented Apr 21, 2020

Kljon commented Apr 22, 2020 • edited

gcampax commented Apr 22, 2020

Kljon commented Apr 22, 2020

Kljon commented May 4, 2020 • edited

gcampax commented Jul 6, 2020

gcampax commented Apr 4, 2021

Limtle commented Apr 17, 2020 •

edited

Kljon commented Apr 20, 2020 •

edited

Kljon commented Apr 22, 2020 •

edited

Kljon commented May 4, 2020 •

edited