Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

gtools error occurred while loading _gtools_internal.ado (could not load plugin) #60

Open
AnneShi opened this issue May 3, 2019 · 71 comments
Open

gtools error occurred while loading _gtools_internal.ado (could not load plugin) #60

AnneShi opened this issue May 3, 2019 · 71 comments
Assignees
@mcaceresb
Labels
bug

Comments

@AnneShi
Copy link

AnneShi commented May 3, 2019

I have installed gtool for a while and the commands had worked well until 2 days ago, when our IT updated windows. The commands from gtool package no longer work. While running gcollapse, the program was interrupted with error message as:

(error occurred while loading _gtools_internal.ado)
r(9998);

I am not able to find out what r(9998) means.
I appreciate your help.
@mcaceresb
Copy link
Owner

Please give the output of

gtools

@AnneShi
Copy link
Author

AnneShi commented May 3, 2019

.
. use S:\Forum\Ashi\Data\2017household\Stata\IPUMS\ps02_armenia, clear
(Armenia person-household file with religion recoded)

. gcollapse hhsize [pw = perwt]
(error occurred while loading _gtools_internal.ado)
r(9998);

end of do-file

r(9998);

@mcaceresb
Copy link
Owner

Sorry, I meant the output of running

which gtools
gtools

@AnneShi
Copy link
Author

AnneShi commented May 3, 2019

. which gtools
c:\ado\plus\g\gtools.ado
*! version 1.5.3 04Apr2019 Mauricio Caceres Bravo, mauricio.caceres.bravo@gmail.com
*! Program for managing the gtools package installation

. gtools
Nothing to do. See help gtools or gtools, examples for usage. Version info:
c:\ado\plus\g\gtools.ado
*! version 1.5.3 04Apr2019 Mauricio Caceres Bravo, mauricio.caceres.bravo@gmail.com
*! Program for managing the gtools package installation
(error occurred while loading _gtools_internal.ado)
(warning: gtools_plugin internal check failed)

@mcaceresb mcaceresb self-assigned this May 3, 2019
@mcaceresb mcaceresb added the bug label May 3, 2019
@mcaceresb
Copy link
Owner

Can you tell me your OS and Stata versions?

@AnneShi
Copy link
Author

AnneShi commented May 3, 2019

Windows 8; Stata/MP 14.2 or Stata/SE 15.1

@mcaceresb
Copy link
Owner

Are you on a server or just a regular Windows 8 machine?

@AnneShi
Copy link
Author

AnneShi commented May 3, 2019

on a network

@AnneShi
Copy link
Author

AnneShi commented May 3, 2019

All commands from gtools worked till two days ago, when the IT updated the windows. I talked to our IT specialists and they are trying to figure out as well

@mcaceresb
Copy link
Owner

(Not 100% what you mean by "network").

Anyway, I wonder if it is related to this bug. It's a bit of a different situation since that was back when I still put the hashing library in a DLL (and I thought it was fixed).

Do you happen to have cygwin installed on your windows system? No worries if you don't.

@AnneShi
Copy link
Author

AnneShi commented May 3, 2019

Server, then.
Yes I read the discussions and tried the suggested solutions therein. None works.
No I do not have cygwin.

@AnneShi
Copy link
Author

AnneShi commented May 3, 2019

Out IT people said they might try to scale the windows security updates they installed a couple of days age -- to see whether those are the cause.
Are the gtools plugins security sensitive?

@mcaceresb
Copy link
Owner

I do recall somebody else reporting that gtools stopped working after a Windows Server security update.

All gtools commands use temporary files, but none require any special permissions to run. I am really not sure why a security update would break gtools. Maybe it's a versioning issue? (i.e. I compile using a set of libraries that get updated and that breaks gtools). I don't have access to Windows server, however, so it's a bit difficult to debug.

When you say "none work" for the solutions in that thread, what did you try? I was going to try and compile a very basic plugin to see if whether the issue was gtools or Stata.

@AnneShi
Copy link
Author

AnneShi commented May 3, 2019

I un-installed and reinstalled gtools.
I didn't do spooky.

@mcaceresb
Copy link
Owner

Can you download the zip file here and unzip to an empty directory, then run do test.do? It's for an unrelated issue but it will test what I want to double-check.

@AnneShi
Copy link
Author

AnneShi commented May 3, 2019

. do "C:\Users\ashi.PEW\Downloads\statatest\plugin\test.do"

. clear all

. program test1, plugin using(test1.plugin)
file test1.plugin not found
r(601);

@mcaceresb
Copy link
Owner

Sorry, you need to run it from that directory.

cd C:\Users\ashi.PEW\Downloads\statatest\plugin\
do test.do

@AnneShi
Copy link
Author

AnneShi commented May 3, 2019

. cd C:\Users\ashi.PEW\Downloads\statatest\plugin
C:\Users\ashi.PEW\Downloads\statatest\plugin

. do test

. clear all

. program test1, plugin using(test1.plugin)
r(9998);

end of do-file
r(9998);

end of do-file

r(9998);

.

@mcaceresb
Copy link
Owner

Ok, that is interesting, because that is just a plain "Hello World" basic plugin example. So I think the issue is not gtools per se, but rather however I am compiling plugins is getting blocked... Let me check if I can update my Windows 10/Cygwin versions and re-compile to see if that helps. Will get to this over the weekend. Sorry I can't be more help right now.

@AnneShi
Copy link
Author

AnneShi commented May 3, 2019

Thank you so much!
I need to finish something too. I have just reverted all gcollapse back to collapse in the codes I am running. It is painful, as I have a lot of codes using commands from the gtools package. I really like them.

mcaceresb added a commit that referenced this issue May 6, 2019
@mcaceresb
Windows version after update; maybe helps with #60
7effd31
@mcaceresb
Copy link
Owner

Try gtools, upgrade(develop), restart stata, and then try running gtools again?

@AnneShi
Copy link
Author

AnneShi commented May 6, 2019 via email

@AnneShi
Copy link
Author

AnneShi commented May 8, 2019

I tried gtools, upgrade(develop) and got this error message:

. gtools, upgrade(develop)
option upgrade() incorrectly specified
r(198);

@mcaceresb
Copy link
Owner

My bad

gtools, upgrade branch(develop)

@AnneShi
Copy link
Author

AnneShi commented May 8, 2019

The above codes to update gtools run successfully, but after I restarted the Stata and run the script using gcollapse, I got the same error message:

image

@binyamink
Copy link

Hi, first of all thanks a lot for the package. I have a similar issue as described above. In particular, I get the "Could not load plugin" error when trying to run the gtools commands. I have the same issue when running your "hello world" test. I have tried to replace gtools_windows_v2.plugin and gtools_windows_v3.plugin with the versions you've uploaded above.

Are there any more updated solutions than the ones found in this thread?
Thanks a lot in any case!

@mcaceresb
Copy link
Owner

What's your OS?

@drkrynstrng
Copy link

Yes, that works. So I will include that line in do-files. I do this for ftools as well, by including the ftools command in do-files to create the ftools Mata library. Just curious, is there a reason these are not automatically indexed? In the do-files I'm working with I change the sysdir for ado-files, so perhaps that is why? I tested it and this appears to be the case. Thanks!

@mcaceresb
Copy link
Owner

@drkrynstrng I think you only need to run mata: mata mlib index once after you install packages, but I suppose I can't hurt to run it always?

Not sure why it happens. I've never had this issue but I have had this complaint before...

@lffontes
Copy link

I'm having the same issue on a Windows 10 pro (64-bits, private machine) with Stata/SE 14.0. See some outputs below.

. gtools
Nothing to do. See help gtools or gtools, examples for usage. Version info:
c:\ado\plus\g\gtools.ado
*! version 1.6.2 25Aug2019 Mauricio Caceres Bravo, mauricio.caceres.bravo@gmail.com
*! Program for managing the gtools package installation
Could not load plugin: c:\ado\plus\g\gtools_windows_v2.plugin
(error occurred while loading _gtools_internal.ado)
(warning: gtools_plugin internal check failed)

. which gtools
c:\ado\plus\g\gtools.ado
*! version 1.6.2 25Aug2019 Mauricio Caceres Bravo, mauricio.caceres.bravo@gmail.com
*! Program for managing the gtools package installation

. which lgtools.mlib
c:\ado\plus\l\lgtools.mlib

.
. findfile lgtools.mlib
c:\ado\plus/l/lgtools.mlib

.
. mata: mata mlib query
.mlib libraries to be searched are
lmatabase;lmataado;lmatafc;lmatagsem;lmatamcmc;lmataopt;lmatapath;lmatapostest;lmatapss;lmatasem;lftools;lgtools;l

moremata;lmoremata10;lmoremata11;lmoremata14;lsynth_mata_subr

@mcaceresb
Copy link
Owner

@lffontes does the ssc version fail as well? (i.e. if you install from ssc w/o updating, do you get the same issue?)

@lffontes
Copy link

@lffontes does the ssc version fail as well? (i.e. if you install from ssc w/o updating, do you get the same issue?)

Yes.

. gtools
Nothing to do. See help gtools or gtools, examples for usage. Version info:
c:\ado\plus\g\gtools.ado
*! version 1.5.1 24Mar2019 Mauricio Caceres Bravo, mauricio.caceres.bravo@gmail.com
*! Program for managing the gtools package installation
Could not load plugin: c:\ado\plus\g\gtools_windows_v2.plugin
(error occurred while loading _gtools_internal.ado)
(warning: gtools_plugin internal check failed)

@lffontes
Copy link

lffontes commented Dec 5, 2019

Any idea on what is going on?

@mcaceresb
Copy link
Owner

Unfortunately I don't, I'm very sorry. My guess is that the version of windows I compile under (MinGW on regular Windows 10 but on a virtual machine) has some library that conflicts with Windows 10 Pro somehow.

I would hope that if the plugin were to compile on a Windows 10 pro machine it might work, but I don't have win 10 pro to test out my theory.

@ast-code
Copy link

ast-code commented Jul 9, 2020

I had the same error message. Found out that gtools works fine if I use the 64bit Stata version (tested for STATA/SE 15.1, Windows 10 pro build 17763).

@stevecicala
Copy link

stevecicala commented Dec 11, 2020 •
edited

Hi Mauricio--I'm getting a similar error, prospectively related to running it on Apple Silicon? I'm using Stata16.1 on MacOS

gtools
Nothing to do. See help gtools or gtools, examples for usage. Version info:
/Users/scicala/Library/Application Support/Stata/ado/plus/g/gtools.ado
*! version 1.7.5 18Apr2020 Mauricio Caceres Bravo, mauricio.caceres.bravo@gmail.com
*! Program for managing the gtools package installation
Could not load plugin: /Users/scicala/Library/Application Support/Stata/ado/plus/g/gtools_macosx_v3.plugin
(error occurred while loading _gtools_internal.ado)
(warning: gtools_plugin internal check failed)

@mcaceresb
Copy link
Owner

@stevecicala I can try to help you debug if you are willing to try to compile the plugin? I don't own a Mac (or Stata 16 for that matter), so I don't think I can figure it out w/o someone who has access to physical hardware.

@stevecicala
Copy link

Sure, happy to try--how should I proceed?

@mcaceresb
Copy link
Owner

@stevecicala Great.

  • You will first need to install Xcode and git.

  • From Xcode, install the command line tools (I think it's Xcode > Preferences > Downloads > Command line tools > Install).

  • Open the terminal.

  • Clone this repo, git clone https://github.com/mcaceresb/stata-gtools

  • Compile the plugin

cd stata-gtools
make clean SPI=3.0 SPIVER=v3
make all   SPI=3.0 SPIVER=v3

  • If all that goes well, let me know the output of ls -lt build/*osx*. They should be time-stamped to whenever you run the compile command.

  • If they are, you can copy them to /Users/scicala/Library/Application Support/Stata/ado/plus/g/. Restart Stata and try again.

@stevecicala
Copy link

@mcaceresb , thanks after the make all SPI=3.0 SPIVER=v3 command, I get:

rm -f build/gtools_macosx_v3.plugin
rm -f  src/plugin/lib
rm -f  src/plugin/spi
ln -sf ../../lib          src/plugin/lib
ln -sf lib/spi-3.0 src/plugin/spi
make: *** No rule to make target `lib/spookyhash/src/context.c', needed by `gtools'.  Stop.

The output of ls -lt build/*osx* is

-rw-r--r--  1 scicala  staff  616620 Dec 11 17:02 build/gtools_macosx_v2.plugin

The build directory then has a copy of gtools_macosx_v2.plugin, but no v3

Stata returns the same error after restart.

@mcaceresb
Copy link
Owner

@stevecicala Sorry I think you need to run git submodule update --init --recursive before the make portion.

@stevecicala
Copy link

Success! Thanks a million, this is a fantastic tool.

Here's the output from the make:

rm -f build/gtools_macosx_v3.plugin
rm -f  src/plugin/lib
rm -f  src/plugin/spi
ln -sf ../../lib 	  src/plugin/lib
ln -sf lib/spi-3.0 src/plugin/spi
mkdir -p ./build
gcc -Wall -O3 -bundle -DSYSTEM=APPLEMAC -o build/gtools_macosx_v3.plugin -Ilib/spookyhash/src src/plugin/gtools.c src/plugin/spi/stplugin.c lib/spookyhash/src/context.c lib/spookyhash/src/globals.c lib/spookyhash/src/spookyhash.c
In file included from src/plugin/gtools.c:48:
src/plugin/extra/greshape_fast.c:244:48: warning: variable 'z' is uninitialized when used
      here [-Wuninitialized]
                    outdbl[selx + kvars + 1] = z;
                                               ^
src/plugin/extra/greshape_fast.c:30:16: note: initialize the variable 'z' to silence this
      warning
    ST_double z;
               ^
                = 0.0
1 warning generated.
cp build/*plugin lib/plugin/

@dstellotri
Copy link

So I have the same issues as before, BUT this is an RDC server, and have no access to internet, with limited amount of things I can upload. I have tried

Nothing to do. See help gtools or gtools, examples for usage. Version info:
c:\ado\plus\g\gtools.ado
*! version 1.5.1 24Mar2019 9 Mauricio Caceres Bravo, mauricio.caceres.bravo@gmail.com
*! Program for managing the gtools package installation
(error occurred while loading _gtools_internal.ado)
(warning: gtools_plugin internal check failed)

I tried some of the other stuff here, but don't know how to fix without downloading anything (no access to internet)

@mcaceresb
Copy link
Owner

@dstellotri What OS is the server? What Stata version? You can always try downloading the build folder, uploading from your local machine, and then installing from the manually downloaded folder.

@dstellotri
Copy link

@dstellotri What OS is the server? What Stata version? You can always try downloading the build folder, uploading from your local machine, and then installing from the manually downloaded folder.

Windows (8 I believe), I cant access to actual version because of no-admin rights. Stata I can have access to 15 and 16. I did try that (downloading the build, and manually installing it, calling it from the proper ado folder).

@mcaceresb
Copy link
Owner

@dstellotri You might not have installed it correctly given that the version is 1.5.1 whereas the build in master is on 1.7.5. Though that might not do the trick... I have found that installing from source tends to fix this issue but doing so on Windows is tricky (I use cygwin and install the mingw version of gcc).

I also don't have access to Windows 8 so I can't even try and build a custom version you can try out (though now that I think about it, if you do have a local windows machine you can try compiling it from source there, and try uploading your locally compiled version. I use an old virtual machine for compiling the plugin so if you have access to hardware with Windows installed that might give you a better chance.)

@dstellotri
Copy link

Right, so I have 1.5.1 because that's what I have on my pc (which runs fine), and I needed to put the same version on the external-computer (the one with the issue), and I can only request a finite number of uploads to that external-pc (Which is why I don't want to ask to uploaded a new version of the same program). I can't install it from sources because that external-pc doesn't have access to internet. Presumably I did that for 1.51.

I can try your second suggestion but that will eat one of my uploads, so I think I can try that once I have a bulk of other things to upload. It's a painful process because of the no-internet and limited uploads. Gracias por ayudar anyways!

@mcaceresb
Copy link
Owner

@dstellotri Ok, entiendo. Recomendaria que compiles el plugin en tu maquina local, que pruebes esa version localmente, y si funciona entonces que mandes esa version al server la proxima vez.

Siento que no pueda ser de mucha ayuda ya que no tengo una maquina con Windows Un_n. Como vez de este issue no eres a unica persona que tiene este problema. Saludos!

@dstellotri
Copy link

dstellotri commented Jul 12, 2021 via email

@mcaceresb
Copy link
Owner

gtools utiliza un plugin que esta escrito en C y tiene que ser compilado antes de poder ser usado. Tengo notas al respecto aqui. La version que instalas de SSC o the github viene con el plugin compilado en una maquina virtual con una version antigua de Windows. Espero que si lo compilas en una maquina con una version mas reciente quizas el resultado sea mejor.

@EdSaiedi
Copy link

Success! Thanks a million, this is a fantastic tool.

Here's the output from the make:

rm -f build/gtools_macosx_v3.plugin
rm -f  src/plugin/lib
rm -f  src/plugin/spi
ln -sf ../../lib 	  src/plugin/lib
ln -sf lib/spi-3.0 src/plugin/spi
mkdir -p ./build
gcc -Wall -O3 -bundle -DSYSTEM=APPLEMAC -o build/gtools_macosx_v3.plugin -Ilib/spookyhash/src src/plugin/gtools.c src/plugin/spi/stplugin.c lib/spookyhash/src/context.c lib/spookyhash/src/globals.c lib/spookyhash/src/spookyhash.c
In file included from src/plugin/gtools.c:48:
src/plugin/extra/greshape_fast.c:244:48: warning: variable 'z' is uninitialized when used
      here [-Wuninitialized]
                    outdbl[selx + kvars + 1] = z;
                                               ^
src/plugin/extra/greshape_fast.c:30:16: note: initialize the variable 'z' to silence this
      warning
    ST_double z;
               ^
                = 0.0
1 warning generated.
cp build/*plugin lib/plugin/

Hi Mauricio, looks like an awesome tool. I run into the same issue as stevecicala above. I have an M1 Macbook Pro 2020 with MacOS BigSur v. 11.2.1, Stata 17 for Mac SE version and get the same command-line output after going through the exact procedure you explained to stevecicala. I copied the two gtools_macosx_v3.plugin and gtools_macosx_v2.plugin files into the path you suggested, but after restarting Stata and running any gtools command, I get r(111) erros and no other change to the data structure or output (before following your instructions to steve, I got the same error as him). Any suggestions?

@mcaceresb
Copy link
Owner

@EdSaiedi Just to double check, you should uninstall gtools (make sure that gtools is not recognized from a fresh Stata session) then run from the cloned repo rm build/*osx*. Last, try building the plugin again but instead of copying just the files, install them locally: net install gtools, from(/path/to/gtools/build)

If that still doesn't work then I am sorry, really, but since I don't have access to OSX hardware (or Stata 17!) I would not know how to debug why the plugin is not loading.

@EdSaiedi
Copy link

Mauricio, thanks a lot!

After uninstalling gtools, running net install gtools, from(/path/to/gtools/build) replace in Stata 17 did the trick! Beforehand however, I had to complie the v2 plugin as well using instructions here (https://gtools.readthedocs.io/en/latest/compiling/index.html) after running your rm build/*osx* command (just substituting 2.0 for 3.0 in the two make commands above). It seems the trick for M1 users is such a local installation, as I noticed a few owners discuss here: #73.

Appreciate the super-fast commands! Takes stata computing to another level! Look forward to the gregress commands passing beta 👍 !

@mcaceresb
Copy link
Owner

@RajeshTharyan I got an e-mail notification about your issue. I think you need to download gtools to the server directly. Hope your issue was solved; unsure why your comment is not appearing in the thread.

@mcaceresb mcaceresb mentioned this issue Jul 27, 2022
Closed
@chuneeus
Copy link

chuneeus commented May 29, 2023 •
edited

Hi, i am trying to install in Stata 15 gtools, to use binscatter2, i get this message (i have a mac computer)

net install gtools, from(`github'/mcaceresb/stata-gtools/tree/master/build/)
file /mcaceresb/stata-gtools/tree/master/build/stata.toc not found

Any help how to solve this problem? Thanks

@mcaceresb
Copy link
Owner

@chuneeus Please run the entire code chunk in the installation instructions:

local github "https://raw.githubusercontent.com"
net install gtools, from(`github'/mcaceresb/stata-gtools/master/build/)

Based on the error message, the local github was not defined.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@mcaceresb mcaceresb

Labels
bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
10 participants
@mcaceresb @AnneShi @stevecicala @EdSaiedi @drkrynstrng @binyamink @chuneeus @lffontes @dstellotri @ast-code