Python: undefined symbol: _ZN8NNPlugin18NeuralNetworkForceC1ERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE

[bgobbi]

Compilation and "make PythonInstall" now works.

However when I try to issue "from openmmnn import *" in python I get:

>>> from openmmnn import *
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/gstore/home/albertgo/.conda/envs/OpenMM/lib/python3.6/site-packages/openmmnn.py", line 28, in <module>
    _openmmnn = swig_import_helper()
  File "/gstore/home/albertgo/.conda/envs/OpenMM/lib/python3.6/site-packages/openmmnn.py", line 24, in swig_import_helper
    _mod = imp.load_module('_openmmnn', fp, pathname, description)
  File "/gstore/home/albertgo/.conda/envs/OpenMM/lib/python3.6/imp.py", line 243, in load_module
    return load_dynamic(name, filename, file)
  File "/gstore/home/albertgo/.conda/envs/OpenMM/lib/python3.6/imp.py", line 343, in load_dynamic
    return _load(spec)
ImportError: /gstore/home/albertgo/.conda/envs/OpenMM/lib/python3.6/site-packages/_openmmnn.cpython-36m-x86_64-linux-gnu.so: undefined symbol: _ZN8NNPlugin18NeuralNetworkForceC1ERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE

I checked and found that in: https://github.com/pandegroup/openmm-nn/blob/master/openmmapi/src/NeuralNetworkForce.cpp the constructur seems defined as:
NeuralNetworkForce::NeuralNetworkForce(const std::string& file) : file(file), usePeriodic(false)
While in: https://github.com/pandegroup/openmm-nn/blob/master/openmmapi/include/NeuralNetworkForce.h it is given as:
NeuralNetworkForce(const std::string& file);

Could this be the cause?
Alberto

[peastman]

Those declarations are identical. And it's working fine for me, so I don't think it's a problem in the SWIG definition. Much more likely this is caused by an ABI inconsistency, for example one library being compiled with C++11 support and another one without it. What OpenMM version are you using, and did you build it from source or install it with conda? 7.2 did not use C++11, but 7.3 does.

Also see https://github.com/peastman/openmm-plumed/issues/2, which was a similar problem with a different plugin.

[bgobbi]

my openMM was installed with conda. During ccmake of the plugin I have had to specify -D_GLIBCXX_USE_CXX11_ABI=0 to make it compile.
nm /gstore/home/albertgo/.conda/envs/OpenMM/lib/python3.6/site-packages/_openmmnn.cpython-36m-x86_64-linux-gnu.so |fgrep _ZN8NNPlugin18NeuralNetworkForceC1ERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE
does return the correct symbol.

Thanks for taking a look!

[peastman]

That sounds like the problem then. Notice the symbol it's looking for contains the substring cxx11. That tells us it expects the plugin to be compiled with the C++11 ABI. Try compiling OpenMM from source, or alternatively install the 7.3 beta with conda. Then compile the plugin against that with C++11 enabled.

[gitkol]

Hi, I installed OpenMM 7.3 with conda and as I understand it was compiled with C++11. I am having the same issue with an unresolved reference
ImportError: /raid/istvan/anaconda3/lib/python3.7/site-packages/_openmmplumed.cpython-37m-x86_64-linux-gnu.so: undefined symbol: _ZN12PlumedPlugin11PlumedForceC1ERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE

I have been through all the issues regarding #16 and #2, the only way I managed to compile the latest plugin was setting
CMAKE_CXX_FLAGS:STRING=-D_GLIBCXX_USE_CXX11_ABI=0
in CMakeCache.txt but, then, it is not clear how I am supposed to compile it with C++11 support.

Can you help me resolve this?

Many thanks,

Istvan

[gitkol]

The closest symbol in libOpenMMPlumed.so is _ZN12PlumedPlugin11PlumedForceC1ERKSs.

[peastman]

OpenMM 7.3 is compiled for C++11. Are you sure you aren't actually trying to link against an older version?

[gitkol]

As far as I can tell, I am not linking against an older version. This is the first OpenMM on this computer and I started with the latest version of everything. My system is Ubuntu 18.04. This issue feels like a catch 22, I ended up installing OpenMM from source yesterday and now the Plumed plugin works flawlessly. :)

Thanks,

Istvan

[peastman]

Ok. I was going to ask you to post the full log so I could see exactly where the error was happening and everything leading up to it. But if you already have it working, probably better not to mess with it!

[gitkol]

Peter, if it helps I'd be happy to reproduce the problem, just let me know what exactly you need.

[peastman]

Can you post the complete error log, and exactly what you did to produce it?

[gitkol]

Will try to reproduce it and capture the output.

[gitkol]

openmm-plumed-bulid-log.txt

I think I captured everything, tell me if you need anything else. I didn't log how I built plumed-2.50 itself, but that shouldn't matter, and as I said everything works well when I build OpenMM from source. (I remember also trying to build the plugin with anaconda2 installation of OpenMM, but the result was the same, although the actual symbol missing might have been different.)

[peastman]

The file you posted is getting errors related to the Plumed plugin, not the neural network plugin. Was that from something else? It may well be the same problem though. There's some inconsistency between the symbols in the plugin library (/raid/istvan/anaconda3/lib/libOpenMMPlumed.so) and the symbols in the compiled Python module (/raid/istvan/anaconda3/lib/python3.7/site-packages/_openmmplumed.cpython-37m-x86_64-linux-gnu.so). I notice in the log you checked for this by executing the command

nm -D /raid/istvan/anaconda3/lib/libOpenMMPlumed.so | grep _ZN12PlumedPlugin11PlumedForceC1ERK

but I think your grep was too specific, because it didn't find the constructor we're looking for. Try searching for something more generic like grep PlumedForce. See if you can find the PlumedForce(std::string) constructor, and see how the symbol differs from the one Python is looking for.

[gitkol]

Sorry, I did realize afterwards that this wasn't a plumed thread, I somehow got here via Google searching my missing symbol.

istvan@istvan-valtax79i:~$ nm -D /raid/istvan/anaconda3/lib/libOpenMMPlumed.so | grep PlumedForce
00000000000068da T _ZN12PlumedPlugin11PlumedForceC1ERKSs
00000000000068da T _ZN12PlumedPlugin11PlumedForceC2ERKSs
0000000000006a60 W _ZN12PlumedPlugin11PlumedForceD0Ev
0000000000006a22 W _ZN12PlumedPlugin11PlumedForceD1Ev
0000000000006a22 W _ZN12PlumedPlugin11PlumedForceD2Ev
0000000000006b58 T _ZN12PlumedPlugin15PlumedForceImpl10initializeERN6OpenMM11ContextImplE
0000000000006cee T _ZN12PlumedPlugin15PlumedForceImpl14getKernelNamesEv
0000000000006ee4 W _ZN12PlumedPlugin15PlumedForceImpl18updateContextStateERN6OpenMM11ContextImplERb
0000000000006c68 T _ZN12PlumedPlugin15PlumedForceImpl19calcForcesAndEnergyERN6OpenMM11ContextImplEbbi
0000000000006f68 W _ZN12PlumedPlugin15PlumedForceImpl20getDefaultParametersEv
0000000000006a86 T _ZN12PlumedPlugin15PlumedForceImplC1ERKNS_11PlumedForceE
0000000000006a86 T _ZN12PlumedPlugin15PlumedForceImplC2ERKNS_11PlumedForceE
0000000000006b32 T _ZN12PlumedPlugin15PlumedForceImplD0Ev
0000000000006af4 T _ZN12PlumedPlugin15PlumedForceImplD1Ev
0000000000006af4 T _ZN12PlumedPlugin15PlumedForceImplD2Ev
0000000000006fc5 W _ZN12PlumedPlugin21CalcPlumedForceKernel4NameEv
0000000000008328 T _ZN6OpenMM16PlumedForceProxyC1Ev
0000000000008328 T _ZN6OpenMM16PlumedForceProxyC2Ev
00000000000088f0 W _ZN6OpenMM16PlumedForceProxyD0Ev
00000000000088c2 W _ZN6OpenMM16PlumedForceProxyD1Ev
00000000000088c2 W _ZN6OpenMM16PlumedForceProxyD2Ev
0000000000007154 W _ZN6OpenMM6Kernel5getAsIN12PlumedPlugin21CalcPlumedForceKernelEEERT_v
0000000000006956 T _ZNK12PlumedPlugin11PlumedForce10createImplEv
0000000000006a12 W _ZNK12PlumedPlugin11PlumedForce30usesPeriodicBoundaryConditionsEv
0000000000006944 T _ZNK12PlumedPlugin11PlumedForce9getScriptEv
0000000000006ed2 W _ZNK12PlumedPlugin15PlumedForceImpl8getOwnerEv
0000000000008542 T _ZNK6OpenMM16PlumedForceProxy11deserializeERKNS_17SerializationNodeE
00000000000083ee T _ZNK6OpenMM16PlumedForceProxy9serializeEPKvRNS_17SerializationNodeE
000000000020abe8 V _ZTIN12PlumedPlugin11PlumedForceE
000000000020ac78 V _ZTIN12PlumedPlugin15PlumedForceImplE
000000000020ac60 V _ZTIN12PlumedPlugin21CalcPlumedForceKernelE
000000000020ad40 V _ZTIN6OpenMM16PlumedForceProxyE
0000000000008990 V _ZTSN12PlumedPlugin11PlumedForceE
0000000000008a40 V _ZTSN12PlumedPlugin15PlumedForceImplE
0000000000008a00 V _ZTSN12PlumedPlugin21CalcPlumedForceKernelE
0000000000008ae0 V _ZTSN6OpenMM16PlumedForceProxyE
000000000020abb8 V _ZTVN12PlumedPlugin11PlumedForceE
000000000020ac00 V _ZTVN12PlumedPlugin15PlumedForceImplE
000000000020aca0 V _ZTVN6OpenMM16PlumedForceProxyE

istvan@istvan-valtax79i:~$ nm -D /raid/istvan/anaconda3/lib/python3.7/site-packages/_openmmplumed.cpython-37m-x86_64-linux-gnu.so | grep PlumedForce
                 U _ZN12PlumedPlugin11PlumedForceC1ERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE
                 U _ZNK12PlumedPlugin11PlumedForce9getScriptB5cxx11Ev

istvan@istvan-valtax79i:~$ nm -D /raid/istvan/anaconda3/lib/libOpenMMPlumed.so | grep getScript
0000000000006944 T _ZNK12PlumedPlugin11PlumedForce9getScriptEv

istvan@istvan-valtax79i:~$ nm -D /raid/istvan/anaconda3/lib/libOpenMMPlumed.so | grep _ZN12PlumedPlugin11PlumedForceC1
00000000000068da T _ZN12PlumedPlugin11PlumedForceC1ERKSs

The Python module has only two symbols related to the Plugin and both of them are missing from the Plugin module. In any case, it works fine when I compile OpenMM from source, I am in business. -Thanks

[peastman]

Ok, I think I sort of see what's happening. I don't think it's related to the use of C++11. It's actually related to the "dual ABI" introduced in GCC 5.1. See https://gcc.gnu.org/onlinedocs/libstdc++/manual/using_dual_abi.html for details. Basically, libstdc++ now includes two different implementations of std::string and std::list. Code can use either implementation, but libraries that use one implementation can't be linked to libraries that use the other (at least if they exchange strings or lists). So all code has to be compiled to use the same one. And different compilers (and different versions of a single compiler) have different defaults, which creates a mess.

The OpenMM conda packages are compiled to use the newer version. But apparently your compiler is defaulting to the older one when compiling the plugin, leading to the incompatibility. Instead of recompiling OpenMM, try specifying -D_GLIBCXX_USE_CXX11_ABI=1 when compiling the plugin.

[gitkol]

I already tried that. Whether not specifying any ABI or ABI=1 results in the same error message that I captured in the openmm-plumed-bulid-log.txt file. I can only compile the plugin when I set ABI=0. The compiler is the stock Ubuntu 18.04 gcc, I think it is version 7.3. From what you say it seems that I'll need a different compiler, maybe building gcc from scratch?

[peastman]

No idea why that would happen. The conda packages are compiled with clang, but that shouldn't matter. Both compilers are capable of using either ABI.

[gitkol]

When I compile both OpenMM and the Plugin with the stock gcc the compatible functions are used by default. I guess, at this point that's my only option. It would be nice to have everything under the same anaconda umbrella, but it is working, so that's perfectly fine for now. -Thanks

[moradza]

How did you solve the problem during the installation I get almost the same error!
What should I install or update?

[bgobbi]

Hi,
you can try following the installation description in:
https://github.com/chemalot/openmm-ani

This has to do with the ABI settings of your libraries. It took a long time for me to make it work and I still am not sure what the final trick was. You have to get all the libraries use the same ABI and/or set the correct GLIBCXX_USE_CXX11_ABI setting.

Alberto

[moradza]

Hi,
I will try it, thanks for the reply.
Alireza