Thank you for the very clear pointers! All changes are implemented as requested.

subprocess did not work for me to source PLUMED. I opted to add the built-in wrapper to the current PYTHONPATH. Please let me know if this is fine.

@apdealbao : Right now it seems that the PLUMED RMSD and MDTraj's RMSD are not looking at the same trajectory, probably a result of the last set of changes. That's the only error in unit testing. (Once it passes tests without errors, I can check things like test coverage, etc.)

Hi @dwhswenson. I am unable to track the error. In the last commit, I updated the tests/test_data/plumed_wrapper/AD_plumed_rmsd.pdb file to be the same as the first frame of tests/test_data/ala_small_traj.pdb, and in my local branch test_rmsd_vs_mdtraj is passed. Maybe there's something I'm missing.

I reran tests, and that didn't fix. Looking at the error, it's clear that MDTraj has it right and PLUMED has it wrong: with t=0 as the reference frame, the RMSD should be 0 at t=0. It is for MDTraj; is not for PLUMED. The numbers look like the wrong thing is being used as reference, but I don't see that in the code.

Is it possible that PLUMED is reordering equivalent hydrogens or something? It seems like some value entering the RMSD is off by 0.101, but I couldn't find an obvious culprit. Let's take closer looks tomorrow, and if we can't figure it out there, I'll ping Gareth and Giovanni to see if they can help.

It's possible that something is going wrong because one has waters and one does not? That's another guess. In any case, I probably won't look into it until tomorrow afternoon.

Alright. I think it should be easy to fix. I also had that 0.101 error before, and I corrected it by updating the tests/test_data/plumed_wrapper/AD_plumed_rmsd.pdb, so that it was the same as the first frame of the new trajectory. Now it's clear that I probably didn't update that properly in my commit and push. I'll try again early tomorrow.

Hi @dwhswenson. Strangely, in my previous commit (70c899c ) I can already see the corrected tests/test_data/plumed_wrapper/AD_plumed_rmsd.pdb. Not sure what is going wrong.

It runs fine for me locally, as well. I'm on a Mac; I think that you are, too, @apdealbao. It is possible that this is something specific to the Linux version of PLUMED.

I suspect we're going to need to ask the PLUMED team for help here. Could you add a pure-(py-)PLUMED version to the test (i.e., manually include the commands your wrapper classes automate), and compare that your wrapper version? Showing an implementation only based on PLUMED will make it easier for them to debug.

Note that you should do git pull before starting on this -- I added some changes to really ensure that that both are using the same reference frame. (Before MDTraj was technically using frame 0 of the trajectory, not the separate PDB file, which contains the same information. Now it uses the separate PDB file.)

I've been trying this on a Linux box. I can sometimes reproduce the error.

The little bash script I'm using:

#!/bin/bash

COUNT=0

for i in `seq 10`
do
    py.test --pyargs openpathsampling.tests.test_plumed_wrapper > /dev/null
    result_code=$?
    if [ $result_code -eq 1 ]
    then
        echo -n "X"
        COUNT=$((COUNT+1))
    elif [ $result_code -eq 0 ]
    then
        echo -n "."
    else
        echo -n "?"
    fi
done
echo " : $COUNT failures"

Output gives . if all the plumed wrapper tests pass; X if a test fails; ? if something weird happens. Here's what I got from three runs on a Linux box:

X...X.X..X : 4 failures
......X.X. : 2 failures
X......XX. : 3 failures

Whether it fails seems to be random, with about 30% probability. And on my Mac:

.......... : 0 failures
.......... : 0 failures
.......... : 0 failures

Now, what's really weird is that (on Linux) I get a serious mistake on the first frame if I only run the RMSD test. There are two ways of doing this:

• ensure that I'm using local data: cd openpathsampling/tests; then py.test test_plumed_wrapper.py::TestPLUMEDCV::test_rmsd_vs_mdtraj
• use installed data: from anywhere py.test -k rmsd --pyargs openpathsampling.tests.test_plumed_wrapper

Both give me no error on Mac, same error on Linux:

_______________________ TestPLUMEDCV.test_rmsd_vs_mdtraj _______________________

self = <openpathsampling.tests.test_plumed_wrapper.TestPLUMEDCV object at 0x147787734a10>

    def test_rmsd_vs_mdtraj(self):
        plmd = PLUMEDInterface(self.topology)
        rmsd_ref_file = data_filename(os.path.join("plumed_wrapper",
                                                   "AD_plumed_rmsd.pdb"))
        md_ref = md.load(rmsd_ref_file)
        rmsd_md = paths.MDTrajFunctionCV("rmsd_md", md.rmsd, self.topology,
                                         reference=md_ref, frame=0,
                                         atom_indices=range(22))
        rmsd_pl = PLUMEDCV("rmsd_pl", plmd, "RMSD REFERENCE="
                           + rmsd_ref_file + " TYPE=OPTIMAL")

        np.testing.assert_almost_equal(rmsd_md(self.trajectory),
>                                      rmsd_pl(self.trajectory), decimal=3)
E       AssertionError:
E       Arrays are not almost equal to 3 decimals
E
E       Mismatched elements: 1 / 10 (10%)
E       Max absolute difference: 0.2579862
E       Max relative difference: 1.
E        x: array([0.   , 0.003, 0.006, 0.008, 0.012, 0.018, 0.024, 0.03 , 0.036,
E              0.042], dtype=float32)
E        y: array([0.258, 0.003, 0.006, 0.008, 0.012, 0.018, 0.024, 0.03 , 0.036,
E              0.042])

The first frame is way off, and everything else is fine. When I only run the RMSD test, I always get an error on Linux. This looks kind of like an uninitialized variable error in C: for example, it could be that the Mac conda-forge compiler automatically initializes things to 0 and the Linux compiler doesn't, and that might be why we see a difference.

Is there anywhere you might need to initialize something for PLUMED?

Hi @dwhswenson. I'm currently trying to make a test using the plumed module directly. It will take me some time, but I'll let you know what comes out. Indeed that are many things that are initialized in the interface and perhaps one is responsible for the error.

hi @dwhswenson. I added a new test that uses the plumed module directly. The tests is successfully passed in my Mac, but it might show an error in Linux.

@apdealbao : Thanks for assembling that extra test. I can confirm that it does reproduce the errors I'd seen previously. However, when I used it as the basis of a simple script (with no OPS at all; only MDTraj and PLUMED), I couldn't reproduce the errors. I'll be going through the test bit by bit to make your new test more like my MDTraj-based script, and we'll see if that makes anything clearer.

ok, @dwhswenson. Let me know if I can help with anything else.

Hi @dwhswenson. Just saw the fix :) Any idea why this didn't work in Linux? Just as a take-home message for me.

No idea at all. I'm going to post an issue on PLUMED; maybe they'll have an idea. Since it is only a problem for RMSD, I'm wondering if it is a detail there. My best guess is that first binding it to a name somehow prevents garbage collection? Anyway, I'll CC you on that issue when I post it.

I figured it out because I had rewritten something based on your test_direct, but it didn't reproduce the error. So I slowly changed test_direct to be more like my working version. I had to triple check to be sure that change actually fixed it, because I see no reason that it would.

Hi @dwhswenson. Busy day. I just squashed the commits. I'll take a look at the other comments and try to address them by tomorrow. Thanks for the guidance!

Sorry for the delay on getting this merged. It has been ready to go in for a while. Merging now!

Great news! Thanks a lot @dwhswenson!