Mongoose Populate

[lxhan]

Versions

• NodeJS: 15.2.1
• mongodb-memory-server-*: 7.3.4
• mongodb: 5.0.1
• mongoose: 5.7.13
• system: MacOS

package: mongo-memory-server

What is your question?

Is mongoose population supported?

Models

const MerchantSchema = Schema({
  restaurants: [{ type: Schema.Types.ObjectId, ref: 'restaurant' }]
});

const RestaurantSchema = Schema({
  owner: Schema.Types.ObjectId,
});

RestaurantSchema.virtual('merchant', {
  ref: 'merchant',
  localField: 'owner',
  foreignField: '_id',
  justOne: true,
})

This code is working with real mongodb and in docker but I can't get it working with jest and mongodb-memory-server.
I tried with normal population and with virtuals but I get empty object where I should get populated.

await Restaurant.findById(id).populate('merchant');

[hasezoey]

Is mongoose population supported?

Why shouldn't it be supported? when you refer to real mongodb as in manually starting and using an binary, then it should not be different, as long as nothing Atlas exclusive (which populate is not), then everything should work the same

but I can't get it working with jest and mongodb-memory-server.

because of the reason mentioned above, it should work the same, only difference by default is that this library here use the storage engine ephemeralForTest, which does not persist data across restarts

---

to help further, please provide a full jest file (or full reproduction repository if multiple files) to help with this

[lxhan]

fake db

import mongoose from 'mongoose';
import { MongoMemoryServer } from 'mongodb-memory-server';

let mongod: MongoMemoryServer;

export const connect = async () => {
  mongod = await MongoMemoryServer.create({
    binary: {
      version: '5.0.1',
    },
  });
  const uri = mongod.getUri();

  const mongooseOptions = {
    useNewUrlParser: true,
    useUnifiedTopology: true,
    useCreateIndex: true,
  };

  await mongoose.connect(uri, mongooseOptions);
};

export const disconnect = async () => {
  await mongoose.connection.close();
  await mongod.stop();
};

test file

beforeAll(() => connect());

afterAll(() => disconnect());

describe('placeOrder', () => {
  it('should create product and populate modifiers', async () => {
    // in generateMockDataForPlaceOrder I'm creating product with 2 modifiers
    await generateMockDataForPlaceOrder();

    // here when populating I'll get product with modifier populated 
    const product = await Product.findById(id).populate({ path: 'modifier' });

    // product.modifier
    {
        id: 1623306319264,
        name: 'mod 2',
        sortOrder: 0,
        isFixed: false,
        numOfModifiers: 3
    }

    expect(product.modifier.name).toEqual('mod 2');
  });
});

[hasezoey]

i forgot to mentioned, mongoose currently (version 5.13) does not support mongodb 5.x, because officially you would require either mongodb(driver) 3.7 or 4.x, and mongoose currently (5.13) uses mongodb 3.6

thanks for the code, after looking at it, i dont see anything wrong (aside from the version mentioned above)
could you list what you mean with real mongodb and docker (as in, which distro, binary version, and way you start it)?

[lxhan]

Initially I used default version 4.x.
By real mongodb I mean atlas.
Docker v 20.10.7

docker-compose.yml

version: "3"

services:
  db:
    user: $UID
    image: mongo
    ports:
      - "27017:27017"
    environment:
      MONGO_INITDB_ROOT_USERNAME: ${INITDB_ROOT_USERNAME}
      MONGO_INITDB_ROOT_PASSWORD: ${INITDB_ROOT_PASSWORD}
      MONGO_INITDB_DATABASE: ${DB_NAME}
      MONGO_USERNAME: ${DB_USER}
      MONGO_PASSWORD: ${DB_PASS}
    volumes:
      - ./data:/data/db
      - ./db-init.sh:/docker-entrypoint-initdb.d/db-init.sh:ro

db-init.sh

mongo -- "$MONGO_INITDB_DATABASE" <<EOF
  db.createUser({
    user: "$MONGO_USERNAME",
    pwd: "$MONGO_PASSWORD",
    roles: [
      { role: 'readWrite', db: "$MONGO_INITDB_DATABASE" }
    ]
  });
EOF

[kornelijussl]

Same problem, works fine except in jest environment.. :(

[kornelijussl]

Ahh, my bad didn't quite get it at first that it's just abbreviation.

Answering your questions:

"mongodb-memory-server": "^8.5.2",
"mongoose": "^6.2.9",
Windows 10 Version 10.0.19044 Build 19044
MongoBinary options: {
"version": "5.0.3",
"downloadDir": "C:\Users\User\.cache\mongodb-binaries",
"os": {
"os": "win32"
},
"arch": "x64",
"systemBinary": "",
"platform": "win32",
"checkMD5": false
}

Not sure what you mean by "real" but both official mongodb and in memory works as expected for all cases except when populate is being used.

Which part of debug output is needed because it's massive.

[hasezoey]

Which part of debug output is needed because it's massive.

if its just the populate that is problematic, then the last ~20 lines before and after the query is executed (eg calling .populate in mongoose)

---

it is weird that just populate does not work, have you ensured that the id on the property that is to be populated actually exists and is with the correct model?
also just to be on the same page, what populate / reference type are we speaking about, normal references or virtual-populate or both? (or in case typegoose's docs are more explanatory: normal populate or virtual-populate)

also, could you provide your mongoose schema and how you are trying to use it?

[kornelijussl]

With Jest debugging on and debug output finally got it resolved. It was my mistake with setting up test environment. Though I was really concerned about mms because in code I was getting the error stating that "MongoDB client is not connected" just before hitting the line with populate. Seemed very odd to me as test above were executing as should. Anyway thank you for your help.

[kornelijussl]

Some updates: it only happens when debugger for jest is turned on, if no breakpoints are set in the server side e.g controllers, service layer then it works ok, but if execution is paused it throws 'MongoClient must be connected to perform this operation' error.

[hasezoey]

but if execution is paused it throws 'MongoClient must be connected to perform this operation' error.

i think mongodb clients need a hearbeat with the server, if it cannot do that the server closes the connection, and while the process is paused it cannot do a heartbeat (or also called "keep-alive" signal)