bson.errors.InvalidDocument when trying to modify and commit Document with EmbeddedDocument as _id

[suconakh]

Im trying to make this model work:

{
    "_id": {
        "user": 1051,
        "peer": 2090
    }
}

What i have is this:

with open('auth.json') as f:
    srv = json.load(f).get('srv')
connection = AsyncIOMotorClient(srv)
instance = MotorAsyncIOInstance(connection.test)


@instance.register
class Id(EmbeddedDocument):
    user = IntegerField(required=True)
    peer = IntegerField(required=True)


@instance.register
class User(Document):
    id = EmbeddedField(Id, attribute='_id')
    name = StringField()

Everything is good when i'm inserting new document into database. But i get an error when i try to commit modified document fetched from databse.

async def async_main():
    user = User(id=Id(user=1051, peer=2090), name='Testing')
    await user.commit()
    # Everything is fine

    user_db = await User.find_one({'name': 'Testing'})
    user_db.name = 'qwe'
    await user_db.commit()
    # bson.errors.InvalidDocument

if __name__ == '__main__':
    loop = get_event_loop()
    loop.run_until_complete(async_main())

Full traceback:

Traceback (most recent call last):
  File "D:\_Projects\python\dbtest\database.py", line 44, in <module>
    loop.run_until_complete(async_main())
  File "C:\Python391\lib\asyncio\base_events.py", line 642, in run_until_complete
    return future.result()
  File "D:\_Projects\python\dbtest\database.py", line 35, in async_main
    await user_db.commit()
  File "D:\_Projects\python\dbtest\py\lib\site-packages\umongo\frameworks\motor_asyncio.py", line 169, in commit
    ret = await self.collection.update_one(
  File "C:\Python391\lib\concurrent\futures\thread.py", line 52, in run
    result = self.fn(*self.args, **self.kwargs)
  File "D:\_Projects\python\dbtest\py\lib\site-packages\pymongo\collection.py", line 1019, in update_one
    self._update_retryable(
  File "D:\_Projects\python\dbtest\py\lib\site-packages\pymongo\collection.py", line 868, in _update_retryable
    return self.__database.client._retryable_write(
  File "D:\_Projects\python\dbtest\py\lib\site-packages\pymongo\mongo_client.py", line 1498, in _retryable_write
    return self._retry_with_session(retryable, func, s, None)
  File "D:\_Projects\python\dbtest\py\lib\site-packages\pymongo\mongo_client.py", line 1384, in _retry_with_session
    return self._retry_internal(retryable, func, session, bulk)
  File "D:\_Projects\python\dbtest\py\lib\site-packages\pymongo\mongo_client.py", line 1416, in _retry_internal
    return func(session, sock_info, retryable)
  File "D:\_Projects\python\dbtest\py\lib\site-packages\pymongo\collection.py", line 860, in _update
    return self._update(
  File "D:\_Projects\python\dbtest\py\lib\site-packages\pymongo\collection.py", line 829, in _update
    result = sock_info.command(
  File "D:\_Projects\python\dbtest\py\lib\site-packages\pymongo\pool.py", line 699, in command
    self._raise_connection_failure(error)
  File "D:\_Projects\python\dbtest\py\lib\site-packages\pymongo\pool.py", line 683, in command
    return command(self, dbname, spec, slave_ok,
  File "D:\_Projects\python\dbtest\py\lib\site-packages\pymongo\network.py", line 120, in command
    request_id, msg, size, max_doc_size = message._op_msg(
  File "D:\_Projects\python\dbtest\py\lib\site-packages\pymongo\message.py", line 714, in _op_msg
    return _op_msg_uncompressed(
bson.errors.InvalidDocument: cannot encode object: <object EmbeddedDocument __main__.Id({'user': 1051, 'peer': 2090})>, of type: <Implementation class '__main__.Id'>

Process finished with exit code 1

I have almost the same issue with odmantic

My environment:

uMongo version: 3.0.0
(I tried with both pymongo 3.11.2 and motor 2.3.0)
MongoDB version: 4.2.11
Python version: 3.9.1 (tags/v3.9.1:1e5d33e, Dec 7 2020, 17:08:21) [MSC v.1927 64 bit (AMD64)]
Platform: Windows-10-10.0.18362-SP0

[lafrech]

Hi. Thanks for reporting.

I believe the issue is here:

umongo/umongo/frameworks/motor_asyncio.py

Line 155 in 7d199db

query['_id'] = self.pk

The pk should be dumped (as in "to_mongo") before being passed to the query. Current code only works if pymongo accepts the id, which is generally the case with an ObjectId but is not the case for special values such as EmbeddedDocument.

This should be fixed in each driver. Could be worth, perhaps even necessary, to add a new method to Document to dump its own pk.

We should check other calls to pk to see if this had been overlooked in other places.

Would you like to tackle this and submit a PR?

[suconakh]

Would you like to tackle this and submit a PR?

Would love to, but I don't think I have enough knowledge to figure it out myself since i'm just learning to code 😞