Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to get file path saved to MemoryFs? #541

Open
coolice2015 opened this issue Jul 1, 2022 · 5 comments
Open

How to get file path saved to MemoryFs? #541

coolice2015 opened this issue Jul 1, 2022 · 5 comments
Labels

Comments

@coolice2015
Copy link

coolice2015 commented Jul 1, 2022

Hi,

May I ask your kind help on a dummie question, I am strugling with in related to MemoryFs ?

I need to pass a screenshot image to telegram chat, and its only accept file path or url for the image:

photo_file_fullpath = "" # variable to store the image full path
bot.sendPhoto (<id>, photo = open (photo_file_fullpath, 'rb')

Its working perfectly when i save the image file on local files system, but I would like to avoid that, so I though I use MemoryFS as "memory filesystem" and access the file form there. To keep it simple, I do the following:

    from fs.memoryfs import MemoryFS
    import pyautogui
    
    screenshot_image = pyautogui.screenshot ()
    
    mem_fs = MemoryFS()
    storage = mem_fs.open ("test_image.png", 'wb')
    screenshot_image.save (storage)
    storage.close ()
    
.... need to acccess the saved image with by memoryfs file path ....

    mem_fs.close()

Could you help me please how can I get the file path of the image file stored in memoryfs as string on windows to be able to pass it to photo_file_fullpath variable and tehrefore sendPhoto could acces it (so it'll behave as a normal file) ?

Many thanks for your time and help!

@lurch
Copy link
Contributor

lurch commented Jul 1, 2022

You can't do that - files in the MemoryFS only exist in memory, and thus can only be accessed by pyfilesystem-aware code.
For anything that doesn't have pyfilesystem integration, I think the best you can do is copy the file from a MemoryFS to a TempFS, and get the syspath of the file in the TempFS.

Although if your third-party code accepts an open-filehandle rather than just a filepath-string, you could use https://docs.pyfilesystem.org/en/latest/reference/base.html#fs.base.FS.openbin directly from the MemoryFS.

@coolice2015
Copy link
Author

Many Thanks!!!

@remixer-dec
Copy link

remixer-dec commented Oct 9, 2023

If the goal is to use MemoryFS with a third party library, where you cannot edit all the code to use MemoryFS manually, there is a way to implement this (but it may not work):

  • put a real path that you want to use as a MemoryFS in a variable
  • replace built-in methods with a wrapper that checks if path matches that variable
  • if it doesn't, call original method, if it does, return MemoryFS method

Here is a basic example that can be a starting point:

import builtins
import fs
import os

if not hasattr(builtins.open, 'patched'):

    mem_fs = fs.open_fs('mem://')
    mem_fs.makedir('vdir')
    vdir_path = '/path/to/your/target/dir/'

    def patch_function(func, patched_func):
        def patched(*args, **kwargs):
            # uncomment for debugging print(args, func.__name__)
            if len(args) > 0 and isinstance(args[0], str):
                if args[0].startswith(vdir_path):
                    virtual_path = args[0].replace(vdir_path, 'vdir/')
                    return patched_func(virtual_path, *args[1:], **kwargs)
            return func(*args, **kwargs)
        return patched

    builtins.open = patch_function(builtins.open, mem_fs.open)
    os.mkdir = patch_function(os.mkdir, mem_fs.makedir)
    os.makedirs = patch_function(os.makedirs, mem_fs.makedirs)
    os.path.exists = patch_function(os.path.exists, mem_fs.exists)
    builtins.open.patched = True

It is far away from production ready, because it cannot handle relative paths and some functions from fs are not fully compatible (for example makedirs is missing exist_ok keyword argument) but for basic use-cases it can work.

I was trying to make a wrapper for a project that auto-converts models to a compatible format and saves them in disk cache, because I'm running low on storage, but unfortunately the saving is implemented via safetensors library and is handled in rust code, that cannot be patched.

@lurch
Copy link
Contributor

lurch commented Oct 9, 2023

@remixer-dec See #155 which I think is what you're suggesting?

@remixer-dec
Copy link

thanks, it looks similar and even has more advanced patches

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

4 participants