Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Storage emulator import/export data error #4322

Closed
rulaman123 opened this issue Mar 17, 2022 · 14 comments · Fixed by #4411
Closed

Storage emulator import/export data error #4322

rulaman123 opened this issue Mar 17, 2022 · 14 comments · Fixed by #4411

Comments

@rulaman123
Copy link

rulaman123 commented Mar 17, 2022

Hi, after starting the storage emulator with import the ui only shows a white page and i get that error:

Failed to load resource: the server responded with a status of 404 (Not Found)
2react-dom.production.min.js:225 FirebaseError: Firebase Storage: Object '...' does not exist. (storage/object-not-found)

[REQUIRED] Environment info

firebase-tools: v10.4.0

Platform: Windows

[REQUIRED] Test case

[REQUIRED] Steps to reproduce

Upload image to storage emulator.
Export image and close emulator.
Open storage emulator with import data.
Error.

[REQUIRED] Expected behavior

[REQUIRED] Actual behavior

@rulaman123 rulaman123 added the bug label Mar 17, 2022
@google-oss-bot
Copy link
Contributor

This issue does not have all the information required by the template. Looks like you forgot to fill out some sections. Please update the issue with more information.

@rulaman123 rulaman123 changed the title Storage emulator export data error Storage emulator import/export data error Mar 18, 2022
@anthony-bernardo
Copy link

same issue for me, I throught it was related to my M1, but it's seams not.

On my old mac (where it's ok) : firebase-cli version is 10.2.1
On my new mac (where it's bugged) : firebase-cli version is 10.4.0

@ermartinez13
Copy link

I've been running into the same issue when I went from version 10.2.1 to 10.4.1. I downgraded to version 10.3.1 and found the bug still present. Only when I went back to v10.2.1 did the bug disappear.

@torstenek
Copy link

Adding some more info:

[REQUIRED] Environment info

I can reproduce this on

  • MB Pro M1 Max
  • MP Pro 2019

[REQUIRED] Steps to reproduce

  1. Start with an empty $FIREBASE_EMULATOR_DATA directory.

  2. Start the emulator with

firebase emulators:start --project myproject -only 'storage' --import $FIREBASE_EMULATOR_DATA --export-on-exit

Now upload, for example, an image myfile.jpg to the bucket (for example by using the web UI). The problem is the same if you upload it via the firebase dart package.

  1. Stop the emulator gracefully and note the "Export complete" log entry indicating that everything went OK.

  2. Start the emulators again

firebase emulators:start --project myproject -only 'storage' --import $FIREBASE_EMULATOR_DATA --export-on-exit

Note the log message

Could not find file "myproject%252Fmyfile.jpg" in storage export.

  1. Verify that the file is indeed missing from the storage emulator.

@torstenek
Copy link

The sequence %252F in the filename could be the result of a double URI encoding.

@tonyjhuang
Copy link
Contributor

Potential duplicate of #4326

@sceee
Copy link
Contributor

sceee commented Mar 31, 2022

Jumping in here as I'm facing the same issue and have more information to add:
After starting the emulator with importing previously exported Storage data, accessing the storage bucket "directory" via the Emulator UI brings me to a white page with an exception in the browser:

FirebaseError: Firebase Storage: Object 'directory/subdirectory/filename' does not exist. (storage/object-not-found)
Not Found

I checked the storage_export\blobs directory in the file system and there, the file seems to be present but in my case the file name looks like this:

my-app.appspot.com%5Cdirectory%5Csubdirectory%5Cfilename

So in my case the %5C are URL encoded backslashes (probably because the export was made on Windows).
By the comments above I am unsure if that's also an issue. Shouldn't these be forward slashes (or url encoded forward slashes -> %2F) according to #4322 (comment) ?
I guess this will also be an issue when going cross-platform.

@alvinchevolleaux
Copy link

alvinchevolleaux commented Apr 4, 2022

I'm having the same issue of the blank page when I click the storage tab of the emulator UI and error messages:

FirebaseError: Firebase Storage: Object 'filename' does not exist. (storage/object-not-found)
Not Found

Failed to load resource: the server responded with a status of 404 (Not Found) :9199/v0/b/bucketname/filename

with a subsequent

Failed to load resource: the server responded with a status of 404 (Not Found)

I do have URL encoding of the filenames in the exported storage blobs. They're in the following format bucketname%2Ffilename. Is this related to the encoding problems affecting other platforms?

I'm on Mac OS with an Intel chip.

@coreform
Copy link

coreform commented Apr 5, 2022

I don't see any logs regarding imports to Storage when importing a production backup that uses Storage. The Emulator Suite Storage tab is also empty of files after such an import.

@tohhsinpei
Copy link
Member

Thanks all for the reports. The main issue looks like a duplicate of #4326 and should be fixed in #4358, which was included in yesterday's release. Let me know if the problem persists.

@sceee You're right, there is still an issue with importing/ exporting across platforms. I'll put in a fix sometime this week; will close this once that is in.

@sceee
Copy link
Contributor

sceee commented Apr 5, 2022

@tohhsinpei thanks. I can confirm that the main issue (Emulator UI crashes on same OS export/import & file not available) is fixed with 10.6.0.

But importing a Windows export on a Mac still makes the emulator UI crash and the file not being available via the API as you wrote. I'm looking forward to the fix, thank you!

@coreform
Copy link

coreform commented Apr 6, 2022

When attempting to seed the emulators with a production backup with firestore-tools 10.6.0, firestore is populated but there is still no sign of Storage logs or data after: firebase emulators:start --import=../backupFolder/

@tonyjhuang
Copy link
Contributor

Hi @coreform, can you file a new issue? This seems unrelated to the original comment in this one.

@alvinchevolleaux
Copy link

Can confirm this is working for me now using 10.6.0.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.