On memory usage when loading a Wasm file with fast interpreter

[nchamzn]

I did some memory profiling when loading Wasm files using the fast interpreter and noticed that not only do we process the Wasm binary to an internal representation, but we also keep the original Wasm binary itself in memory (And have references to it in the internal representation).

Is there any scope to optimise this, e.g. by copying only relevant sections from the Wasm binary so that we can free the rest of the buffer? Are there any known optimisations that can be made to those internal structures to reduce the memory usage of Wamr?

[wenyongh]

Hi, if the whole wasm file is loaded by wasm_runtime_load or wasm_runtime_load_ex, it is not easy to free the buffer of wasm file since some fields in the WASMModule may be referring to it. But if the wasm file is read section by section into a wasm_section_list_t and the host embedder allocates different buffers for each section, and the sections are loaded by wasm_runtime_load_from_sections, then after loading, some of the sections may be freed. You can also refer to the code in wamr-app-framework rep:
https://github.com/bytecodealliance/wamr-app-framework/blob/main/app-mgr/app-manager/module_wasm_app.c#L787-L788

Note that it only frees SECTION_TYPE_DATA section, but for fast-interp, maybe you can try to free SECTION_TYPE_FUNC also.

[nchamzn]

Thanks, I just had a go with this suggestion and it works nicely, however some of the code linked is using wamr internals (E.g. os_mmap, many compiler definitions etc) and my implementation involved a lot of copy/paste.

Would you be OK with a PR which added:

destroy_all_wasm_sections
destroy_part_wasm_sections
destroy_all_aot_sections
destroy_part_aot_sections

and then another API something like:

struct load_section_result {
    wasm_section_list_t sections;
    bool is_aot;
};

read_to_section_result read_to_sections(uint8_t* data, size_t size)

exported from the wamr library?

[nchamzn]

Note that it only frees SECTION_TYPE_DATA section, but for fast-interp, maybe you can try to free SECTION_TYPE_FUNC also

Is there any guidance on which sections I must keep after module instantiation - if I only ever instantiate the module once.

[wenyongh]

Thanks, I just had a go with this suggestion and it works nicely, however some of the code linked is using wamr internals (E.g. os_mmap, many compiler definitions etc) and my implementation involved a lot of copy/paste.

Would you be OK with a PR which added:

destroy_all_wasm_sections
destroy_part_wasm_sections
destroy_all_aot_sections
destroy_part_aot_sections

and then another API something like:

struct load_section_result {
    wasm_section_list_t sections;
    bool is_aot;
};

read_to_section_result read_to_sections(uint8_t* data, size_t size)

exported from the wamr library?

How about:

typedef struct load_section_result_t {
    wasm_section_list_t sections;
    bool is_aot;
} load_section_result_t;

WASM_RUNTIME_API_EXTERN bool
wasm_runtime_read_to_sections(uint8_t *buf, size_t size, load_section_result_t *p_section_result);
// return true/false to indicate success or fail
// put result in p_section_result when success

WASM_RUNTIME_API_EXTERN void
wasm_runtime_destroy_sections(load_section_result_t *section_result, bool destroy_all_sections);
// destroy_all_sections: true to destroy all sections, false to destroy part sections
// and aot or wasm can be found in section_result->is_aot

[wenyongh]

Note that it only frees SECTION_TYPE_DATA section, but for fast-interp, maybe you can try to free SECTION_TYPE_FUNC also

Is there any guidance on which sections I must keep after module instantiation - if I only ever instantiate the module once.

Maybe you can refer to:
https://github.com/bytecodealliance/wamr-app-framework/blob/main/app-mgr/app-manager/module_wasm_app.c#L728-L746
https://github.com/bytecodealliance/wamr-app-framework/blob/main/app-mgr/app-manager/module_wasm_app.c#L728-L746

Sorry that my understanding in the above post is invalid, the sections in sections1 should be abled to be freed after loading and sections in section2 can be freed after instantiation. But I am not sure for the bytecode for interpreter and machine code for AOT, maybe you can investigate the source code for more details.