the really meaning of --native

[zdyj3170101136]

if i use python to call c api to allocate too much memory.

what is the true meaning of --native?

1, only i could see python callstack with --native.

2,i could see python stack but only i could see c/c++ callstack with --native?

if in second case, in most of case, even i use python to call c api to allocate memory, without --native is enough for me to debug.

[pablogsal]

Please, for questions like this use the discussions forum: https://github.com/bloomberg/memray/discussions

[pablogsal]

@zdyj3170101136 can you provide an example repository or code snippet of what you are trying to do? --native allows you to include native stacks (like C/C++...) that may also include functions in the Python interpreter but also on your extension module. Unfortunately without a reproducer or some example code we won't be able to help more

[zdyj3170101136]

i use pillow to read data.

#from ddtrace.profiling import Profiler

#prof = Profiler(
#        url="http://127.0.0.1:8081",
  #enable_code_provenance=True, # provide library version
#)
#prof.start() # Should be as early as possible, eg before other imports, to ensure everything is profiled
import os
import PIL
from PIL import Image
import time
dir = "/home/centos/pillow-bench/Images_0/0"
onlyfiles = []

#import tracemalloc

#tracemalloc.start()

filedir = dir
files = [f for f in os.listdir(filedir) if os.path.isfile(os.path.join(filedir, f))]
filepaths = map( lambda x: os.path.join(filedir, x), files )
onlyfiles += list(filepaths)

images = []
s = 0
for i in range(len(onlyfiles)):
    fname = onlyfiles[i]
    #print(fname)
    image = PIL.Image.open(fname)
    # Do something
    images.append(image)
    image.tobytes("xbm", "rgb")
    if i / 10000 == 3:
        print('iteration ', i)
        break

the memray output:

[zdyj3170101136]

seems even without --native the python frame of memory allocation is accurate.

so without --native only means i could see those func name of c and c++?

[pablogsal]

Yeah --native allows you to include c/c++ data in the flamegraph. Please refer to the documentation as all of this is covered there.

[godlygeek]

seems even without --native the python frame of memory allocation is accurate.

Without --native allocations performed by native code are counted as though they were made by the most recent ancestor Python frame in the stack.

With --native allocations performed by native code are shown as being made by a native frame in a hybrid stack that includes both Python and native frames intermingled.