我想找出在调用函数期间(在Python中)分配的最大RAM量是多少。还有其他与跟踪RAM使用情况有关的问题:
建议使用哪个Python内存分析器?
如何剖析Python中的内存使用情况?
但是这些似乎允许您在heap()调用方法(例如guppy)时更多地跟踪内存使用情况。但是,我要跟踪的是外部库中的一个函数,该函数无法修改,并且会逐渐占用大量RAM,但是一旦函数执行完成,就可以释放它。有什么方法可以找出函数调用期间使用的RAM总量是多少?
heap()
这个问题似乎很有趣,它使我有理由研究Guppy / Heapy,对此我表示感谢。
我尝试了大约2个小时,让Heapy监视函数调用/进程,而又没有 零 运气地修改其源代码。
我确实找到了一种使用内置Python库完成任务的方法resource。请注意,文档没有指出RU_MAXRSS值返回的内容。另一个SO用户注意到它在kB中。运行Mac OSX 7.3并看着我的系统资源在下面的测试代码中攀升,我相信返回的值以 字节为单位 ,而不是千 字节 。
resource
RU_MAXRSS
我如何使用resource库来监视库调用的10000英尺视图是在一个单独的(可监视的)线程中启动该函数,并在主线程中跟踪该进程的系统资源。下面是您需要运行以对其进行测试的两个文件。
图书馆资源监视器 - whatever_you_want.py
import resource import time from stoppable_thread import StoppableThread class MyLibrarySniffingClass(StoppableThread): def __init__(self, target_lib_call, arg1, arg2): super(MyLibrarySniffingClass, self).__init__() self.target_function = target_lib_call self.arg1 = arg1 self.arg2 = arg2 self.results = None def startup(self): # Overload the startup function print "Calling the Target Library Function..." def cleanup(self): # Overload the cleanup function print "Library Call Complete" def mainloop(self): # Start the library Call self.results = self.target_function(self.arg1, self.arg2) # Kill the thread when complete self.stop() def SomeLongRunningLibraryCall(arg1, arg2): max_dict_entries = 2500 delay_per_entry = .005 some_large_dictionary = {} dict_entry_count = 0 while(1): time.sleep(delay_per_entry) dict_entry_count += 1 some_large_dictionary[dict_entry_count]=range(10000) if len(some_large_dictionary) > max_dict_entries: break print arg1 + " " + arg2 return "Good Bye World" if __name__ == "__main__": # Lib Testing Code mythread = MyLibrarySniffingClass(SomeLongRunningLibraryCall, "Hello", "World") mythread.start() start_mem = resource.getrusage(resource.RUSAGE_SELF).ru_maxrss delta_mem = 0 max_memory = 0 memory_usage_refresh = .005 # Seconds while(1): time.sleep(memory_usage_refresh) delta_mem = (resource.getrusage(resource.RUSAGE_SELF).ru_maxrss) - start_mem if delta_mem > max_memory: max_memory = delta_mem # Uncomment this line to see the memory usuage during run-time # print "Memory Usage During Call: %d MB" % (delta_mem / 1000000.0) # Check to see if the library call is complete if mythread.isShutdown(): print mythread.results break; print "\nMAX Memory Usage in MB: " + str(round(max_memory / 1000.0, 3))
停止的主题 - stoppable_thread.py
import threading import time class StoppableThread(threading.Thread): def __init__(self): super(StoppableThread, self).__init__() self.daemon = True self.__monitor = threading.Event() self.__monitor.set() self.__has_shutdown = False def run(self): '''Overloads the threading.Thread.run''' # Call the User's Startup functions self.startup() # Loop until the thread is stopped while self.isRunning(): self.mainloop() # Clean up self.cleanup() # Flag to the outside world that the thread has exited # AND that the cleanup is complete self.__has_shutdown = True def stop(self): self.__monitor.clear() def isRunning(self): return self.__monitor.isSet() def isShutdown(self): return self.__has_shutdown ############################### ### User Defined Functions #### ############################### def mainloop(self): ''' Expected to be overwritten in a subclass!! Note that Stoppable while(1) is handled in the built in "run". ''' pass def startup(self): '''Expected to be overwritten in a subclass!!''' pass def cleanup(self): '''Expected to be overwritten in a subclass!!''' pass