视频1 视频21 视频41 视频61 视频文章1 视频文章21 视频文章41 视频文章61 推荐1 推荐3 推荐5 推荐7 推荐9 推荐11 推荐13 推荐15 推荐17 推荐19 推荐21 推荐23 推荐25 推荐27 推荐29 推荐31 推荐33 推荐35 推荐37 推荐39 推荐41 推荐43 推荐45 推荐47 推荐49 关键词1 关键词101 关键词201 关键词301 关键词401 关键词501 关键词601 关键词701 关键词801 关键词901 关键词1001 关键词1101 关键词1201 关键词1301 关键词1401 关键词1501 关键词1601 关键词1701 关键词1801 关键词1901 视频扩展1 视频扩展6 视频扩展11 视频扩展16 文章1 文章201 文章401 文章601 文章801 文章1001 资讯1 资讯501 资讯1001 资讯1501 标签1 标签501 标签1001 关键词1 关键词501 关键词1001 关键词1501 专题2001
2款Python内存检测工具介绍和使用方法
2020-11-27 14:38:28 责编:小采
文档


去年自己写过一个程序时,不太确定自己的内存使用量,就想找写工具来打印程序或函数的内存使用量。
这里将上次找到的2个内存检测工具的基本用法记录一下,今后分析Python程序内存使用量时也是需要的。

memory_profiler模块(与psutil一起使用)
注:psutil这模块,我太喜欢了,它实现了很多Linux命令的主要功能,如:ps, top, lsof, netstat, ifconfig, who, df, kill, free 等等。
示例代码(https://github.com/smilejay/python/blob/master/py2014/mem_profile.py):

代码如下:


#!/usr/bin/env python

'''
Created on May 31, 2014

@author: Jay
@description: use memory_profiler module for profiling programs/functions.
'''

from memory_profiler import profile
from memory_profiler import memory_usage
import time


@profile
def my_func():
a = [1] * (10 ** 6)
b = [2] * (2 * 10 ** 7)
del b
return a


def cur_python_mem():
mem_usage = memory_usage(-1, interval=0.2, timeout=1)
return mem_usage


def f(a, n=100):
time.sleep(1)
b = [a] * n
time.sleep(1)
return b

if __name__ == '__main__':
a = my_func()
print cur_python_mem()
print ""
print memory_usage((f, (1,), {'n': int(1e6)}), interval=0.5)

运行上面的代码,输出结果为:

代码如下:


jay@Jay-Air:~/workspace/python.git/py2014 $python mem_profile.py
Filename: mem_profile.py

Line # Mem usage Increment Line Contents
================================================
15 8.0 MiB 0.0 MiB @profile
16 def my_func():
17 15.6 MiB 7.6 MiB a = [1] * (10 ** 6)
18 168.2 MiB 152.6 MiB b = [2] * (2 * 10 ** 7)
19 15.6 MiB -152.6 MiB del b
20 15.6 MiB 0.0 MiB return a


[15.61328125, 15.6171875, 15.6171875, 15.6171875, 15.6171875]

[15.97265625, 16.00390625, 16.00390625, 17.0546875, 23.63671875, 23.63671875, 23.0625]

Guppy (使用了Heapy)
Guppy is an umbrella package combining Heapy and GSL with support utilities such as the Glue module that keeps things together.
示例代码(https://github.com/smilejay/python/blob/master/py2014/try_guppy.py):

代码如下:


#!/usr/bin/env python

'''
Created on May 31, 2014

@author: Jay

@description: just try to use Guppy-PE (useing Heapy) for memory profiling.
'''


from guppy import hpy

a = [8] * (10 ** 6)

h = hpy()
print h.heap()
print h.heap().more
print h.heap().more.more

注意其中,要输出更多信息的.more用法。
运行上面的程序,输出结果为:

代码如下:


jay@Jay-Air:~/workspace/python.git/py2014 $python try_guppy.py
Partition of a set of 26963 objects. Total size = 11557848 bytes.
Index Count % Size % Cumulative % Kind (class / dict of class)
0 177 1 8151560 71 8151560 71 list
1 12056 45 996840 9 9148400 79 str
2 5999 22 488232 4 9636632 83 tuple
3 324 1 283104 2 9919736 86 dict (no owner)
4 68 0 2116 2 10136152 88 dict of module
5 199 1 210856 2 10347008 90 dict of type
6 16 6 210688 2 10557696 91 types.CodeType
7 1610 6 193200 2 107506 93 function
8 199 1 177008 2 10927904 95 type
9 124 0 135328 1 11063232 96 dict of class
<91 more rows. Type e.g. '_.more' to view.>
Index Count % Size % Cumulative % Kind (class / dict of class)
10 1045 4 83600 1 11148456 96 __builtin__.wrapper_descriptor
11 109 0 69688 1 11218144 97 dict of guppy.etc.Glue.Interface
12 3 1 34232 0 11252376 97 __builtin__.weakref
13 427 2 30744 0 11283120 97 types.BuiltinFunctionType
14 411 2 29592 0 11312712 98 __builtin__.method_descriptor
15 25 0 26200 0 113312 98 dict of guppy.etc.Glue.Share
16 108 0 25056 0 11363968 98 __builtin__.set
17 818 3 19632 0 11383600 98 int
18 66 0 18480 0 11402080 98 dict of guppy.etc.Glue.Owner
19 16 0 17536 0 11419616 99 dict of abc.ABCMeta
<81 more rows. Type e.g. '_.more' to view.>
(后面省略了部分输出)

另外,还有一个叫“PySizer”的也是做memory profiling的,不过没怎么维护了。

下载本文
显示全文
专题