Python运行慢?来,带你上火箭
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;">全文共<strong style="color: blue;">5573</strong>字,预计学习时长<strong style="color: blue;">16</strong>分钟</p>
<div style="color: black; text-align: left; margin-bottom: 10px;"><img src="https://p3-sign.toutiaoimg.com/pgc-image/45c4843a5d9b4455a095b22f420c17e1~noop.image?_iz=58558&from=article.pc_detail&lk3s=953192f4&x-expires=1723895559&x-signature=01lhxfOvfp8YOMHCmIqMnPnQcqc%3D" style="width: 50%; margin-bottom: 20px;">
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;">图源:Unsplash</p>
</div>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;">Python是一个很酷的语言,<span style="color: black;">由于</span>你<span style="color: black;">能够</span>在很短的时间内利用很少的代码做<span style="color: black;">非常多</span>事情。不仅如此,它还能<span style="color: black;">容易</span>地支持多任务,<span style="color: black;">例如</span>多进程等。</p>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;">但Python运行的慢是历来被诟病的,<span style="color: black;">有些</span>人黑Python的一点是Python的程序运行速度奇慢。</p>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;">这一方面和语言<span style="color: black;">相关</span>,另一方面可能<span style="color: black;">便是</span>你代码的问题。其实,无论<span style="color: black;">运用</span>哪种编程语言,特定程序的运行速度很大程度上都取决于该程序的<span style="color: black;">研发</span>人员及其编写快而优的程序的技巧和能力。</p>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;">语言方面的问题<span style="color: black;">咱们</span><span style="color: black;">处理</span>不了,<span style="color: black;">因此</span>只能在编程技巧上来<span style="color: black;">加强</span>程序的运行效率。</p>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;">是时候证明给<span style="color: black;">哪些</span>python黑粉,让<span style="color: black;">她们</span><span style="color: black;">瞧瞧</span><span style="color: black;">怎样</span><span style="color: black;">提高</span>Python程序性能并使其像坐上火箭<span style="color: black;">同样</span>运行飞快!</p>
<div style="color: black; text-align: left; margin-bottom: 10px;"><img src="https://p3-sign.toutiaoimg.com/pgc-image/81ee40392c6f416582180223f6b2c580~noop.image?_iz=58558&from=article.pc_detail&lk3s=953192f4&x-expires=1723895559&x-signature=UQuboxD2C%2BbXmreMcrNdpHsXFRk%3D" style="width: 50%; margin-bottom: 20px;">
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;">图源:Unsplash</p>
</div>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><strong style="color: blue;">时序分析</strong></p>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;">优化之前,<span style="color: black;">首要</span>要找到是哪部分代码拖慢了<span style="color: black;">全部</span>程序的运行。有时候程序的"瓶颈"不是很<span style="color: black;">显著</span>,<span style="color: black;">倘若</span>找不到,以下是<span style="color: black;">有些</span><span style="color: black;">意见</span>以供参考:</p>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;">重视</span>:这是一个计算e的x次幂的演示程序(出自Python文档):</p># slow_program.pyfrom decimal import*defexp(x):getcontext().prec +=2i, lasts, s, fact, num =0, 0, 1, 1, 1while s != lasts:lasts = si +=1fact *= inum *= xs += num / factgetcontext().prec -=2return+sexp(Decimal(150))exp(Decimal(400))exp(Decimal(3000))<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;">在GitHub上查看rawslow_program.py<span style="color: black;">所有</span>代码</p>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><strong style="color: blue;">最省力的“性能分析”</strong></p>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;">首要</span>,最简单且最省力的<span style="color: black;">处理</span><span style="color: black;">方法</span>是<span style="color: black;">运用</span>Unix的time命令:</p>~ $ time python3.8 slow_program.pyreal 0m11,058suser 0m11,050ssys 0m0,008s<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;">在GitHub上查看rawbase_time.shell<span style="color: black;">所有</span>代码</p>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;">倘若</span>只是给<span style="color: black;">全部</span>程序计时,它<span style="color: black;">特别有</span>用,但还不足够……</p>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><strong style="color: blue;">最<span style="color: black;">仔细</span>的性能分析</strong></p>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;">性能分析的另一<span style="color: black;">办法</span>是cProfile,从中能得到很大的信息量:</p>~ $ python3.8 -m cProfile -s time slow_program.py1297 function calls (1272 primitive calls) in 11.081 secondsOrdered by: internal timencalls tottime percall cumtime percall filename:lineno(function)3 11.079 3.693 11.079 3.693 slow_program.py:4(exp)1 0.000 0.000 0.002 0.002 {built-in method _imp.create_dynamic}4/1 0.000 0.000 11.081 11.081 {built-in method builtins.exec}6 0.000 0.000 0.000 0.000 {built-in method __new__ of type object at 0x9d12c0}6 0.000 0.000 0.000 0.000 abc.py:132(__new__)23 0.000 0.000 0.000 0.000 _weakrefset.py:36(__init__)245 0.000 0.000 0.000 0.000 {built-in method builtins.getattr}2 0.000 0.000 0.000 0.000 {built-in method marshal.loads}10 0.000 0.000 0.000 0.000 <frozen importlib._bootstrap_external>:1233(find_spec)8/4 0.000 0.000 0.000 0.000 abc.py:196(__subclasscheck__)15 0.000 0.000 0.000 0.000 {built-in method posix.stat}6 0.000 0.000 0.000 0.000 {built-in method builtins.__build_class__}1 0.000 0.000 0.000 0.000 __init__.py:357(namedtuple)48 0.000 0.000 0.000 0.000 <frozen importlib._bootstrap_external>:57(_path_join)48 0.000 0.000 0.000 0.000 <frozen importlib._bootstrap_external>:59(<listcomp>)1 0.000 0.000 11.081 11.081 slow_program.py:1(<module>)...<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;">在GitHub上查看rawcprofile.shell<span style="color: black;">所有</span>代码</p>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;">这儿</span>用cProfile模块和time参数运行测试脚本,以便按内部时间(cumtime)对行进行排序。从中<span style="color: black;">能够</span>得到<span style="color: black;">非常多</span>信息,以上所列结果约为<span style="color: black;">实质</span>输出的10%。由此可见,exp函数<span style="color: black;">便是</span>拖慢程序的“罪魁祸首”(太神奇啦!),现在<span style="color: black;">瞧瞧</span>更详尽的时序和性能分析......</p>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><strong style="color: blue;">对特定函数计时</strong></p>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;">已然</span><span style="color: black;">晓得</span>拖慢程序运行的函数,下一步可<span style="color: black;">运用</span>简单的修饰器,专门对该函数计时,不<span style="color: black;">测绘</span>其余代码。如下所示:</p>deftimeit_wrapper(func):@wraps(func)defwrapper(*args, **kwargs):start = time.perf_counter() # Alternatively, you can use time.process_time()func_return_val = func(*args, **kwargs)end = time.perf_counter()print({0:<10}.{1:<8} : {2:<8}.format(func.__module__, func.__name__, end - start))return func_return_valreturn wrapper<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;">在GitHub上查看rawtimeit_decorator.py<span style="color: black;">所有</span>代码</p>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;">该修饰器<span style="color: black;">能够</span>应用于功能测试,如下所示:</p>@timeit_wrapperdefexp(x):...print({0:<10}{1:<8}{2:^8}.format(module, function, time))exp(Decimal(150))exp(Decimal(400))exp(Decimal(3000))<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;">在GitHub上查看</p>rawtimeit_decorator_usage.py<span style="color: black;">所有</span>代码
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;">输出如下:</p>~ $ python3.8 slow_program.pymodule function time__main__ .exp :0.003267502994276583__main__ .exp :0.038535295985639095__main__ .exp : 11.728486061969306<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;">在GitHub上查看</p>rawrun_with_timeit_decorator.shell<span style="color: black;">所有</span>代码
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;">要<span style="color: black;">思虑</span>的一个问题是<span style="color: black;">实质</span>/想要<span style="color: black;">测绘</span>的时间类型是什么。Time程序包<span style="color: black;">供给</span>了time.perf_counter和time.process_time。两者的区别是:perf_counter返回绝对值,其中<span style="color: black;">包含</span>Python程序进程未运行时的时间,<span style="color: black;">因此呢</span>可能会受计算机负载的影响;而process_time仅返回用户时间(不<span style="color: black;">包含</span>系统时间),这仅是程序的运行时间。</p>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><strong style="color: blue;">加快程序运行速度</strong></p>
<div style="color: black; text-align: left; margin-bottom: 10px;"><img src="https://p3-sign.toutiaoimg.com/pgc-image/759bdfc2531446f08bda8b11b7466bca~noop.image?_iz=58558&from=article.pc_detail&lk3s=953192f4&x-expires=1723895559&x-signature=gWOC7uba0OaQVXuwz%2BAqJX%2F5pAg%3D" style="width: 50%; margin-bottom: 20px;">
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;">图源:Unsplash</p>
</div>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;">这是全文有趣的部分,关于<span style="color: black;">怎样</span>加快Python的程序运行速度。我并<span style="color: black;">无</span>列出<span style="color: black;">有些</span><span style="color: black;">能够</span><span style="color: black;">奥妙</span><span style="color: black;">处理</span>性能问题的小技巧或代码段,而是<span style="color: black;">触及</span><span style="color: black;">通常</span>性的构想和策略,它们能<span style="color: black;">极重</span>地<span style="color: black;">加强</span>性能,某些<span style="color: black;">状况</span>下<span style="color: black;">乃至</span>能将性能<span style="color: black;">加强</span>30%。</p>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><strong style="color: blue;"><span style="color: black;">运用</span>内置数据类型</strong></p>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;">显而易见,内置数据类型运行<span style="color: black;">火速</span>,尤其是与自定义类型(例如树或链表)相比。<span style="color: black;">重点</span>是<span style="color: black;">由于</span>内置程序是用C语言实现的,远超过用Python编码的运行速度。</p>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><strong style="color: blue;"><span style="color: black;">运用</span>lru_cache缓存/记忆</strong></p>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;">我<span style="color: black;">已然</span>在上一篇博文中讲过这块内容,但<span style="color: black;">这里</span>还是要用简单的示例说明:</p>import functoolsimport time# caching up to 12 different results@functools.lru_cache(maxsize=12)defslow_func(x):time.sleep(2) # Simulate long computationreturn xslow_func(1) # ... waiting for 2 sec before getting resultslow_func(1) # already cached - result returned instantaneously!slow_func(3) # ... waiting for 2 sec before getting result<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;">在GitHub上查看rawlru_cache.py<span style="color: black;">所有</span>代码</p>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;">以上函数<span style="color: black;">运用</span>time.sleep模拟<span style="color: black;">海量</span>运算。<span style="color: black;">第1</span>次<span style="color: black;">运用</span>参数1调用该函数时,返回结果<span style="color: black;">必须</span>2秒。再次调用时,结果已被缓存,<span style="color: black;">因此呢</span>会跳过函数主体并立即返回结果。<span style="color: black;">更加多</span>内容请参见此处。</p>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><strong style="color: blue;"><span style="color: black;">运用</span>局部变量</strong></p>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;">这与在<span style="color: black;">每一个</span><span style="color: black;">功效</span>域中<span style="color: black;">查询</span>变量的速度<span style="color: black;">相关</span>。我用了“<span style="color: black;">每一个</span><span style="color: black;">功效</span>域”这个字眼,<span style="color: black;">由于</span>它<span style="color: black;">不仅</span>是“<span style="color: black;">运用</span>局部变量还是全局变量”的问题。<span style="color: black;">实质</span>上,即使在函数的局部变量(最快)、类级属性(如self.name-较慢)和全局变量(如导入的函数,time.time-最慢)之间,<span style="color: black;">查询</span>速度<span style="color: black;">亦</span>有所<span style="color: black;">区别</span>。</p>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;">能够</span><span style="color: black;">经过</span>运行无用的任务来<span style="color: black;">加强</span>性能,如下所示:</p># Example #1classFastClass:defdo_stuff(self):temp =self.value # this speeds up lookup in loopfor i inrange(10000):... # Do something with `temp` here# Example #2import randomdeffast_function():r = random.randomfor i inrange(10000):print(r()) # calling `r()` here, is faster than global random.random()<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;">在GitHub上查看rawlocal_vars.py<span style="color: black;">所有</span>代码</p>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><strong style="color: blue;"><span style="color: black;">运用</span>函数(Function)</strong></p>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;">这怎么和假想的<span style="color: black;">区别</span>?理论上调用函数不是会将<span style="color: black;">更加多</span>的东西放到堆栈上,加大返回结果的<span style="color: black;">包袱</span>吗?但<span style="color: black;">实质</span>上,<span style="color: black;">运用</span>函数确实能加快运行速度,这与前一点<span style="color: black;">相关</span>。将<span style="color: black;">全部</span>代码放在一个文件中而非函数中,它是全局变量而非局部变量,运行速度就会慢得多。<span style="color: black;">因此呢</span>,<span style="color: black;">能够</span>将<span style="color: black;">全部</span>代码包裹在main函数中并<span style="color: black;">经过</span>一次调用来加速代码,如下所示:</p>defmain():... # All your previously global codemain()<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;">在GitHub上查看rawglobal_vars.py<span style="color: black;">所有</span>代码</p>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><strong style="color: blue;">避免<span style="color: black;">拜访</span>属性(Attribute)</strong></p>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;">可能拖慢程序的一个<span style="color: black;">原由</span>是<span style="color: black;">运用</span>点运算符(.)<span style="color: black;">拜访</span>对象属性。该运算符<span style="color: black;">经过</span><span style="color: black;">运用</span>__getattribute__<span style="color: black;">办法</span>触发了字典<span style="color: black;">查询</span>,使代码产生额外<span style="color: black;">包袱</span>。<span style="color: black;">那样</span>,<span style="color: black;">怎样</span>避免或减少属性<span style="color: black;">拜访</span>?</p># Slow:import redefslow_func():for i inrange(10000):re.findall(regex, line) # Slow!# Fast:from re import findalldeffast_func():for i inrange(10000):findall(regex, line) # Faster!<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;">在GitHub上查看rawimports.py<span style="color: black;">所有</span>代码</p>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><strong style="color: blue;"><span style="color: black;">小心</span><span style="color: black;">运用</span>字符串</strong></p>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;">在循环里<span style="color: black;">运用</span>格式符(%s)或.format()时,字符串操作可能会变得非常慢。有<span style="color: black;">无</span>更好的<span style="color: black;">选取</span>?Raymond Hettinger在<span style="color: black;">近期</span>发布的推文中<span style="color: black;">说到</span>:唯一应该<span style="color: black;">运用</span>的是f-string(格式化字符串常量),它是最易读、最简洁且最快捷的<span style="color: black;">办法</span>。<span style="color: black;">按照</span>这篇推文,下面列出了可用的<span style="color: black;">办法</span>(由快到慢):</p>f{s}{t} # Fast!s + + t .join((s, t))%s %s% (s, t){} {}.format(s, t)Template($s $t).substitute(s=s, t=t) # Slow!<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;">在GitHub上查看rawstrings.py<span style="color: black;">所有</span>代码</p>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;">本质上,生成器并<span style="color: black;">无</span>变得更快,<span style="color: black;">由于</span>它在设计上<span style="color: black;">准许</span>延迟计算以节省内存而非节约时间。然而节省的内存<span style="color: black;">亦</span><span style="color: black;">能够</span>加快程序<span style="color: black;">实质</span>运行速度。怎么做?<span style="color: black;">倘若</span>有一个很大的数据集且不<span style="color: black;">运用</span>生成器(迭代器),<span style="color: black;">那样</span>数据可能会溢出CPU的L1 cache(1级缓存),这将大大减慢内存的<span style="color: black;">查询</span>速度。</p>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;">在性能方面,极重要的一点是:CPU<span style="color: black;">能够</span>将正在处理的所有数据尽可能地<span style="color: black;">保留</span>在缓存中。</p>
<div style="color: black; text-align: left; margin-bottom: 10px;"><img src="https://p3-sign.toutiaoimg.com/pgc-image/1e2f053e110f42f1bfef23a729a7b926~noop.image?_iz=58558&from=article.pc_detail&lk3s=953192f4&x-expires=1723895559&x-signature=j%2BnqZr%2Fry5pnX%2FCxcuNF%2FpfIceA%3D" style="width: 50%; margin-bottom: 20px;">
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;">图源:Unsplash</p>
</div>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><strong style="color: blue;">结语</strong></p>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;">优化的首要规则<span style="color: black;">便是</span>“不优化”。</p>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;">若真的有必要优化,那我<span style="color: black;">期盼</span>这些技巧会有所<span style="color: black;">帮忙</span>。</p>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;">然则</span>,优化代码时<span style="color: black;">必定</span>要小心,<span style="color: black;">由于</span>优化的结果可能是代码难以阅读<span style="color: black;">从而</span>难以<span style="color: black;">守护</span>,这就得不偿失了。</p>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;">最后,<span style="color: black;">期盼</span><span style="color: black;">大众</span>能搭上python号火箭,编码越来越快!</p>
<div style="color: black; text-align: left; margin-bottom: 10px;"><img src="https://p3-sign.toutiaoimg.com/pgc-image/5e60a4b262fe466fa86d827552676705~noop.image?_iz=58558&from=article.pc_detail&lk3s=953192f4&x-expires=1723895559&x-signature=vhCou0AagtMfE6cJ0lA0q4zo%2BWs%3D" style="width: 50%; margin-bottom: 20px;"></div>
<div style="color: black; text-align: left; margin-bottom: 10px;"><img src="https://p3-sign.toutiaoimg.com/pgc-image/c827b3193c28426bb117012ed1834dc6~noop.image?_iz=58558&from=article.pc_detail&lk3s=953192f4&x-expires=1723895559&x-signature=RFdAj9TGgrPbk4MiUISMU9b%2F6dE%3D" style="width: 50%; margin-bottom: 20px;"></div>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><strong style="color: blue;">留言点赞关注</strong></p>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><strong style="color: blue;"><span style="color: black;">咱们</span><span style="color: black;">一块</span>分享AI学习与发展的干货</strong></p>
<p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;">如转载,请后台留言,遵守转载规范</p>
软文发布论坛开幕式圆满成功。 http://www.fok120.com 期待楼主的下一次分享!”
页:
[1]