pytest（py.test）在cygwin中启动非常慢

小编典典

pytest（py.test）在cygwin中启动非常慢

python

在cygwin中，py.test启动非常慢。它看起来不像是集合问题，原因有两个：相同的测试在linux中快速启动。有时，如果在cygwin中足够快地重新运行同一测试，则它会在不到1秒的时间内启动。通过time命令运行时，如果提供该--collection-only选项以避免运行实际测试，则它会在0.4秒或11.7秒内启动。我还为钩子添加了打印件，pytest_configure()并pytest_ignore_collect()确保确实在开始收集之前。

还有其他问题，请链接如何加快py.test等。但是我还没有看到为什么在cygwin下它运行缓慢，以及如何解决该问题。

更新：通过剖析运行它python -m cProfile -s cumulat ~/.../py.test conftest.py。以下是结果中的前20名。我倾向于认为这是posix.statcygwin或cygwin
python包中的，或者在exists和isfile各个地方出现的问题。

   104699 function calls (102659 primitive calls) in 12.223 CPU seconds
   Ordered by: cumulative time
 ncalls  tottime  percall  cumtime  percall filename:lineno(function)
      1    0.016    0.016   12.223   12.223 {execfile}
      1    0.000    0.000   12.223   12.223 <string>:1(<module>)
      1    0.000    0.000   12.207   12.207 py.test:4(<module>)
      1    0.000    0.000   12.051   12.051 config.py:23(main)
  48/22    0.000    0.000   12.051    0.548 core.py:526(_docall)
  48/22    0.000    0.000   12.051    0.548 core.py:520(__call__)
 129/82    0.000    0.000   12.051    0.147 core.py:387(execute)
      1    0.000    0.000   11.926   11.926 config.py:634(pytest_cmdline_parse)
      1    0.000    0.000   11.926   11.926 config.py:70(_prepareconfig)
      1    0.000    0.000   11.926   11.926 config.py:741(parse)
    4/3    0.000    0.000   11.926    3.975 core.py:97(wrapped_call)
    4/3    0.000    0.000   11.926    3.975 core.py:121(__init__)
      1    0.000    0.000   11.911   11.911 config.py:706(_preparse)
     70    0.000    0.000   11.817    0.169 local.py:363(check)
    260   11.817    0.045   11.817    0.045 {posix.stat}  <<<<this one???
      1    0.000    0.000    9.302    9.302 config.py:698(_initini)
      1    0.000    0.000    9.286    9.286 config.py:896(determine_setup)
    188    0.000    0.000    9.286    0.049 genericpath.py:15(exists)  <<<<this one???
     18    0.000    0.000    6.861    0.381 config.py:845(exists)  <<<<this one???
      1    0.000    0.000    6.861    6.861 config.py:851(getcfg)  <<<<this one???
      1    0.000    0.000    2.531    2.531 config.py:694(pytest_load_initial_conftests)
      1    0.000    0.000    2.531    2.531 config.py:477(setinitial)
      1    0.000    0.000    2.531    2.531 config.py:503(_try_load_conftest)
     13    0.000    0.000    2.531    0.195 config.py:511(getconftestmodules)
     32    0.000    0.000    2.531    0.079 genericpath.py:26(isfile)  <<<<this one???
      8    0.000    0.000    2.425    0.303 common.py:261(exists)
      1    0.000    0.000    0.156    0.156 pytest.py:4(<module>)
      1    0.000    0.000    0.125    0.125 main.py:73(wrap_session)
      1    0.000    0.000    0.125    0.125 config.py:615(do_configure)
      1    0.000    0.000    0.125    0.125 main.py:115(pytest_cmdline_main)

有关在分析中进行排序的注释：很难弄清楚要使用哪个关键字。添加一行以打印出sort_arg_defsinpstats.py可以给出一个想法：

$ cat -n /usr/lib/python2.x.x/pstats.py
214         sort_arg_defs = self.get_sort_arg_defs()
215         sort_tuple = ()
216         self.sort_type = ""
217         connector = ""
218         for word in field:
219             sort_tuple = sort_tuple + sort_arg_defs[word][0]
220             self.sort_type += connector + sort_arg_defs[word][1]
221             connector = ", "

结论摘要 ：格式错误的文件路径//setup.py（/setup.py在开始时应没有双斜杠）显示了以下四个软件部分之一中的问题：pytestconfig.determine_setup（），pytest使用的py.path config，cygwin python库或cygwinposix实现。

阅读 212

2021-01-20

共1个答案

小编典典

问题是对于pytest搜索//pytest.ini，//tox.ini，//setup.cfg，和//setup.py。他们每个人造成genericpath.exists()或genericpath.isfile()消耗了大约2.5秒。

解决的办法是在下面添加以下行，genericpath.exists()并genericpath.isfile()跳过这四个特定路径。

if path.startswith(r'//'):
    return False

另一种解决方法是修改_pytest/config.py，使其不会在搜索路径中形成双斜杠。

用于查找确切问题的代码粘贴在下面。进行设置myshow = True以显示要搜索的每个文件的时间消耗。

$ diff -u /usr/lib/python2.6/genericpath.py genericpath.py
--- /usr/lib/python2.6/genericpath.py   2012-06-09 08:33:12.000000000 -0700
+++ genericpath.py      2015-06-11 11:46:33.674285900 -0700
@@ -9,14 +9,29 @@
 __all__ = ['commonprefix', 'exists', 'getatime', 'getctime', 'getmtime',
            'getsize', 'isdir', 'isfile']

+myshow = False
+import time as mytime
+mybasetime = mytime.time()
+def myshowtime():
+    currenttime = mytime.time()
+    tmdiff = currenttime - mybasetime
+    global mybasetime
+    mybasetime = currenttime
+    return tmdiff

 # Does a path exist?
 # This is false for dangling symbolic links on systems that support them.
 def exists(path):
     """Test whether a path exists.  Returns False for broken symbolic links"""
+    pretime = myshowtime()
+    if path.startswith(r'//'):
+        if myshow: print "\n  genericpath exists  %8.3f %8.3f False " % (pretime, myshowtime()), " ", path, "\n"
+        return False
     try:
         st = os.stat(path)
+        if myshow: print "\n  genericpath exists  %8.3f %8.3f True  " % (pretime, myshowtime()), " ", path, "\n"
     except os.error:
+        if myshow: print "\n  genericpath exists  %8.3f %8.3f False " % (pretime, myshowtime()), " ", path, "\n"
         return False
     return True

@@ -25,9 +40,15 @@
 # for the same path ono systems that support symlinks
 def isfile(path):
     """Test whether a path is a regular file"""
+    pretime = myshowtime()
+    if path.startswith(r'//'):
+        if myshow: print "\n  genericpath isfile  %8.3f %8.3f False " % (pretime, myshowtime()), " ", path, "\n"
+        return False
     try:
         st = os.stat(path)
+        if myshow: print "\n  genericpath isfile  %8.3f %8.3f True  " % (pretime, myshowtime()), " ", path, "\n"
     except os.error:
+        if myshow: print "\n  genericpath isfile  %8.3f %8.3f False " % (pretime, myshowtime()), " ", path, "\n"
         return False
     return stat.S_ISREG(st.st_mode)

2021-01-20