我正在处理一个大型文本文件(500k行),其格式如下:
S1_A16 0.141,0.009340221649748676 0.141,4.192618196894668E-5 0.11,0.014122135626540204 S1_A17 0.188,2.3292323316081486E-6 0.469,0.007928706856794138 0.172,3.726771730573038E-5
我正在使用下面的代码返回每个系列的相关系数,例如S!_A16:
import numpy as np import pandas as pd import csv pd.options.display.max_rows = None fileName = 'wordUnigramPauseTEST.data' df = pd.read_csv(fileName, names=['pause', 'probability']) mask = df['pause'].str.match('^S\d+_A\d+') df['S/A'] = (df['pause'] .where(mask, np.nan) .fillna(method='ffill')) df = df.loc[~mask] result = df.groupby(['S/A']).apply(lambda grp: grp['pause'].corr(grp['probability'])) print(result)
但是,在一些大文件上,这将返回错误:
Traceback (most recent call last): File "/Users/adamg/PycharmProjects/Subj_AnswerCorrCoef/GetCorrCoef.py", line 15, in <module> print(result) File "/Users/adamg/anaconda/lib/python2.7/site-packages/pandas/core/base.py", line 35, in __str__ return self.__bytes__() File "/Users/adamg/anaconda/lib/python2.7/site-packages/pandas/core/base.py", line 47, in __bytes__ return self.__unicode__().encode(encoding, 'replace') File "/Users/adamg/anaconda/lib/python2.7/site-packages/pandas/core/series.py", line 857, in __unicode__ result = self._tidy_repr(min(30, max_rows - 4)) TypeError: unsupported operand type(s) for -: 'NoneType' and 'int'
我了解这与该print声明有关,但是如何解决?
print
编辑 :这与最大行数有关。有谁知道如何容纳更多的行?
错误信息:
TypeError: unsupported operand type(s) for -: 'NoneType' and 'int'
说None负一个int是TypeError。如果查看回溯中的倒数第二行,您会发现唯一的减法是
None
int
max_rows - 4
所以max_rows一定是None。如果您进入/Users/adamg/anaconda/lib/python2.7/site- packages/pandas/core/series.py857号线附近,问自己如何max_rows最终等于None,您会以某种方式看到
max_rows
/Users/adamg/anaconda/lib/python2.7/site- packages/pandas/core/series.py
get_option("display.max_rows")
一定在返回None。
这部分代码正在调用_tidy_repr,用于总结系列。None是正确的值来设定的,当你想熊猫来显示 所有 的线Series。因此,如果max_rows为None ,则不应到达代码的这一部分。
_tidy_repr
Series
我已提出请求以更正此问题。