我在Linux 3.16.0上运行Python 3.4.3。我想subprocess.Popen用一个长参数(复杂的Bash调用)运行一个命令,大约200KiB。
subprocess.Popen
根据getconf和xargs,这应该在我的范围内:
getconf
xargs
$ getconf ARG_MAX 2097152 $ xargs --show-limits < /dev/null Your environment variables take up 3364 bytes POSIX upper limit on argument length (this system): 2091740 POSIX smallest allowable upper limit on argument length (all systems): 4096 Maximum length of command we could actually use: 2088376 Size of command buffer we are actually using: 131072
但是,Python失败的限制很小:
>>> subprocess.Popen('echo %s > /dev/null' % ('a' * (131072-4096)), shell=True, executable='/bin/bash') <subprocess.Popen object at 0x7f4613b58410> >>> subprocess.Popen('echo %s > /dev/null' % ('a' * (262144-4096)), shell=True, executable='/bin/bash') Traceback (most recent call last): [...] OSError: [Errno 7] Argument list too long
请注意,Python限制与“实际使用”命令缓冲区xargs报告大致相同。这表明xargs以某种方式足够聪明,可以从较小的限制开始并根据需要增加它,但Python却没有。
问题:
单个字符串参数的最大大小限制为131072。与python无关:
~$ /bin/echo "$(printf "%*s" 131071 "a")">/dev/null ~$ /bin/echo "$(printf "%*s" 131072 "a")">/dev/null bash: /bin/echo: Argument list too long
实际上,MAX_ARG_STRLEN它决定了单个字符串的最大大小:
MAX_ARG_STRLEN
作为自2.6.23以来的附加限制,一个参数不得超过MAX_ARG_STRLEN(131072)。如果您生成一个长调用,例如“ sh -c’使用长参数生成’”,这可能变得很重要。(由Xan Lopez和Ralf Wildenhues指出)
见的讨论ARG_MAX,“参数数目和一个参数的最大长度”,并在这个问题上unix.stackexchange。
ARG_MAX
unix.stackexchange
您可以在中看到它binfmts.h:
binfmts.h
/* * These are the maximum length and maximum number of strings passed to the * execve() system call. MAX_ARG_STRLEN is essentially random but serves to * prevent the kernel from being unduly impacted by misaddressed pointers. * MAX_ARG_STRINGS is chosen to fit in a signed 32-bit integer. */ #define MAX_ARG_STRLEN (PAGE_SIZE * 32) #define MAX_ARG_STRINGS 0x7FFFFFFF ~$ echo $(( $(getconf PAGE_SIZE)*32 )) 131072
您可以传递多个length的字符串131071:
131071
subprocess.check_call(['echo', "a"*131071,"b"*131071], executable='/bin/bash',stdout=open("/dev/null","w"))
但是单个字符串arg不能超过131071字节。