我想使用AWS Lambda功能生成视频。
AWS Lambda
我已按照此处和此处的说明进行操作。
现在,我可以通过以下过程来构建Lambda函数:
Lambda
触发Amazon Linux EC2实例并以root身份在其上运行:
Amazon Linux EC2
#! /usr/bin/env bash # Install the SciPy stack on Amazon Linux and prepare it for AWS Lambda yum -y update yum -y groupinstall "Development Tools" yum -y install blas --enablerepo=epel yum -y install lapack --enablerepo=epel yum -y install atlas-sse3-devel --enablerepo=epel yum -y install Cython --enablerepo=epel yum -y install python27 yum -y install python27-numpy.x86_64 yum -y install python27-numpy-f2py.x86_64 yum -y install python27-scipy.x86_64 /usr/local/bin/pip install --upgrade pip mkdir -p /home/ec2-user/stack /usr/local/bin/pip install moviepy -t /home/ec2-user/stack cp -R /usr/lib64/python2.7/dist-packages/numpy /home/ec2-user/stack/numpy cp -R /usr/lib64/python2.7/dist-packages/scipy /home/ec2-user/stack/scipy tar -czvf stack.tgz /home/ec2-user/stack/*
我将生成的tarball压缩到笔记本电脑中。然后运行此脚本以构建一个zip存档。
#! /usr/bin/env bash mkdir tmp rm lambda.zip tar -xzf stack.tgz -C tmp zip -9 lambda.zip process_movie.py zip -r9 lambda.zip *.ttf cd tmp/home/ec2-user/stack/ zip -r9 ../../../../lambda.zip *
process_movie.py 目前,脚本仅是测试以查看堆栈是否正常的测试:
process_movie.py
def make_movie(event, context): import os print(os.listdir('.')) print(os.listdir('numpy')) try: import scipy except ImportError: print('can not import scipy') try: import numpy except ImportError: print('can not import numpy') try: import moviepy except ImportError: print('can not import moviepy')
然后,我将生成的归档文件上传到S3作为我的lambda函数的来源。当我测试功能时,我得到以下信息callstack:
lambda
callstack
START RequestId: 36c62b93-b94f-11e5-9da7-83f24fc4b7ca Version: $LATEST ['tqdm', 'imageio-1.4.egg-info', 'decorator.pyc', 'process_movie.py', 'decorator-4.0.6.dist-info', 'imageio', 'moviepy', 'tqdm-3.4.0.dist-info', 'scipy', 'numpy', 'OpenSans-Regular.ttf', 'decorator.py', 'moviepy-0.2.2.11.egg-info'] ['add_newdocs.pyo', 'numarray', '__init__.py', '__config__.pyc', '_import_tools.py', 'setup.pyo', '_import_tools.pyc', 'doc', 'setupscons.py', '__init__.pyc', 'setup.py', 'version.py', 'add_newdocs.py', 'random', 'dual.pyo', 'version.pyo', 'ctypeslib.pyc', 'version.pyc', 'testing', 'dual.pyc', 'polynomial', '__config__.pyo', 'f2py', 'core', 'linalg', 'distutils', 'matlib.pyo', 'tests', 'matlib.pyc', 'setupscons.pyc', 'setup.pyc', 'ctypeslib.py', 'numpy', '__config__.py', 'matrixlib', 'dual.py', 'lib', 'ma', '_import_tools.pyo', 'ctypeslib.pyo', 'add_newdocs.pyc', 'fft', 'matlib.py', 'setupscons.pyo', '__init__.pyo', 'oldnumeric', 'compat'] can not import scipy 'module' object has no attribute 'core': AttributeError Traceback (most recent call last): File "/var/task/process_movie.py", line 91, in make_movie import numpy File "/var/task/numpy/__init__.py", line 122, in <module> from numpy.__config__ import show as show_config File "/var/task/numpy/numpy/__init__.py", line 137, in <module> import add_newdocs File "/var/task/numpy/numpy/add_newdocs.py", line 9, in <module> from numpy.lib import add_newdoc File "/var/task/numpy/lib/__init__.py", line 13, in <module> from polynomial import * File "/var/task/numpy/lib/polynomial.py", line 11, in <module> import numpy.core.numeric as NX AttributeError: 'module' object has no attribute 'core' END RequestId: 36c62b93-b94f-11e5-9da7-83f24fc4b7ca REPORT RequestId: 36c62b93-b94f-11e5-9da7-83f24fc4b7ca Duration: 112.49 ms Billed Duration: 200 ms Memory Size: 1536 MB Max Memory Used: 14 MB
我不明白为什么python找不到文件夹结构中存在的核心目录。
编辑:
遵循@jarmod的建议,我将该lambda函数简化为:
def make_movie(event, context): print('running make movie') import numpy
我现在有以下错误:
START RequestId: 6abd7ef6-b9de-11e5-8aee-918ac0a06113 Version: $LATEST running make movie Error importing numpy: you should not try to import numpy from its source directory; please exit the numpy source tree, and relaunch your python intepreter from there.: ImportError Traceback (most recent call last): File "/var/task/process_movie.py", line 3, in make_movie import numpy File "/var/task/numpy/__init__.py", line 127, in <module> raise ImportError(msg) ImportError: Error importing numpy: you should not try to import numpy from its source directory; please exit the numpy source tree, and relaunch your python intepreter from there. END RequestId: 6abd7ef6-b9de-11e5-8aee-918ac0a06113 REPORT RequestId: 6abd7ef6-b9de-11e5-8aee-918ac0a06113 Duration: 105.95 ms Billed Duration: 200 ms Memory Size: 1536 MB Max Memory Used: 14 MB
在此线程中的所有帖子的帮助下,这里是记录的解决方案:
要使其正常工作,您需要:
EC2用至少2GO RAM启动一个实例(以便能够编译NumPy&SciPy)
EC2
NumPy
SciPy
安装所需的依赖项
sudo yum -y update
sudo yum -y upgrade sudo yum -y groupinstall “Development Tools” sudo yum -y install blas –enablerepo=epel sudo yum -y install lapack –enablerepo=epel sudo yum -y install Cython –enablerepo=epel sudo yum install python27-devel python27-pip gcc virtualenv ~/env source ~/env/bin/activate pip install scipy pip install numpy pip install moviepy
将目录中所有目录的内容(_markerlib,pip ,pkg_resources,setuptools 和easyinstall *除外)复制到您的语言环境计算机中stack:
stack
home/ec2-user/env/lib/python2.7/dist-packages
home/ec2-user/env/lib64/python2.7/dist-packages
从您的EC2实例获取所有必需的共享库:
libatlas.so.3
libf77blas.so.3
liblapack.so.3
libptf77blas.so.3
libcblas.so.3
libgfortran.so.3
libptcblas.so.3
libquadmath.so.0
lib
imageio是的依赖项moviepy,您需要下载其依赖项的一些二进制版本:libfreeimage和ffmpeg;他们可以在这里找到。将它们放在堆栈文件夹的根目录并重命名libfreeimage-3.16.0-linux64.so为libfreeimage.so
imageio
moviepy
libfreeimage
ffmpeg
libfreeimage-3.16.0-linux64.so
libfreeimage.so
您现在应该拥有一个stack包含以下内容的文件夹:
zip -r9 stack.zip . -x ".*" -x "*/.*"
使用以下内容lambda_function.py作为您的入口点lambda
lambda_function.py
from __future__ import print_function
import os import subprocess
SCRIPT_DIR = os.path.dirname(os.path.abspath(file)) LIB_DIR = os.path.join(SCRIPT_DIR, ‘lib’) FFMPEG_BINARY = os.path.join(SCRIPT_DIR, ‘ffmpeg’)
def lambda_handler(event, context): command = ‘LD_LIBRARY_PATH={} IMAGEIO_FFMPEG_EXE={} python movie_maker.py’.format( LIB_DIR, FFMPEG_BINARY, ) try: output = subprocess.check_output(command, shell=True) print(output) except subprocess.CalledProcessError as e: print(e.output)
写一个movie_maker.py依赖于脚本moviepy,numpy…
movie_maker.py
numpy
将这些脚本添加到您的stack.zip文件中 zip -r9 lambda.zip *.py
zip -r9 lambda.zip *.py
上载zipS3并将其用作您的源lambda
S3
您也可以在stack.zip 此处下载。
stack.zip