我正在尝试检索具有ajax加载向下滚动功能alla Twitter的页面中的元素。由于某些原因,此功能无法正常运行。我添加了一些打印语句以对其进行调试,并且我总是得到相同数量的项目,然后函数返回。我在这里做错了什么?
wd = webdriver.Firefox() wd.implicitly_wait(3) def get_items(items): print len(items) wd.execute_script("window.scrollTo(0, document.body.scrollHeight);") # len(items) and len(wd.find_elements-by...()) both always seem to return the same number # if I were to start the loop with while True: it would work, but of course... never end while len(wd.find_elements_by_class_name('stream-item')) > len(items): items = wd.find_elements_by_class_name('stream-item') print items wd.execute_script("window.scrollTo(0, document.body.scrollHeight);") return items def test(): get_page('http://twitter.com/') get_items(wd.find_elements_by_class_name('stream-item'))
尝试介于两者之间
wd = webdriver.Firefox() wd.implicitly_wait(3) def get_items(items): print len(items) wd.execute_script("window.scrollTo(0, document.body.scrollHeight);") # len(items) and len(wd.find_elements-by...()) both always seem to return the same number # if I were to start the loop with while True: it would work, but of course... never end sleep(5) #seconds while len(wd.find_elements_by_class_name('stream-item')) > len(items): items = wd.find_elements_by_class_name('stream-item') print items wd.execute_script("window.scrollTo(0, document.body.scrollHeight);") return items def test(): get_page('http://twitter.com/') get_items(wd.find_elements_by_class_name('stream-item'))
注意:硬睡眠只是为了证明它有效。请改用waits包来等待智能状态。