the script running not return consistent data run run. believe iterate through pages, not waiting pages load javascript , ajax in full. in attempt remedy added explicit wait below, returns following error:
connectionrefusederror: [winerror 10061] no connection made because target machine actively refused it
from selenium import webdriver selenium.webdriver.common.by import selenium.webdriver.support.ui import webdriverwait selenium.webdriver.support import expected_conditions ec browser = webdriver.chrome() url = 'http://www.website.com' browser.get(url) try: element = webdriverwait(browser, 10).until(ec.presence_of_all_elements_located((by.id, "results-main"))) finally: browser.quit() print (browser.page_source)
an implicit wait run without errors, not return consistent source code.
im assuming getting exception @ browser.page_source because doing browser.quit quit() - used shut down web driver instance or destroy web driver
you need page source before quit either adding print statement @ end of try block or moving quit after print :)
Comments
Post a Comment