How to get page html code using selenium?

Question

I am trying to parse a cloudflare website using selenium. I can find individual elements on the page, but I did not find how to get the entire code of the page.

options = webdriver.ChromeOptions()
options.add_argument('user-agent=')
options.add_argument('--disable-blink-features=AutomationControlled')

s = Service(executable_path='')
driver = webdriver.Chrome(service=s, options=options)

try:
    driver.get('https://mangalib.me/manga-list')
    time.sleep(10)
    print(driver.find_element_by_xpath(''))
except Exception as ex:
    print(ex)
finally:
    driver.close()
    driver.quit()

Asked By: lian. lun

||

Source

Answer 1

to get the entire source code you just do:

driver.get('https://mangalib.me/manga-list')
html = driver.page_source

then you can do whatever you want with it

Answered By: CDJuaum

Answer 2

If you are using java

In webdriver interface "getpagesource()" is a method present there.
Below this code snippet I mentioned. It will give you the total html code of the webpage.

public void getPgSource() {
    WebDriverManager.chromedriver().setup();
    ChromeDriver driver = new ChromeDriver();
    driver.get("https://www.tutorialspoint.com/java/java_basic_syntax.htm");
    String s = driver.getPageSource();
    System.out.println(s);
}

This method returns the string data type so you can use accordingly.

Answered By: Satyajit Behera

How to get page html code using selenium?

Question:

Answers: