Crawl by many keywords in python

Question

I’m doing a project about crawling website under specific keywords in python.
My code (below) only can handle one keyword at onece.How do I fix it to handle many keywords?

like : keywordlist = ["worm", "inflammation", "fever"]

I want print every result after search.
Thank you for any responses in advance!

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.common.keys import Keys

browser = webdriver.Chrome()

browser.get('https://pubmed.ncbi.nlm.nih.gov/')

print("Enter a keyword to search: ", end='')
keyword = input()

# HTML located tags 
elem = browser.find_element(By.ID, 'id_term')  # by_id find the search box
elem.send_keys(keyword + Keys.ENTER) # input keyword 

print("searchQuery : ", browser.title)
print(browser.current_url) # get url of results


browser.quit()

Asked By: user20137369

||

Source

Answer 1

If you want it to handle more than one search, try putting your existing code into a function with a for loop connected to the number of keywords inputted.

Maybe something like this:

keyword = input("Enter a keyword(s): ")
keywords = keyword.split(" ")

if len(keywords) == 1:
# Your code

    elem = browser.find_element(By.ID, 'id_term')  # by_id find the search box
    elem.send_keys(keyword + Keys.ENTER) # input keyword 

    print("searchQuery : ", browser.title)
    print(browser.current_url) # get url of results

elif len(keywords) > 1:
# Your code in a for loop
    for x in len(keywords):
        elem = browser.find_element(By.ID, 'id_term')  # by_id find the search box
        elem.send_keys(keyword + Keys.ENTER) # input keyword 

        print("searchQuery : ", browser.title)
        print(browser.current_url) # get url of results

Hope this helps!

Answered By: SmokeAndMirrors

Crawl by many keywords in python

Question:

Answers: