Slow scrolling down the page using Selenium

After some experiments, finally I found a good solution:

    def __scroll_down_page(self, speed=8):
    current_scroll_position, new_height= 0, 1
    while current_scroll_position <= new_height:
        current_scroll_position += speed
        self.__driver.execute_script("window.scrollTo(0, {});".format(current_scroll_position))
        new_height = self.__driver.execute_script("return document.body.scrollHeight")

I needed it for the same issue , i needed to scrape a social media website

y = 1000
    for timer in range(0,50):
         driver.execute_script("window.scrollTo(0, "+str(y)+")")
         y += 1000  
         time.sleep(1)

the sleep every 1000 is to allow to load

You can do smooth scrolling using Selenium like below:

total_height = int(driver.execute_script("return document.body.scrollHeight"))

for i in range(1, total_height, 5):
    driver.execute_script("window.scrollTo(0, {});".format(i))

from selenium import webdriver
import time

driver = webdriver.Chrome()
driver.get("https://en.wikipedia.org")
height = browser.execute_script("return document.body.scrollHeight")
for scrol in range(100,height,100):
    browser.execute_script(f"window.scrollTo(0,{scrol})")
    time.sleep(0.1)

It’s worked for me. If you want to scroll the page till the end to visible all the page elements may it’s valuable for you.
if you want to increase the scroll speed change the scroll speed just change 100 to 200.

In Python Selenium, get Y position of your element, and then slowly scroll down.

y = driver.execute_script("return document.querySelector('YOUR-CSS-SELECTOR').getBoundingClientRect()['y']")
for x in range(0, int(y), 100):
    driver.execute_script("window.scrollTo(0, "+str(x)+");")

@Test
public void clickMeButton() {
   WebElement clickMeButton = driver.findElement(By.name("et_builder_submit_button"));

   // softly scroll to the element group
   JavascriptExecutor js = (JavascriptExecutor) driver;

   for (int i = 0; i < 800; i += 7) {
      js.executeScript("window.scrollTo(0, " + i + ")");
   }

   System.out.println("selected button");
   clickMeButton.click();
   System.out.println("clicked");
}

Assuming the page is being lazy loaded, jumping to points iteratively with a window.scrollTo() could possibly leave out elements that fall out of the range. Also, assuming the height of the page was dynamic pertaining to the content loaded, items that haven’t been loaded yet could be left out. So I opted for window.scrollBy() instead:

height = driver.execute_script("return document.body.scrollHeight")
for i in range(height):
   driver.execute_script('window.scrollBy(0,20)') # scroll by 20 on each iteration
   height = driver.execute_script("return document.body.scrollHeight") # reset height to the new height after scroll-triggered elements have been loaded. 

In my case, a new batch of elements was loaded after scrolling to the 30th, giving the page a new height I needed to account for.

time.sleep() make the program slower not good for production

This is a more efficient and controlled way to scroll down to the bottom.
Use the below function written by me.
if you increase the  increment value  a+=5 scroll speed become higher (Warning Not more than new_height) and vice versa

def pageBottom(driver):
    bottom=False
    a=0
    while not bottom:
        new_height = driver.execute_script("return document.body.scrollHeight")
        driver.execute_script(f"window.scrollTo(0, {a});")
        if a > new_height:
            bottom=True
        a+=5

Example Usage:

service = Service("chromedriver.exe")
driver = webdriver.Chrome(service=service)
driver.get("https://docs.scrapy.org/en/latest/intro/tutorial.html")
pageBottom(driver) #<---Go to Bottom