Accessing Shadow DOM tree with Selenium

It should also be noted that the Selenium binary Chrome driver now supports Shadow DOM (since Jan 28, 2015) : http://chromedriver.storage.googleapis.com/2.14/notes.txt

This worked for me (using selenium javascript bindings):

driver.executeScript("return $('body /deep/ <#selector>')")

That returns the element(s) you’re looking for.

Normally you’d do this:

element = webdriver.find_element_by_css_selector(
    'my-custom-element /deep/ .this-is-inside-my-custom-element')

And hopefully that’ll continue to work.

However, note that /deep/ and ::shadow are deprecated (and not implemented in browsers other than Opera and Chrome). There’s much talk about allowing them in the static profile. Meaning, querying for them will work, but not styling.

If don’t want to rely on /deep/ or ::shadow because their futures are a bit uncertain, or because you want to work better cross-browser or because you hate deprecation warnings, rejoice as there’s another way:

# Get the shadowRoot of the element you want to intrude in on,
# and then use that as your root selector.
shadow_root = webdriver.execute_script('''
    return document.querySelector(
        'my-custom-element').shadowRoot;
    ''')
element = shadow_root.find_element_by_css_selector(
    '.this-is-inside-my-custom-element')

More about this:

• https://github.com/w3c/webcomponents/issues/78
• https://groups.google.com/a/chromium.org/forum/#!msg/blink-dev/68qSZM5QMRQ/pT2YCqZSomAJ

The accepted answer is no longer valid and some of the other answers have some drawbacks or are not practical (the /deep/ selector doesn’t work and is deprecated, document.querySelector('').shadowRoot works only with the first shadow element when shadow elements are nested), sometimes the shadow root elements are nested and the second shadow root is not visible in document root, but is available in its parent accessed shadow root. I think is better to use the selenium selectors and inject the script just to take the shadow root:

def expand_shadow_element(element):
    shadow_root = driver.execute_script('return arguments[0].shadowRoot', element)
    return shadow_root

outer = expand_shadow_element(driver.find_element_by_css_selector("#test_button"))
inner = outer.find_element_by_id("inner_button")
inner.click()

To put this into perspective I just added a testable example with Chrome’s download page, clicking the search button needs open 3 nested shadow root elements:

import selenium
from selenium import webdriver
driver = webdriver.Chrome()


def expand_shadow_element(element):
    shadow_root = driver.execute_script('return arguments[0].shadowRoot', element)
    return shadow_root

driver.get("chrome://downloads")
root1 = driver.find_element_by_tag_name('downloads-manager')
shadow_root1 = expand_shadow_element(root1)

root2 = shadow_root1.find_element_by_css_selector('downloads-toolbar')
shadow_root2 = expand_shadow_element(root2)

root3 = shadow_root2.find_element_by_css_selector('cr-search-field')
shadow_root3 = expand_shadow_element(root3)

search_button = shadow_root3.find_element_by_css_selector("#search-button")
search_button.click()

Doing the same approach suggested in the other answers has the drawback that it hard-codes the queries, is less readable and you cannot use the intermediary selections for other actions:

search_button = driver.execute_script('return document.querySelector("downloads-manager").shadowRoot.querySelector("downloads-toolbar").shadowRoot.querySelector("cr-search-field").shadowRoot.querySelector("#search-button")')
search_button.click()

I am using C# and Selenium and managed to find an element inside a nestled shadow DOM using java script.
This is my html tree:

html tree

I want the url on the last line and to get it I first select the “downloads-manager” tag and then the first shadow root right below it.
Once inside the shadow root I want to find the element closest to the next shadow root. That element is “downloads-item”. With that selected I can enter the second shadow root. From there I select the img item containing the url by id = “file-icon”. At last I can get the attribute “src” which contains the url I am seeking.

The two lines of C# code that does the trick:

IJavaScriptExecutor jse2 = (IJavaScriptExecutor)_driver;
var pdfUrl = jse2.ExecuteScript("return document.querySelector('downloads-manager').shadowRoot.querySelector('downloads-item').shadowRoot.getElementById('file-icon').getAttribute('src')");

I found a much easier way to get the elements from Shadow Dom.
I am taking the same example given above, for search icon of Chrome Download Page.

IWebDriver driver;

public IWebElement getUIObject(params By[] shadowRoots)
        {
            IWebElement currentElement = null;
            IWebElement parentElement = null;
            int i = 0;
            foreach (var item in shadowRoots)
            {
                if (parentElement == null)
                {
                    currentElement = driver.FindElement(item);
                }
                else
                {
                    currentElement = parentElement.FindElement(item);
                }
                if(i !=(shadowRoots.Length-1))
                {
                    parentElement = expandRootElement(currentElement);
                }
                i++;
            }
            return currentElement;
        }

 public IWebElement expandRootElement(IWebElement element)
        {
            IWebElement rootElement = (IWebElement)((IJavaScriptExecutor)driver)
        .ExecuteScript("return arguments[0].shadowRoot", element);
            return rootElement;
        }

Google Chrome Download Page

Now as shown in image we have to expand three shadow root elements in order to get our search icon.
To to click on icon all we need to do is :-

  [TestMethod]
        public void test()
        {
           IWebElement searchButton= getUIObject(By.CssSelector("downloads-manager"),By.CssSelector("downloads-toolbar"),By.Id("search-input"),By.Id("search-buton"));
            searchButton.Click();
        }

So just one line will give you your Web Element, just need to make sure you pass first shadow root element as first argument of the function “getUIObject” second shadow root element as second argument of the function and so on, finally last argument for the function will be the identifier for your actual element (for this case its ‘search-button’)

Until Selenium supports shadow DOM out of the box, you can try the following workaround in Java. Create a class that extends By class:

import org.openqa.selenium.By;
import org.openqa.selenium.JavascriptExecutor;
import org.openqa.selenium.SearchContext;
import org.openqa.selenium.WebDriverException;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.WrapsDriver;
import org.openqa.selenium.internal.FindsByCssSelector;

import java.io.Serializable;
import java.util.List;

public class ByShadow {
    public static By css(String selector) {
        return new ByShadowCss(selector);
    }

    public static class ByShadowCss extends By implements Serializable {

        private static final long serialVersionUID = -1230258723099459239L;

        private final String cssSelector;

        public ByShadowCss(String cssSelector) {
            if (cssSelector == null) {
                throw new IllegalArgumentException("Cannot find elements when the selector is null");
            }
            this.cssSelector = cssSelector;
        }

        @Override
        public WebElement findElement(SearchContext context) {
            if (context instanceof FindsByCssSelector) {
                JavascriptExecutor jsExecutor;
                if (context instanceof JavascriptExecutor) {
                    jsExecutor = (JavascriptExecutor) context;
                } else {
                    jsExecutor = (JavascriptExecutor) ((WrapsDriver) context).getWrappedDriver();
                }
                String[] subSelectors = cssSelector.split(">>>");
                FindsByCssSelector currentContext = (FindsByCssSelector) context;
                WebElement result = null;
                for (String subSelector : subSelectors) {
                    result = currentContext.findElementByCssSelector(subSelector);
                    currentContext = (FindsByCssSelector) jsExecutor.executeScript("return arguments[0].shadowRoot", result);
                }
                return result;
            }

            throw new WebDriverException(
                    "Driver does not support finding an element by selector: " + cssSelector);
        }

        @Override
        public List<WebElement> findElements(SearchContext context) {
            if (context instanceof FindsByCssSelector) {
                JavascriptExecutor jsExecutor;
                if (context instanceof JavascriptExecutor) {
                    jsExecutor = (JavascriptExecutor) context;
                } else {
                    jsExecutor = (JavascriptExecutor) ((WrapsDriver) context).getWrappedDriver();
                }
                String[] subSelectors = cssSelector.split(">>>");
                FindsByCssSelector currentContext = (FindsByCssSelector) context;
                for (int i = 0; i < subSelectors.length - 1; i++) {
                    WebElement nextRoot = currentContext.findElementByCssSelector(subSelectors[i]);
                    currentContext = (FindsByCssSelector) jsExecutor.executeScript("return arguments[0].shadowRoot", nextRoot);
                }
                return currentContext.findElementsByCssSelector(subSelectors[subSelectors.length - 1]);
            }

            throw new WebDriverException(
                    "Driver does not support finding elements by selector: " + cssSelector);
        }

        @Override
        public String toString() {
            return "By.cssSelector: " + cssSelector;
        }
    }
}

And you can use it without writing any additional functions or wrappers. This should work with any kind of framework. For example, in pure Selenium code this would look like this:

WebElement searchButton =
    driver.findElement(ByShadow.css(
        "downloads-manager >>> downloads-toolbar >>> cr-search-field >>> #search-button"));

or if you use Selenide:

SelenideElement searchButton =
    $(ByShadow.css("downloads-manager >>> downloads-toolbar >>> cr-search-field >>> #search-button"));

For getting the filename of the latest downloaded file in Chrome

def get_downloaded_file(self):
  filename = self._driver.execute_script("return document.querySelector('downloads-manager').shadowRoot.querySelector('#downloadsList downloads-item').shadowRoot.querySelector('div#content  #file-link').text")
  return filename

Usage:

driver.get_url('chrome://downloads')
filename = driver.get_downloaded_file()

And for configuring the option for setting the default download directory in selenium for chrome browser, where the corresponding file could be gotten:

..
chrome_options = webdriver.ChromeOptions()
..
prefs = {'download.default_directory': '/desired-path-to-directory'} # unix
chrome_options.add_experimental_option('prefs', prefs)
..