Forem: jainishpatel78

Use Cases of XPath Python In Selenium With Examples

jainishpatel78 — Mon, 13 May 2024 12:06:16 +0000

Despite having a variety of programming languages to choose from, like JavaScript, Java, Typescript, C#, Ruby, etc., Python is considered the easiest to learn due to its easy-to-learn and understandable syntax.

It has various use cases, web apps using Flask and Django, desktop apps using Tkinter, and libraries like Scikit-learn and TensorFlow for implementing AI and ML models. Testing is also a domain where Python has a stronghold with support from various testing frameworks like Selenium, Appium, etc.

During testing, locating specific elements plays pivotal to recognize DOM manipulation. This manipulation allows interaction with the web elements within the user interface. This is where Python **XPath **comes into play, offering a powerful toolset for traversing the Document Object Model and pinpointing elements efficiently.

In this article, we will cover everything, starting from what XPath Python is, how to install and use XPath Python, extraction of text from HTML, types of XPath, Using Python with various functions and mathematical operators, and some common advanced techniques.

Need to extract text from HTML code? Our free online tool makes it easy and fast. Save time by extracting only the text you need from HTML code in no time.

What is XPath and Why is it used with Selenium?

Automation testing is a viable substitute for manual testing due to the expanding variety of browser types like Chrome, Firefox, Safari, Microsoft Edge, etc., and their multiple versions. Selenium WebDriver is an open-source application programming interface (API) used to automate the testing of web applications.

In Selenium, XPath uses an XML path to go through a web page’s HTML code. You can refer to the following video on our YouTube Channel for an in-depth tutorial on Selenium WebDriver.

XPath refers to locating components and attributes inside an HTML / XML document. Similar to how file systems utilize paths to explore directories and files, XPath uses path expressions to choose specific nodes or groups of nodes within an HTML document. XPath is recommended when other CSS locators (ID, Class, etc.) are absent from an XML/HTML page.

Choosing the right element locator is a critical aspect of effective test automation. It’s not just about functionality; it’s about efficiency and accuracy.

To understand the preferences of the testing community, we conducted a social media poll with the question, “What is your favorite element locator in test automation?” The responses offer valuable insights into the most favored locator strategies.

Source

A considerable percentage of participants prefer XPath as their primary element locator due to its versatility and capability, making it a prominent selection for complex web elements.

XPath can be used for various purposes apart from Automation testing. Different use cases of it will be covered in the upcoming sections. First, let’s learn how to use Python XPath with Selenium and then with other libraries like requests.

Extract text from JSON data with our powerful and easy-to-use online free tool. Get the data you need quickly and efficiently, no technical skills required.

How to Install and Setup Python XPath

To use Python XPath with Selenium, you are only required to install and run Selenium correctly, and to use Xpath with other libraries, you need the lxml library.

Install Selenium

1.Install Python 3 using the installer, which can be downloaded from the official website; follow the instructions and set up Python for your system(A prerequisite).

Now open the command prompt or terminal and run the following command:

python --version

Console Output:

3.Now open your preferred IDE or editor and set up the *venv *(Virtual Environment) if you are using various versions of Python. This article uses VS Code.

4.Open a VS Code Terminal and run the following command to install Selenium. The version of Selenium used in this article is 4.10.0. Check our blog on Selenium 4 to know more.

pip install selenium

Get plain text from XML documents. Simply copy and paste your XML data to extract text using our online free Extract Text from XML tool. Give it a try now!

Install lxml Library

A robust and effective Python library for handling XML and HTML documents is the lxml. It offers a Python interface to the C-written libxml2 and libxslt libraries. With the aid of lxml, you may traverse and modify the structures of XML and HTML documents as well as create new ones.

To install the lxml library run the following command:

pip install lxml

Verifying the Selenium Installation

By running the following script you can make sure that everything is working fine. The below code will open the LambdaTest E-commerce playground, a demo website and get its title (Your Store) printed.

from selenium import webdriver

driver = webdriver.Chrome()

driver.get("https://ecommerce-playground.lambdatest.io/")

# Print the title of the website 
print(driver.title)
driver.close()

Console Output:

Finding an Element Using Python XPath in Selenium

Now that we are done with the installation part, let’s find an element using XPath Python in Selenium.

To find the location of a specific element in a document, you need to obtain the XPath of that element. For that, follow the steps:

1.Inspect the website by right-clicking > Inspect or (ctrl + Shift + C or cmd + Option + C).

2.Select the element to see the HTML code. Now right-click on the code and Go to the copy option where you can find the XPath of that element.

XPath will look something like this:

//*[@id=”table-id”]/tbody/tr[1]

Strip HTML is a free online tool that strips out tags and other formatting from text. Simply paste any text into the tool and click on Strip HTML to remove unnecessary formatting.

Using Python XPath to Locate Elements

To locate an element, the find_element() method is used. It returns the first matching element in the HTML document. To illustrate this, we will find the element by the XPath which we copied.

from selenium import webdriver
from selenium.webdriver.common.by import By

driver = webdriver.Chrome()

driver.get("https://www.lambdatest.com/selenium-playground/table-pagination-demo")

row_data = driver.find_element(By.XPATH,'//*[@id="table-id"]/tbody/tr[1]')
print("Row 1: ", row_data.text)
driver.close()

Console Output:

In the XPath //[@id=”table-id”]/tbody/tr[1], **tr[1]* represents the first row of the table; However, If you change the XPath to //[@id=”table-id”]/tbody/tr* still the output remains the same because the find_element() method will only get the first appearance of the XPath from the HTML Document.

To get all the elements, you don’t have to mention each; rather, use the find_elements() method to get all the elements with the same XPath.

This is how you can use find_elements() in the code:

from selenium import webdriver
from selenium.webdriver.common.by import By

driver = driver = webdriver.Chrome()

driver.get("https://www.lambdatest.com/selenium-playground/table-pagination-demo")

rows = driver.find_elements(By.XPATH,'//*[@id="table-id"]/tbody/tr')
for row in rows:
    print(row.text)
driver.close()

Following XPath is used in the code:

Console Output:

JSON Validator is a free and easy-to-use tool to validate JSON data, which makes it easier to fix the issue by providing error line and character position.

Retrieving attribute values

The get_attribute() method is used to retrieve an element’s attributes, such as the *href *attribute from an anchor tag. This function will initially attempt to return the value of the specified property. It returns the value of the corresponding attribute if the requested property doesn’t exist. *None *is returned if there isn’t an attribute with that name. You can check out the list of attributes that can be fetched.

Code on how to use get_attribute():

table = driver.find_element(By.XPATH,'//*[@id="table-id"]')
print(table.get_attribute("class"))

Following XPath is used in the code:

Console Output:

HTML Validator is an online tool that checks HTML syntax and quality. Improve your website’s performance and functionality with ease.

Extracting Text Content and Manipulating Using Python

You can extract text from the element using the text property. Apart from this, you can use various methods of Python like split(), sort(), reverse(), replace(), and many more.

The code below uses the split() method to organize the data.
This is how you can use it in the code:

rows = driver.find_elements(By.XPATH,'//*[@id="table-id"]/tbody/tr')
data = []
for row in rows:
    row_data = row.text.split(" ")
    if len(row_data) > 1:
        data.append(row_data)

for data_row in data:
    print(data_row)

Following XPath is used in the code:

Console Output:

Handling multiple matches and filtering results

You can filter the results by adding more precise conditions using XPath. For instance, the following XPath phrase may be used to discover all div elements that belong to a particular class:

divs = driver.find_elements(By.XPATH,'//*[@id="header"]/nav/div/div/div[2]/div/div/div[1]/div')

for div in divs:
    print(div.text)

XPath using in the code:

Console Output:

To verify the fetched data is correct, you can check by finding the element. To do so, follow this step:

Open Developer Tools using the Inspect option.
In the Elements panel, press Ctrl+F or cmd+F.
Add the XPath in the Find Box and you will get the number of appearings.

//*[@id="header"]/nav/div/div/div[2]/div/div/div[1]/div

Refer to the below image for more understanding.

Minify your JS code without changing its functionality with our easy-to-use JS Minifier that reduces the size of your scripts and improve website speed.

Additionally, you may traverse and filter elements based on how they relate to other items using XPath axes and predicates. For instance, the following XPath phrase can be used to locate every element included within the button’s particular div element:

buttons = driver.find_elements(By.XPATH,'//*[@id="header"]/nav/div/div/div[2]/div/div/div[1]/div/button')
for button in buttons:
print(button.text)

XPath using in the code:

Console Output:

These examples show how to use XPath in Selenium with Python at its most basic level. You can modify the XPath expressions based on the structure and properties of the HTML or XML document you are working with.

Types of XPath in Selenium

In XPath, there are two methods for locating an element: absolute and relative. The XPath node, which can either be the starting point of the path to the node where the element is situated or a point relative to some other point along the DOM, defines the paths.

Absolute XPath

Absolute XPath begins at the document’s root node and descends through the element hierarchy until it reaches the target element.
It is less dependable if the page’s structure changes because it uses the entire path starting with the root node.
A single forward slash (/) marks the beginning of an absolute XPath expression.
To get the Absolute XPath copy full Xpath from the **copy **option.

Example:

/html/body/div[1]/div/section[2]/div/div/div/table

Minimize the file size of your CSS files for faster loading web pages with our free online CSS Minify tool. Easy and efficient minification in just a few clicks.

Relative XPath

Relative XPath employs a reference point to find the target element after starting from any node in the document.
It is more flexible and advised because it considers the element’s position relative to its parent or sibling components.
Expressions in XPath that are relative start with a double forward slash (//).
To get the Absolute XPath to copy Xpath from the **copy **option.

Example:

//*[@id=”table-id”]

Relative XPath expressions are commonly used in Selenium due to their flexibility and robustness. They allow you to locate elements based on their attributes, text content, or their relationship with other elements. Additionally, you can leverage various XPath axes, functions, and predicates to create more specific and powerful expressions for element locating.

Breakdown of XPath Expressions

XPath is made of various symbols. Each one has its significance and can be used in different ways. The following table will explain each symbol in detail:

XPath Example Syntax:

Using Python XPath with Functions, Mathematical and logical Operators

Using functions, mathematical operators, and logical operators with Python XPath, you may build strong and adaptable expressions to pick out particular web page items. In this section, let’s look at some examples of how to combine these features:

Functions in Python XPath

Various functions in XPath Python can be used for Strings, Numbers, Nodes, and Boolean. In this section, we will look at various functions for each of them.

String Functions

You can carry out a variety of operations on components using XPath functions, such as extracting attributes and counting items. The following are a few often-used functions: text(), contains(), starts-with(), concat(), etc. Let’s look at each function in detail.

text():

This function helps to extract text from an HTML/XML element. It can be used when you want the test from tags like a, p, h1 to h6, and many more. In the example below, the main h1 tag is fetched, and then the text is extracted from it.

Code Example:

driver.get("https://www.lambdatest.com/selenium-playground/table-pagination-demo")

table_title = driver.find_element(By.XPATH,'//*[@id="__next"]/div/section[1]/div/div/h1')
print('Title: ',table_title.text)

Make your XML code easy to read and understand with our free online XML Prettify tool. Format your code for better readability and save time!

Console Output:

contains():

This function checks whether an element contains a specific substring or not. You can compare the substring in various ways like text, ClassNames, IDs, and many more. In the below example, find the string “Roland” in the td(cell) of an HTML document.

Code Example:

driver.get("https://www.lambdatest.com/selenium-playground/table-pagination-demo")

table_cell = driver.find_element(By.XPATH,'//tr/td[contains(text(), "Roland")]')
print(table_cell.text)

Console Output:

General Format for contains():

starts-with():

This function checks whether an element’s text starts with a specific substring. There are various ways to compare the substring, like text, ClassNames, IDs, and many more. In the below example, find the string that starts with “R” in the td(cell) of an HTML document.

Code Example:

driver.get("https://www.lambdatest.com/selenium-playground/table-pagination-demo")

table_cell = driver.find_element(By.XPATH,'//tr/td[contains(text(), "Roland")]')
print(table_cell.text)

Console Output:

General Format for contains():

Concatenation:

String concatenation means adding strings together. In the example below, the title and sub_title of the HTML doc are combined.

Code Example:

driver.get("https://www.lambdatest.com/selenium-playground/table-pagination-demo")

main_title = driver.find_element(By.XPATH,'//*[@id="__next"]/div/section[1]/div/div/h1').text
sub_title = driver.find_element(By.XPATH,'//*[@id="__next"]/div/section[2]/div/div/div/h2').text
combined_text = main_title + " " + sub_title
print(combined_text)

Console Output:

Convert your XML files to JSON format without any hassle using our reliable and efficient XML to JSON converter tool. No installation or download required.

Numeric functions

count(node_expression):

This function counts the number of elements in a specified node. There is no direct function in Python; However, it can be achieved using this technique.

Code Example:

driver.get("https://www.lambdatest.com/selenium-playground/table-pagination-demo")

# Select the option with "10" from the drop-down to show maximum 10 rows per page
driver.find_element(By.CSS_SELECTOR, '#maxRows option[value="10"]').click()

rows = driver.find_elements(By.XPATH, '//tr/td')

print(len(rows))

Console Output:

number():

XPath number function converting string to a number.

Code Example:

driver.get("https://www.lambdatest.com/selenium-playground/table-pagination-demo")

# Select the option with "10" from the drop-down to show maximum 10 rows per page
driver.find_element(By.CSS_SELECTOR, '#maxRows option[value="10"]').click()

rows = driver.find_elements(By.XPATH, '//tr/td[1]')
index_list = []
for row in rows:
    if row.text !='':
        index_list.append(int(row.text))

for index in index_list:
    print(index, type(index))

Console Output:

sum(node_expression):

This function returns the sum of element values in a specified node.

Code Example:

driver.get("https://www.lambdatest.com/selenium-playground/table-pagination-demo")

# Select the option with "10" from the drop-down to show maximum 10 rows per page
driver.find_element(By.CSS_SELECTOR, '#maxRows option[value="10"]').click()

rows = driver.find_elements(By.XPATH, '//tr/td[1]')
index_list = []
for row in rows:
    if row.text !='':
        index_list.append(int(row.text))

for index in index_list:
    print(index, type(index))

Console Output:

floor():

floor function returns the largest integer round value that is equal to or less than a parameter value. Another way we can say the last round value is the value return of the function. To use floor() in Python, you must import the math library. eg. FLOOR(10.45) returns 10.

Code Example:

driver.get("https://www.lambdatest.com/selenium-playground/table-pagination-demo")

# Select the option with "10" from the drop-down to show maximum 10 rows per page
driver.find_element(By.CSS_SELECTOR, '#maxRows option[value="10"]').click()

rows = driver.find_elements(By.XPATH, '//tr/td[1]')
index_list = []
for row in rows:
    if row.text !='':
        index_list.append(int(row.text))

for index in index_list:
    print(index, type(index))

Console Output:

ceiling():

XPath ceiling function returns the smallest integer round value that is greater than or equal to a parameter value. Another way we can say the next round value is that value returns of the function. To use ceil() in Python, you must import the math library. eg. CEILING(10.45) return 11.

Code Example:

driver.get("https://ecommerce-playground.lambdatest.io/index.php?route=product/product&product_id=42")

price = driver.find_element(By.XPATH,'//*[@id="entry_216831"]/div/div/h3').text

price = float(price.replace("$",''))
discount = price - (20/price)*100

print("Discounted Price: ", math.ceil(discount))

JSON to XML Converter is a simple and fast tool that converts JSON to XML data. Use this tool to convert your JSON documents into valid XML for easy data exchange.

Console Output:

round():

The XPath round function returns the round number of the specified nth number of decimal places. eg. ROUND(10.45) return 10.

Code Example:

driver.get("https://ecommerce-playground.lambdatest.io/index.php?route=product/product&product_id=42")

price = driver.find_element(By.XPATH,'//*[@id="entry_216831"]/div/div/h3').text

price = float(price.replace("$",''))
discount = price - (5/price)*100

print("Discounted Price: ", round(discount,3))

Console Output:

Node Functions

node():

In XPath, there are seven kinds of nodes: element, attribute, text, namespace, processing-instruction, comment, and root nodes. The function returns the node value.

Code Example:

driver.get("https://www.lambdatest.com/selenium-playground/table-pagination-demo")

node = driver.find_element(By.XPATH, "//tr/td[node()]").text
print(node)

Console Output:

last():

This function returns the size of the total context in a given context. The name of this function lasts, so it means the function does not return the last node value.

Code Example:

driver.get("https://www.lambdatest.com/selenium-playground/table-pagination-demo")

last = driver.find_element(By.XPATH,'//tr/td[last()]').text
print(last)

Our Adler32 Hash Calculator is the perfect tool to create secure hashes to protect data from unauthorized access. Start creating your hashes now!

Console Output:

position():

function returns the position of an element in the set (list) of elements in a given context.

Code Example:

driver.get("https://www.lambdatest.com/selenium-playground/table-pagination-demo")

position = driver.find_element(By.XPATH, "//tr/td[position()]").text
print(position)

Console Output:

id(dtd_id):

function return nodes based on passed DTD unique ID.

Code Example:

driver.get("https://www.lambdatest.com/selenium-playground/table-pagination-demo")

id = driver.find_element(By.XPATH, '//*[@id="table-id"]').text
print(id)

Console Output:

name(node_expression):

function returns the string name of the last expression (node-set).

Code Example:

driver.get("https://www.lambdatest.com/selenium-playground/table-pagination-demo")

id = driver.find_element(By.XPATH, '//*[@id="table-id"]').text
print(id)

Console Output:

comment():

This function matches the comment node and returns that specific comment node.

local-name(node_expression):

function returns the last expression’s local string name.

Need to create secure Gost hashes? Protect your sensitive information with our Gost hash calculator. Get started today and keep your data safe from hackers.

Boolean functions

XPath Boolean functions are used to convert an argument(as a number, string) to a boolean and return either True or False.

boolean(number|string|node-expression|object): returns true if expression is positive.
true(): returns true if the passed string is a normal string.
**false(): **returns false if the passed string is not a normal string.
**lang(): **returns true if the context node language is the same as the specified string argument.

Mathematical and logical operators

You may execute arithmetic operations and combine several criteria to generate more complicated expressions using XPath’s mathematical and logical operators. Based on a variety of parameters, these operators can be used to navigate and choose particular items.

Mathematical Operators

There are various types of mathematical operators like ‘=’, ‘!=’, ‘+’, ‘-‘, ‘*’, ‘/’, ‘>=’, and ‘<=’, which are provided by Python and can be used with XPath.

There are various use cases for using these operators, such as:

Calculating Engagement on a Social Media Post: You are creating a feature to show the engagement on a particular post; in this case, you have to get the data and then apply some calculations to get the engagement.
**Dynamic Web scraping:** Consider a situation where you want to scrap article data from a blogging site, but the blogs are uploaded daily. In this case, you can scrape the number of web pages available and then run the loop according to it.

Let’s look at some basic operators and how to use them with Python XPath. For the below examples, we are using a demo web application maintained by LambdaTest.

Addition (+) and Subtraction (-)

We are getting the total viewers and reward points to get the product’s engagement.

Code Example:

driver.get("https://ecommerce-playground.lambdatest.io/index.php?route=product/product&product_id=42")

viewers = driver.find_element(By.XPATH,'//*[@id="entry_216826"]/ul/li[2]/span[2]').text
reward_points = driver.find_element(By.XPATH,'//*[@id="entry_216826"]/ul/li[3]/span[2]').text

total_points = int(viewers) + int(reward_points)

print("Total rewards: ", total_points)

Console Output:

Multiplication (*) and Division (/)

We are getting a discount(10%) on the product by the formula.

Code Example:

driver.get("https://ecommerce-playground.lambdatest.io/index.php?route=product/product&product_id=42")

price = driver.find_element(By.XPATH,'//*[@id="entry_216831"]/div/div/h3').text
price = float(price.replace("$",''))
discount = price - (10/price)*100


print("Discounted Price: ", round(discount,2))

Console Output:

Logical Operators

There are three main logical operators we will cover in this section.

And operator:

Combines two conditions, both of which must be true for the element to be selected. In the below example, there are two conditional checks: First Name = ‘Chelsea’ and email = ‘c.harper@randatmail.com’

Code Example:

driver.get("https://www.lambdatest.com/selenium-playground/table-pagination-demo")

# Select the option with "10" from the drop-down to show maximum 10 rows per page
driver.find_element(By.CSS_SELECTOR, '#maxRows option[value="10"]').click()


rows = driver.find_elements(By.XPATH, '//tr[td[2][contains(text(), "Chelsea")] and td[4][contains(text(), "c.harper@randatmail.com")]]')

# Print the text of each row
for row in rows:
    print(row.text)

Console Output:

Or operator:

Combines two conditions, either of which can be true for the element to be selected. In the below example, there are two conditional checks: First Name = ‘Chelsea’ or email = ‘b.grant@randatmail.com’

Code Example:

driver.get("https://www.lambdatest.com/selenium-playground/table-pagination-demo")

# Select the option with "10" from the drop-down to show maximum 10 rows per page
driver.find_element(By.CSS_SELECTOR, '#maxRows option[value="10"]').click()

rows = driver.find_elements(By.XPATH, '//tr[td[2][contains(text(), "Chelsea")] or td[4][contains(text(), "b.grant@randatmail.com")]]')

# Print the text of each row
for row in rows:
    print(row.text)

Console Output:

Not operator:

Negates a condition, selecting elements that do not meet the specified condition. In the below example, there are two conditional checks: First Name != ‘Chelsea’ and email != ‘b.grant@randatmail.com’

Code Example:

driver.get("https://www.lambdatest.com/selenium-playground/table-pagination-demo")

# Select the option with "10" from the drop-down to show maximum 10 rows per page
driver.find_element(By.CSS_SELECTOR, '#maxRows option[value="10"]').click()

rows = driver.find_elements(By.XPATH, '//tr[not(td[2][contains(text(), "Chelsea")]) and not(td[4][contains(text(), "b.grant@randatmail.com")])]')

# Print the text of each row
for row in rows:
    print(row.text)

Console Output:

Custom functions and extensions

XPath also allows you to define custom functions and extensions in some environments. However, the support for custom functions and extensions may vary depending on the XPath implementation and the XML/HTML processor you are using, like Selenium.

In Python, you can create a custom function that does a certain task or calculation and outputs a result. Create a function, for instance, to determine whether a given string is included in an element’s text.

Code Example:

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

def contains_text(driver, element, text):
    try:
        return WebDriverWait(driver, 10).until(
            EC.text_to_be_present_in_element((By.XPATH, element), text)
        )
    except:
        return False

Advanced XPath Techniques

There are various techniques to make XPath more reliable and reusable like Handling dynamic content, XPath- Multiple Attribute, and using variables and conditions which we are going to discuss in this section in detail.

Handling dynamic content

The content you’re trying to locate may be dynamic, such as elements with changing IDs or classes. This is a common practice that some websites follow to prevent automating and web scraping. However, the structure stays the same in terms of hierarchy. By using XPath, you can handle this situation. To manage dynamic material, you can employ a variety of techniques:

XPath functions

To locate elements with partial attribute values, XPath offers operations like contains(), starts-with(), and ends-with().

Code Example:

# Find a button with a dynamic ID or className
dynamic_id = "xyz-431353"
button = driver.find_element(By.XPATH, f"//button[contains(@id, '{dynamic_id}')]")

Using Axes

Use XPath axes such as following-sibling, preceding-sibling, etc., to move to the target element in relation to other already known elements.

Code Example:

label_text = "name"
span_element = driver.find_element(By.XPATH, f"//label[text()='{label_text}']/following-sibling::span")

XPath- Multiple Attribute

By placing the | operator between square brackets [], you may use XPath to pick elements based on some attributes. This enables you to filter elements that fit various criteria simultaneously. To do this, you reference an attribute in the XPath expression by using the @ symbol followed by the attribute name.

Make your data tamper-proof with our Whirlpool Hash Calculator. Create secure hashes quickly and easily with just a few clicks. Try it out now!

Using XPath variables and conditions

XPath supports variables and can be utilized to shorten and reuse your expressions. When utilizing your Python XPath expressions in Selenium, you can pass variables to them.

Code Example:

from selenium.webdriver.common.by import By

# Define the XPath expression with variables
element_type = "button"
element_text = "Submit"
xpath_expression = f"//{element_type}[text()='{element_text}']"

# Find the element using the XPath expression
element = driver.find_element(By.XPATH, xpath_expression)

You can also use conditions in your XPath expressions using the if statement.

Code Example:

# Find elements based on a condition (e.g., containing a specific class)
is_enabled = True
condition = "enabled" if is_enabled else "disabled"
xpath_expression = f"//button[contains(@class, '{condition}')]"
elements = driver.find_elements(By.XPATH, xpath_expression)

Keep in mind that using complicated XPath expressions can make it more difficult to read and maintain your code. When feasible, utilize CSS selectors because they are typically quicker and easier to understand. Use XPath only in circumstances where CSS selectors are insufficient. Additionally, to make locating items in your HTML more dependable and effective, choose to use distinct IDs, classes, or data properties.

Conclusion

In a nutshell, when used with Selenium, XPath is a strong and crucial tool for web scraping and automating online applications. Using an element’s attributes, textual content, or connections to other elements enables developers and testers to find that element in an HTML or XML document. Python XPath offers several methods, tools, and axes for effectively navigating and choosing elements.

Before using XPath with Selenium, you must install and configure the necessary Python libraries. When everything is set up, you can find DOM items using XPath expressions. Relative XPath allows you to navigate between elements based on their relationships, while Absolute XPath specifies the whole route from the root. XPath is adaptive to changing web page structures because it can handle dynamic material utilizing functions like *contains(), starts-with(), *and more.

Using XPath, you may extract attribute values and text content to validate and modify data on web pages. You can manage numerous matches and filter outcomes based on particular criteria using XPath. Advanced XPath techniques, such as using variables and conditions and creating custom functions to make XPath statements more dynamic and reusable.

To sum up, XPath is an excellent option for finding objects on web pages, navigating intricate DOM structures, and effectively handling dynamic material thanks to its variety and potent capabilities. Learning XPath enables developers and testers to create automated scripts that are reliable and maintainable, thereby enhancing the quality and efficiency of web applications.

Frequently Asked Questions (FAQs)

Does XPath work with HTML in Python?

Yes, Python supports using XPath with HTML. A strong query language used to browse and pick elements from XML or HTML documents is called XPath. It enables you to extract particular information or elements from the text depending on their positioning and characteristics. Using XPath will be fruitful when unique identifiers are not present like ID, or Class.

How to check if the XPath element exists in Selenium Python?

The find_elements()/ find_element() method in Selenium Python can be used to determine whether an element is present based on an XPath expression. A list of web elements that match the provided XPath is returned by the following method. The element does not exist on the page if the list is empty, which means no elements were detected. By using a simple if else statement or try/catch method you can check for the element.

How to find XPath in Selenium Python?

In Selenium Python, determining an element’s XPath is a rather simple process. To find an element’s XPath, you can do so in a number of different ways. Here are three such methods: 1. Using browser developer tools, 2. Using Selenium’s find_elements()/ find_element(), and 3. Using browser extensions.

Mastering Web Automation With Python Selenium On Chrome

jainishpatel78 — Thu, 25 Jan 2024 14:54:22 +0000

Python is the 4th most preferred programming language according to StackOverflow Survey 2022. It is easy to learn, versatile, and has a large developer community. Python’s open-source nature, high-level syntax, and interpretive capabilities make it an ideal choice.

With frameworks like Flask and Django for backend development and libraries like Selenium for automation testing, Python offers comprehensive options. As software becomes more complex, manual testing is time-consuming. Python’s support for testing frameworks such as Unittest, pytest, Robot, Behave, and Lettuce makes it a popular choice for automation testing.

In this article, we will go through critical areas to help you run Python Selenium on Chrome for web automation. Firstly, we will start with an installation and setup process for Python and Selenium. Next, we will discuss handling various input types and interacting with warnings, alerts, and pop-ups. At the end of the article, you will be able to create tests for automation testing.

Dive into automation testing using Selenium with Java with this detailed tutorial. learn java for automation testing confidently.

Python Selenium Chrome Driver Setup

Selenium is a library that allows to control web browsers and automate tasks/tests. It provides a way to simulate user interaction with a web page, including filling out forms, clicking buttons/links, navigating through pages/windows, and taking screenshots of the window.

Selenium WebDriver uses different WebDrivers, like ChromeDriver for Chrome, **gecko **driver for Firefox, and many others. It gives Selenium WebDriver the ability to control the browser.

Installation and setup process of ChromeDriver for different operating systems

There are multiple steps to be taken to install the driver correctly. Let’s look at each step one by one.

Check the version of Chrome installed on your device. To check the version, go to the top right corner, select the 3 dots icon -> settings, and then select About Chrome from the left panel.

Follow this ChromeDriver link to download it on your system.

Test your native app and website on real iOS and Android devices hosted on the cloud. LambdaTest is a convenient, cost-effective and centralised solution for running realtime and Automated test on device clouds for mobile app testing.

Select the ChromeDriver according to the version of the Chrome browser. After that, we have to choose the operating system on which we desire to run it.

Once the download is complete, extract the chromedriver.exe by unzipping the folder.

Till this point, the process is the same for all operating systems. Now we will see the setup process for each OS.

Windows Operating System

Copy the path for the file chromedriver.exe such that I am storing the driver in “D:\chromedriver.exe”.

2. Right-click on My Computer and click on Properties options.

You will be redirected to a new window that will look similar to the image below; after that, select the Advanced System Settings option.

From the system propertie*s options, select the *Environment Variables** button.

Now, from the system variables, select the path and press the Edit button.

Paste the copied path of the file and save the changes by pressing the **OK **button.

Test your native app and website on real iOS and Android devices hosted on the cloud. LambdaTest is a convenient, cost-effective and centralised solution for running realtime and mobile testing platform with real devices.

macOS Operating System

Open the *Finder *and use ⌘+shift+G
Go to the following path “/usr/local/bin”
Add the “chromedriver.exe” file here

Linux Operating System

If you have already followed the steps to download ChromeDriver, skip steps 1, 2, and 3.

Go to your temp folder:

cd /tmp/
Download the latest Linux-based Chromedriver:

wget https://chromedriver.storage.googleapis.com/2.37/chromedriver_linux64.zip
Extract Chromedriver from its archive:

unzip chromedriver_linux64.zip
Move Chromedriver to the applications folder:

sudo mv chromedriver /usr/bin/chromedriver

If you want to explore the way to set the environment variable for the driver using a terminal, follow the documentation by Selenium.

Explore seamless app testing on real mobile devices with our hassle-free solutions. Ensure optimal performance across all online device testing effortlessly.

How to Check if the Installation is Successful

Windows OS:
Open the command prompt and run the following command:

chromedriver

If the driver is installed correctly, it will open the chromedriver.exe program window.

macOS or Linux-based OS:
Run the following command in the terminal; if installed correctly, it will output the version of the driver.

chromedriver — version or chromedriver -version or chromedriver -v

Installation of ChromeDriver using webdriver-manager

In the year 2022, Selenium introduces Selenium Manager. It is a CLI (Command-Line Interface) program that was created in **Rust *programming language to support cross-platform execution. When no browser driver is found on the PATH or a third-party driver manager is not utilized, Selenium Manager is transparently triggered by the Selenium bindings. In Python, a module known as *webdriver-manager** is used to manage WebDriver binaries for multiple programming languages and browsers. It makes it easier to download and maintain the correct WebDriver executables that are needed for browser automation and to manage WebDriver binaries for browsers like Chrome, Firefox, and Edge, utilize this library.

You must first install the package before you can use webdriver-manager in your Python project. The Python package installer pip can be used to accomplish this by using the following command:

pip install webdriver-manager

Once the module is installed you can use the webdriver-manager to initialize the ChromeDriver as shown in the below code:

from selenium import webdriver
from selenium.webdriver.chrome.service import Service as ChromeService
from webdriver_manager.chrome import ChromeDriverManager
# 
driver = webdriver.Chrome(service=ChromeService(ChromeDriverManager().install()))
driver.close()

Test your native app and website on real iOS and Android devices hosted on the cloud. LambdaTest is a convenient, cost-effective and centralised solution for running realtime and online real device testing.

Opening a Website in Chrome Browser with the help of Python Selenium Script

For this guide, we will use VS Code.

Before moving forward, the following steps need to be done to run the Python scripts with ease.

Creating a virtual environment

It is not a mandatory step; However, if you want to use different versions of Python on your system for different projects, a virtual environment is a way to do it.

Let’s discuss how to create an environment using different methods.

Using GUI:
From VS Code, you can create local environments using virtual environments(venv) or Anaconda(conda) by opening the Command Palette (Windows: Ctrl+Shift+P or macOS: ⇧+⌘+P), start typing the Python: Create Environment command to search and then select the command.

If you select Venv another menu will open to select the version of Python.

After that, the environment folder will be created in your workspace, and a notification will display the creation progress in the bottom right corner.

Explore seamless app testing on real device testing online free with our hassle-free solutions. Ensure optimal performance across real-world scenarios effortlessly.

Using Terminal:
If you choose to create a virtual environment using this method, use the following command (where .venv is the name of the environment folder):

# Windows
# You can also use `py -3 -m venv .venv`
python -m venv .venv


# macOS/Linux
# You may need to run `sudo apt-get install python3-venv` first on Debian-based OSs
python3 -m venv .venv

Once the virtual environment (.venv) is created successfully, create a new Python file with the desired name.

Open a VS Code Terminal and run the following command to install Selenium. The version of Selenium used in this article is *4.9.1 *. To know more about Selenium v4.

pip install selenium

The required setup is completed. Let’s start with the rollercoaster ride of learning Python Selenium using Chrome.

Running Python Selenium using ChromeDriver

To open a website on Chrome Browser, run the block of code:

from selenium import webdriver


driver = webdriver.Chrome()
driver.get("https://google.com")
driver.close()

The import statement is used to get webdriver from the selenium package.

It is used to initialize a new instance of the Chrome WebDriver.

get(): This is a method provided by the WebDriver object, which is used to navigate to a specific URL. It instructs the web browser to open the specified URL.

close(): This method is used to close the current browser window or tab.

Test your native app and website on real iOS and Android devices hosted on the cloud. LambdaTest is a convenient, cost-effective and centralised solution for running realtime and Automated test on real device cloud.

Creating Python Selenium Test Cases

In this section, we will see how to control various actions such as navigating to web pages, clicking buttons, and filling out forms which are very commonly used by most websites. For this, we will use amazon.com to run our Selenium scripts.

find_element method

To perform various actions, we need to locate the element. A key Selenium feature that enables you to locate web items on a page is the find_element method. This approach allows you to search for elements using various criteria, including an ID, name, class name, CSS selector, or XPath. During test execution, you can quickly recognize and interact with the desired elements by setting the proper locator strategy.

Selenium WebDriver offers a range of methods to locate elements on a webpage, providing flexibility and versatility for web automation tasks. These methods enable developers to pinpoint specific elements based on various attributes. Some of the commonly used methods include finding elements by ID, NAME, or XPATH. Additionally, elements can be located by their LINK_TEXT or PARTIAL_LINK_TEXT. Other options include finding elements by their TAG_NAME, by CLASS_NAME, or by using CSS_SELECTOR. By leveraging these various methods, developers can effectively locate and interact with elements on webpages using Selenium with Python. The below table shows the syntax to use different methods.

To use **By **following import statement needs to be imported:

from selenium.webdriver.common.by import By

Find element by ID:
To get a form from the below HTML snippet:

Follow the find_element method will be used:

contact_form = driver.find_element(By.ID, ‘contactForm’)

Other methods will work similarly to the ID method.

Experience efficient Android device test with LambdaTest. Test your apps interactively on a variety of Android devices for optimal performance and user experience.

Adding value to the input field

In automation, interacting with input fields is a regular task. Right-click anywhere on the webpage and choose *inspect **element to view the element’s details. Hover over the element you want to access. After utilizing the find_element function to identify the input element, you may imitate keyboard input by using the *send_keys() method. You can simulate user input by filling the input field with the desired data by giving the desired text or value as an argument.

The below code shows how to use the method:

The below code is using element’s ID to access it:

search_for_key = driver.find_element(By.ID, "twotabsearchtextbox")
# Below code will add iPhone 14 in the input field.
search_for_key.send_keys("iphone 14")

Triggering Button

Buttons play an important role in web applications, from submitting a form to adding some functionalities, and testing their functionality is vital. For this task, we will use the **Xpath *to locate the element. To get the path of the element, *right-click on the element and select copy *from the available options; select copy *Xpath**(relative path of the element).

The code to store the element in a variable by XPath will look as below:

search_btn = driver.find_element(By.XPATH, ‘//*[@id=”nav-search-submit-button”]’)

click() method will click the button.

search_btn.click()search_btn.click()

Controlling Dropdown

Select elements or dropdown menus are frequently found in web forms. Consider a situation where you need to control the dropdown menu. As shown below, we need to select Digital Music.

To use Dropdown get the XPath of the option you want to select. In the below code option[10] is assigned to Digital Music.

select_category_btn = driver.find_element(By.XPATH, ‘//*[@id=”searchDropdownBox”]/option[10]’).click()

After selecting the category below, the piece of code will add the text to the search field.

search_for_key = driver.find_element(By.ID, "twotabsearchtextbox")
search_for_key.send_keys("iphone 14")

The below code will press the search button to trigger the search.

search_btn = driver.find_element(By.XPATH, '//*[@id="nav-search-submit-button"]')
search_btn.click()

To know more about other methods, follow a detailed article for handling Dropdowns.

Clearing Edit Box or Text Input

Another important action in Selenium test cases is clearing the text input or edit box’s contents. To clear the content/text of an edit box or text input, you can use the clear() method provided by the WebElement class.

Here’s an example:

search_for_key = driver.find_element(By.ID, "twotabsearchtextbox")
# Clear the input field
search_for_key.clear()

Advanced Python Selenium Techniques in Chrome

Selenium offers a large range of advanced techniques that let you manage many different actions encountered while performing web testing as well as complex automation activities. In this section, we will cover advanced Python Selenium techniques in Chrome, including drag-and-drop, moving through history and location, switching between windows and frames, handling popups and alarms, adding cookies, adding waits, and taking screenshots.

Perform manual or automated cross browser device testing on 3000+ browsers online. Deploy and scale faster with the most powerful cross browser testing tool online.

Drag and Drop

This functionality is commonly found in modern web apps. Selenium also provides this functionality to simulate drag-and-drop actions on web elements. By locating the source and the target element using the find_element method, you can use the drag_and_dro*p or *drag_and_drop_by_offset methods to perform drag-and-drop operations, imitating user interactions and validating the expected behavior of the application.

# Get the element by its name
source = driver.find_element(By.NAME, "source")
target = driver.find_element(By.NAME, "target")


# initialize the ActionChains
action_chains = ActionChains(driver)


# drag_and_drop method is a part of ActionChains
action_chains.drag_and_drop(source, target).perform()
search_for_key = driver.find_element(By.ID, "twotabsearchtextbox")

For more detailed information on this topic, please refer to our comprehensive blog post on Drag and Drop.

Switching between windows and frames

Most websites nowadays have various iframes and windows which are used to implement various features WebDriver also provides support moving between named windows using the “switch_to.window” method:

# To change the window
driver.switch_to.window("windowName")


# To change the frame
driver.switch_to.frame("frameName")

Scroll a web application

Most websites have scroll to make it more interactive and engaging. The selenium scroll-down code is shown below. It calls the method execute_script() with the javascript to scroll to the end of the web page.

from selenium import webdriver
import time


driver = webdriver.Chrome()
driver.get("https://www.amazon.com")


# scroll the website or webpage to the complete body height
driver.execute_script("window.scrollTo(0, document.body.scrollHeight)")
time.sleep(3)
driver.close()

Popups and Alerts

Handling popups and alerts are paramount when automating web testing. Popups and alerts can appear during various scenarios, such as confirmation messages, error alerts, warnings, or authentication dialogs.

# import Alert
from selenium.webdriver.common.alert import Alert
driver = webdriver.Chrome()
# create an alert object
alert = Alert(driver)
# To print the alert text
print(alert.text)
# accept the alert
alert.accept()

To delve deeper into this topic, we recommend referring to our comprehensive blog post specifically covering Alert and Popup.

Test your native app and website on real iOS and Android devices hosted on the cloud. LambdaTest is a convenient, cost-effective and centralised solution for running realtime and Automated test on real mobile devices.

Navigating through history and location

Selenium enables you to control the browser’s location and navigate its history. To simulate user navigation behaviors, you can use Selenium WebDriver class methods like back(), forward(), and refresh(). You can test the behavior of web applications using these techniques to navigate to other pages, go back to the previous page, go forward to the next page, or refresh the current page.

The below code will showcase the usage of the methods:

# Get the website URL
driver.get("https://www.google.com")
driver.get("https://wwww.amazon.com/")
 # Step backward in the browser history
driver.back()
 # Step forward in the browser history
driver.forward()

Adding Cookies

A cookie is a piece of data from a website stored within a web browser that the website can retrieve later. Web applications utilize cookies heavily to store session data and user preferences.

For adding cookies while running tests, Selenium offers several options. You can define the cookie’s name, value, domain, path, and other parameters using the add_cookie method, whereas the get_cookies() method is used to get the values of cookies. You may mimic personalized or logged-in user sessions by adding cookies, allowing for thorough testing of application operation. The cookies will be stored for a particular domain.

# Set a valid Domain
driver.get("http://www.example.com")


# Now set the cookie. It is valid for the entire domain
cookie = {'key' : 'value'}
driver.add_cookie(cookie)


# get the value stored cookie
driver.get_cookies()

To learn more about this topic, we advise you to read our in-depth blog post about Handling Cookies.

Adding Waits

One must incorporate waits in order to time test execution with the loading and rendering of the web page. You can tell Selenium to wait until a specific condition is met before moving on to the next phase by using explicit or implicit waits. This increases test stability and dependability by ensuring that the test case waits for particular items to be visible, clickable, or present.

There are two types of waits explicit and implicit.

Explicit waits: This piece of code that you declare to wait until a specific event takes place before continuing on the website. The extreme case of this is time.sleep(), which sets the condition to an exact time period to wait.

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
driver = webdriver.Chrome()
try:
   # wait 5 seconds before looking for element
   element = WebDriverWait(driver, 5).until(
       EC.presence_of_element_located((By.ID, "elementName"))
   )
finally:
   # else quit
   driver.quit()

Implicit waits: It tells WebDriver to poll the DOM for a certain amount of time when trying to find any element (or elements) not immediately available. The default setting is 0.

driver.implicitly_wait(10) # seconds

We suggest reading our in-depth blog post on Types of waits to learn it in depth.

Taking Screenshots

It is practical to take screenshots while a test execution for documentation, debugging, and reporting purposes. Selenium offers tools for taking screenshots of a web page whenever you want. You can save the screenshot to a particular location by employing the *save_screenshot *method. By taking screenshots, you may visually check the condition of the website and collect proof of test execution for subsequent investigation.

To store screenshots pillow library is required. It can be installed using the following command:

python3 -m pip install - upgrade pip
python3 -m pip install - upgrade Pillow

The below code will take a screenshot and open the code is implemented with it open a new window to show the preview of the screenshot.

from PIL import Image
driver = webdriver.Chrome()
driver.get("http://amazon.com")
# Taking the image
driver.save_screenshot("image.png")
# Loading the image
image = Image.open("amazonWebsiteImage.png")
# Showing the image
image.show()

To delve deeper into this topic, we recommend you to explore our extensive blog post covering the topic of Python Selenium Screenshots.

Want to ensure your website is optimized for mobile traffic? Our comprehensive guide to website testing on different devices and everything you need to know.

Chrome Options and Preferences

ChromeOptions class has been introduced in the latest/updated version of Selenium. It is helpful to make changes in the Chrome browser, whereas, DesiredCapabilities is an old concept (its usage in Java is deprecated.) to configure or make changes in the browser. Save this answer.

In Selenium 3, capabilities were defined in a session by using Desired Capabilities classes. As of Selenium 4, you must use the browser options classes. For remote driver sessions, a browser options instance is required as it determines which browser will be used.

Page Load Strategy

The page load strategy specifies how long the browser should wait for the page to load before continuing. Different page load techniques, including regular, eager, and none, are offered by Selenium. You can manage how Selenium interacts with the page while running the test by defining the preferred page load strategy in Chrome Options. By selecting the proper page load strategy, you may increase test stability by making sure that your tests wait until all required items have loaded completely.

Types of loading:

normal : Used by default, waits for all resources to download
eager : DOM access is ready, but other resources like images may still be loading
none : Any Does not block WebDriver at all

The below code will showcase how to incorporate eager technique to load the website.

from selenium import webdriver
from selenium.webdriver.chrome.options import Options
# initialize options to use it
options = Options()


options.page_load_strategy = 'eager'


# adding options to local chrome driver
driver = webdriver.Chrome(options=options)
driver.get("http://www.google.com")
driver.quit()

Headless Mode

You can use Chrome’s headless mode to run it without displaying the graphical user interface(GUI). It is helpful for reducing test execution time or for conducting tests in a server environment. You can automate the web without a visible browser window by turning on the headless mode in Chrome Options. When performing tests on distant servers or in a continuous integration (CI) environment, headless mode is especially useful.

from selenium import webdriver
from selenium.webdriver.chrome.options import Options
options = Options()
# add_argument parameter tells Chrome browser that it should be run without UI (Headless)
options.add_argument('--headless')
driver = webdriver.Chrome(options=options)
driver.get("http://www.amazon.com")
driver.quit()

Proxy Setting

Proxy settings are essential for scenarios where you need to route network traffic through a proxy server. With Selenium, you may modify proxy settings by using Chrome Options. By specifying the proxy server’s address and port, you can route your browser’s requests through the proxy server during test execution. Applications that depend on certain network settings can be tested using this, and it can also be used to simulate various geographic locations.

With the help of a proxy within tests, we can verify if the website user interface matches the location.

from selenium import webdriver
from selenium.webdriver.chrome.options import Options
options = Options()
#proxyserver
py = "128.21.0.0:8080"
#proxy parameter to options
options.add_argument('--proxy-server=%s' % py)
#options to Chrome()
driver = webdriver.Chrome(options=options)
driver.implicitly_wait(0.6)
driver.get("http://www.amazon.com")
driver.quit()

Tips and tricks to improve the Chrome browsing experience for web automation

Here are some tips and tricks to enhance the Chrome browsing experience for web automation:

Use Chrome Developer Tools: This tool helps to debug and inspect any website. To access it right-click on a web page and select Inspect or by pressing Ctrl+Shift+I or ⌥ Option + ⌘ Cmd + I. The Developer Tools allow you to analyze network traffic, manipulate the DOM, and test JavaScript code.
Employ User-Agent Switching: The User-Agent header identifies the operating system and browser being used. Depending on the user agent, some websites may act differently or restrict access to a particular functionality. You can change the user agent to imitate various devices or browsers to get around this.
Utilize Headless Mode: Chrome offers a headless mode, which allows you to run Chrome without a GUI(graphical user interface). It can be advantageous for automation since it reduces the usage of resources and allows you to execute tasks in the background.
Handle Page Load Delays: Asynchronous loading of web pages is common, and automation scripts must handle delays effectively. The “Network API,” the powerful API offered by Chrome’s Developer Tools, enables you to keep monitoring network traffic and wait for particular requests to finish before moving further. This might assist in coordinating your automation duties with the manner in which the website loads.
Clear Browser Data: To prevent interruption from past sessions when automating activities frequently, it’s crucial to delete browser data including cookies, cache, and local storage. By using the Chrome settings or programming tools for browser automation, you can manually delete this data.
**Optimize Network Performance: **One should take advantage of Chrome’s network-related features to enhance the execution of automation scripts. Using Chrome’s DevTools or automation libraries, you can, for instance, disable images and pointless resources, simulate slow network conditions, or deny particular requests.
Avoid Captchas and IP Blocking: To prevent automation, several websites use security measures like Captchas or IP blocking. You can utilize machine learning techniques or third-party captcha-solving services to bypass captchas. Using proxy servers or VPNs, you can rotate IP addresses for IP blocking.
Use Page Object Model (POM): By portraying web pages as objects, the Page Object Model design pattern aids in the organization and maintenance of automation code. Your automation scripts’ capacity to be reused, read, and maintained is enhanced. Your code will become more modular and easier to maintain if POM is implemented.
Implement Page Interactions: Interacting with site elements like buttons, forms, dropdown menus, etc. is common in automation. To find and interact with these items, Chrome automation frameworks like Puppeteer and Selenium WebDriver offer APIs. Your automation capabilities will be improved by effectively understanding and utilizing these APIs.
Stay Up-to-date: Regular upgrades and new features are introduced as Chrome continues to develop. Keep up with the most recent Chrome upgrades, which may include adjustments to the browser’s functionality, security improvements, and automation-related capabilities. This will enable you to take advantage of new functionality and modify your automation techniques as necessary.

Note: Revolutionize your user experience with Puppeteer Visual Testing — Capturing, Comparing, and Creating Seamless Experiences!

How to Integrate Selenium with Python Testing Libraries

Popular Python testing tools like Unittest and Pytest may be simply combined with Selenium to increase the functionality of your test automation system. In this guide, we will explore how to integrate Selenium with these testing libraries, providing a structured and efficient approach to writing and executing tests.

Unittest

The base of the Python Unittest library is a third-party package called PyUnit. Based on the well-known JUnit framework, Steve Purcell came up with the idea for PyUnit. It later developed into a recognized Python module starting with version 2.5.

For a long time, Python’s standard library has included the **unittest *module. Therefore, installing it using something like pip install unittest is typically not necessary. It works without installation if you simply run **import unittest* in your Python code.

To get a better understanding and working of the **unittest **module we are going to run the following code:

import unittest
from selenium import webdriver
from selenium.webdriver.common.by import By
class SearchText(unittest.TestCase):
   def set_up(self):
       # create a new Chrome session
       self.driver = webdriver.Chrome()
       self.driver.implicitly_wait(10)
       self.driver.maximize_window()
       # navigate to the amazon.com
       self.driver.get("http://www.amazon.com/")


   def test_search_by_text_keyword(self):
       # get the search bar
       self.search_field = self.driver.find_element(By.ID, "twotabsearchtextbox")
       self.search_field.send_keys("iphone 14")
       # enter search keyword and submit
       self.search_btn = self.driver.find_element(By.XPATH, '//*[@id="nav-search-submit-button"]')
       self.search_btn.click()
   def tear_down(self):
       # close the browser window
       self.driver.quit()
if __name__ == '__main__':
   unittest.main()

The above code will run from top to bottom. Firstly, the set_up() method will initialize the driver and opens the mentioned URL. Then test_search_by_text_keyword() method will be executed where the search box will be selected and added value “iphone 14” and press the search button to search for the item. The test will be ended by closing the browser via tear_down() method.

To use the **unittest *module and write a simple test by inheriting the *TestCase *class, you’ll need to import the *unittest *module and define a class that inherits the *TestCase **class.

The test case’s entry point is a set_up() method. It can be used to perform a predetermined set of operations before running a single test or every test included in the class.

Similar to the set_up() method, test methods get implemented in the TestCase class.

Test site on mobile easily with LT Browser, free responsive testing tool. Just enter the URL to test website on all devices and check if it is compatible across 50+ device viewports.

Pytest

The PyPy project gave rise to the Python testing framework Pytest. It can be used to create a variety of software tests, including functional, end-to-end, unit, and integration tests.

You must run the following command on the terminal (or command prompt), which makes use of the Python package management (pip), to install pytest:

pip install pytest

Once the pytest module is installed successfully. We can run our test using pytest *by the following code and name the file appropriately with the .*py extension:

# Import necessary modules
import pytest


from selenium import webdriver
from selenium.webdriver.common.by import By
# In order for pytest to recognise the test, the test name must begin with test_<name> (for example, test_lambdatest_todo_app).
def test_case():
   driver = webdriver.Chrome()
   driver.maximize_window()
   driver.get("https://www.amazon.com/")
   search_for_key = driver.find_element(By.ID, "twotabsearchtextbox")
   search_for_key.send_keys("iphone 14")
   search_btn = driver.find_element(By.XPATH, '//*[@id="nav-search-submit-button"]')
   search_btn.click()
   driver.close()

Run the Python pytest file using the following command:

pytest <filename>.py

Once the test is completed you get information regarding it in the terminal:

Check out our detailed blog on Pytest to know more about it.

Challenges of Testing on the Local Grid

Testing on a local grid might provide a variety of difficulties. The lack of readily available physical devices and browsers is one of the major problems. A local grid makes it challenging to test across a variety of device and browser configurations, which could have a negative effect on test coverage as a whole. It also needs a lot of resources, including infrastructure setup, software, and hardware, to maintain and manage. It can take time and money to scale the grid to handle rising test demand.

In order to address these issues, cloud-based testing grids are required. Cloud-based testing grids provide a large inventory of actual devices and browsers, making them a scalable and adaptable solutions. Testers have access to a variety of setups and can run parallel tests, which speeds up test execution and improves test coverage. Because the testing environment is hosted on the infrastructure of the cloud provider, the cloud-based method eliminates the requirement for managing and maintaining local infrastructure.

To improve automated testing, a platform like LambdaTest provides cloud-based testing grids. Testers may easily perform Selenium tests in Python on a variety of Chrome browser versions across various operating systems by using LambdaTest. The platform enables testers to build and run automation scripts quickly by providing a seamless interface with well-known test frameworks like Pytest and Unittest.

Running Testcase on a cloud-based testing grid

In this section, we are going to run a test on LambdaTest using Pytest as the testing framework. We are going to run our test on WINDOWS 10 operating system with Chrome version 114.0.

Before Running a Python test on LambdaTest, follow a few simple steps:

Create an account on Lambdatest and complete all the required processes.
Go to the dashboard by clicking the dashboard button on the top right corner or Follow this LambdaTest Dashboard.
Get your credentials from the profile icon located in the top right corner, and then select the profile option it will redirect to the profile screen. Select Password & Security here; you can find your Username and Access Key and save it for future use. Or use this Profile link.

Import required modules

import pytest
from selenium import webdriver
from selenium.webdriver.chrome.options import Options as ChromeOptions
from selenium.webdriver.common.by import By

username and access_key is important to run your test on LambdaTest

username = ""
access_key = ""

Capabilities define the OS, Browser name, and other necessary details

lt_options = {
"user": username,
"accessKey": access_key,
"build": "First build",
"name": "First Test",
"platformName": "Windows 10",
"video": True,
"w3c": True, # informing latest Selenium 4 being used
"browserName": "Chrome",
"browserVersion": "114.0",
"selenium_version": "4.8.0"
}

def test_app():
# To run the test on the platform
remote_url = "@hub.lambdatest.com/wd/hub">http://{}:{}@hub.lambdatest.com/wd/hub".format(username, access_key)
browser_options = ChromeOptions()

# adding the capability to the chrome
browser_options.set_capability('LT:Options', lt_options)

# initializing remote server
driver = webdriver.Remote(command_executor=remote_url, options=browser_options)
driver.get("https://www.amazon.com/")
search_for_key = driver.find_element(By.ID, "twotabsearchtextbox")
search_for_key.send_keys("iphone 14")
search_btn = driver.find_element(By.XPATH, '//*[@id="nav-search-submit-button"]')
search_btn.click()
driver.close()

Understanding the code step by step:

Add your credentials here as it will help the lambdaTest to run tests on your account:

Get your desired capabilities that can be generated from the capabilities generator.

Congratulations on achieving a milestone.

Find details of your test case under Automation>Builds click on the First test to get details of the implementation.

Explore various available test details to get a better idea of the platform.

Conclusion

In a nutshell, Python Selenium with Chrome provides a reliable and adaptable approach to performing testing and automating web-based tasks. In this blog, we’ve covered a variety of Python Selenium with Chrome, starting with how to install and configure ChromeDriver on various operating systems. We gained advanced skills, including drag-and-drop, dealing with pop-ups and warnings, and navigating through history and location. We also learned how to open websites, interact with items on web pages, and execute other basic tasks.

We also go in-depth on some of the extra functionality and personalization choices offered by ChromeOptions, like proxy settings, headless mode, and page load strategies. Additionally, we covered Tips and Tricks to improve Chrome browsing for web automation.

We explored the integration of Selenium with well-known Python testing frameworks like Unittest and Pytest to broaden the scope of testing. As a result, developers can use these frameworks’ robust functionality and reporting skills in conjunction with Selenium.

You may streamline your web automation responsibilities, increase testing productivity, and ensure the quality of your online applications by becoming proficient with Python Selenium with Chrome. You now have a good base to build upon as you continue your exploration and use Python Selenium to its best potential in your projects. Happy automating!

To dive deep into Selenium with Python, follow Python Tutorial with Selenium.

Frequently Asked Questions (FAQs)

What are the significant changes from Selenium 3 to Selenium 4?

Compared to Selenium 3, Selenium 4 brings substantial modifications and enhancements. A new architecture dubbed Selenium Grid 4, stronger support for contemporary web technologies, including the W3C WebDriver Protocol, improved documentation, new APIs for interacting with web elements, and increased debugging abilities are just a few of the essential enhancements.

What is the difference between ChromeOptions and Desired Capabilities?

Both ChromeOptions and Desired Capabilities are used in Selenium for customizing the behavior of the Chrome browser.

ChromeOptions is a class specific to the Chrome browser that allows you to set various browser options such as window size, user agent, and proxy settings. It provides a more convenient and readable way to configure Chrome-specific options.
Desired Capabilities on the other hand, is a generic concept in Selenium that applies to all browsers. It allows you to set a range of capabilities for browser automation, including browser name, version, platform, and other specific settings. Desired Capabilities can be used to customize the behavior of any browser supported by Selenium, not just Chrome.

What are the Python Selenium Chrome best practices?

For efficient automation, it’s crucial that you stick to recommended practices while utilizing ChromeDriver with Python Selenium. For better speed, use headless mode, use explicit waits rather than static sleep, and keep Chrome and ChromeDriver up to date. For maintainable code, specify window size, avoid pointless interactions, and apply the page object pattern. Utilize effective locators like IDs or CSS_SELECTORS, gently handle exceptions, and properly close the browser and driver. Effectively manage sessions and cookies. By following these tips you can improve the dependability, performance, and maintainability of your Python Selenium Chrome automation scripts.

How To Setup And Install PhantomJS In Python: All You Need To Know

jainishpatel78 — Mon, 22 Jan 2024 08:32:23 +0000

Running automation scripts on a normal browser like Chrome, Firefox, or Edge Browser takes too many resources and time as they run test scripts that include all the UI/UX of the website, such as animations and visual effects. So the concept of Headless browsers is introduced. Headless browsers are web browsers that can operate without a graphical user interface (GUI), allowing developers to automate tasks like web scraping and testing. They interact with websites programmatically, processing HTML and executing JavaScript instead of rendering web pages visually. PhantomJS was one of the earliest and simplest headless browsers, offering a JavaScript API for browser manipulation. However, alternatives like Puppeteer (for Node.js) and Selenium WebDriver have gained more traction in recent years.

In this article, we will cover various aspects of PhantomJS, starting from How to install PhantomJS, practical examples, advantages and disadvantages, and running test scripts with it. At last, we will see a test time execution comparison with PhantomJS and other browsers.

New to visual testing? Discover the essentials in our comprehensive guide. Learn what is visual testing, why it matters, and how to begin.

What is a Headless Browser?

A web browser that doesn’t use a Graphical User Interface (GUI) is known as a headless browser. An interface for a headless browser differs from that of a standard browser, such as Chrome, Firefox, Microsoft Edge, or Safari, in that it interacts with websites programmatically rather than rendering the content on a visible window. Testers and Developers can use scripts or code to control it, which enables them to automate a variety of web-related tasks like website monitoring, web scraping, and automated testing. PhantomJS, Headless Chrome, and Firefox are a few examples of popular headless browsers.

Several use cases in web development and testing necessitate headless browsers. Web scraping, the process of taking data from websites for analysis or commercial gain, is a major contributing factor. Large-scale scraping tasks can benefit from headless browsers since they offer a quicker and more effective method of retrieving data programmatically without requiring a visible browser window. Furthermore, automated testing is done with headless browsers to verify the functioning and speed of websites. Testers and Developers can save time and resources by conducting tests in a headless environment while guaranteeing the application operates successfully in various scenarios by executing scripts in headless mode.

Headless browsers are an invaluable tool for web developers and testers because they provide a solid and adaptable interface for programmatic interaction with web content.

What is PhantomJS?

A headless browser called PhantomJS enables programmers to automate interactions with online pages and carry out operations like web scraping, automated testing, and website performance analysis. The WebKit rendering engine, also utilized by Safari browsers, serves as its foundation.

PhantomJS is suitable for server-side operations and automation activities since it runs in a headless mode or without a graphical user interface(GUI). It offers a JavaScript API that enables programmers to control the content of web pages and script browser behaviors.

The capability of PhantomJS to process and render web pages with JavaScript is one of its core capabilities. It has a JavaScript engine integrated within it that can run and parse JavaScript code, making it helpful for scraping dynamically created content or interacting with JavaScript-heavy websites.

It provides a range of features, such as network monitoring, screenshot taking, DOM modification, web page navigation, and form submission. It enables programmed data extraction from web pages and simulation of user interactions.

It’s crucial to remember that PhantomJS development has been halted since the 5th of August 2017; therefore, it might not be the option for a headless browser that is being maintained or updated the most. Because of that, Selenium 3.8.1 also deprecated PhantomJS from its support and recommends using Chrome and Firefox in headless mode.

Explore the best 25 visual testing tools for a seamless user experience in 2023. Find your perfect tool today with our comprehensive list.

Integration of PhantomJS in Python

The Selenium WebDriver enables smooth integration of PhantomJS in Python. A popular automation system called Selenium offers a Python API for managing web browsers. You can use PhantomJS’s capability for a variety of tasks by combining Selenium and PhantomJS.

The significance of utilizing PhantomJS in Python for activities like website testing, data extraction, and screenshot generation is due to its headless nature and features like network monitoring and JavaScript support:

Website testing: Automated website testing is possible with PhantomJS without a visible browser window. It can mimic user interactions, evaluate UI components, test functionality, and record test results.
Data extraction: Web scraping and data extraction are effective when Python, Selenium, and PhantomJS are used together. It can manage web pages with a lot of JavaScript, dynamically generated content, and intricate DOM structures. Web page navigation, element extraction, and data retrieval are all possible for developers.
Creating screenshots: PhantomJS can display online pages and take screenshots, making it possible to conduct visual tests, create website thumbnails, or take pictures when web scraping. This can be helpful for visual documentation creation, preview creation, and website appearance monitoring.

You can automate web browser interactions, conduct effective testing, retrieve data from websites, and create screenshots headlessly by utilizing the combination of PhantomJS in Python through Selenium.

How to install PhantomJS in Python

In this section, we will see the installation process in detail, from setting up Python to PhantomJS WebDriver and Selenium. With that, we will verify it by running a simple script and scraping the website title on which the script runs.

Installation and Environment Setup

The process is quite simple if it follows the chronology. To install PhantomJS in Python, we will see it in a detailed manner, starting from installing Python -> PhantomJS -> Selenium.

Installation and Environment Setup

To install Python on your computer, you need to follow these steps:

Visit the Python official website.
Python provides installers for various platforms, including Windows, macOS, and Linux. It will be selected according to your system automatically on which you are opening the above site.
Download the latest version of Python, it can easily be downloaded by the Download Python 3.x.x button.
Run the installer file once it has been downloaded to begin the installation procedure.
Make sure you click the box on the installation wizard that states Add Python to PATH or **Add Python to environment variables **(the words vary depending on the version). This makes sure that Python is accessible from anywhere on your machine.
You can change the directory according to your preferences.
Click **Install **to begin the process of installation.
Hold off until the installation is finished. The installer will configure Python on your machine and copy the required files.
You should get a notification indicating a successful installation once the installation is complete.
Open a command prompt or terminal and enter the following command:

python — version

Console Output:

If the above command runs fine, Python is installed successfully.
Now that Python is installed successfully let’s install PhantomJS.

With LambdaTest’s Smart Visual UI Testing, ensure visual perfection and deliver seamless websites and web applications 3000+ devices. Try now!

Installing PhantomJS and setup

From the official website, get the PhantomJS binary that matches your operating system.

Extract the downloaded archive into the desired location.
You can find the phantomjs.exe file in the bin folder of the extracted folder.
Add the location of the phantomjs.exe file to the Environment Variables.

You are just one step away before using it. To do that, Selenium WebDriver is needed, a part of the Selenium module. So let’s download Selenium.

Installing Selenium for Python

Open a terminal **or **command prompt.
Install the Selenium package for Python using the *pip *Python package manager.

Run the following command:

pip install selenium==3.8.0

Console Output:

Note: From Selenium 3.8.1 PhantomJS is deprecated. So, to use the PhantomJS it is recommended to use **Selenium 3.8.0 or lower version. You can also refer to the changelog for more information.

Verifying using a simple Python script with PhantomJS

Here is a little piece of code that shows how to initialize the PhantomJS WebDriver and carry out fundamental operations:

from selenium import webdriver

# Initialize the PhantomJS WebDriver
driver = webdriver.PhantomJS()
driver.set_window_size(1120, 550)

# Navigate to a web page
driver.get('https://www.google.com')

# Get the page title
print("Page title:", driver.title)

# Take a screenshot of the page
driver.save_screenshot('screenshot.png')

# Close the WebDriver
driver.quit()

Output:

Let’s understand the code Step-by-Step:

Import statement to get the webdriver from the selenium module.

There are 2 ways to use PhantomJS. You can use it according to your requirements. But, we are using the first way as we already added the Driver to the system’s environment Variables.

As we are using a headless browser just to confirm that the website is opening this line, we will take the screenshot and store it in the same location where the Python file is.

Output Screenshot:

With LambdaTest’s Smart Automated Visual Testing, ensure visual perfection and deliver seamless websites and web applications 3000+ devices. Try now!

Practical Examples of PhantomJS

With the help of the headless web browser PhantomJS, you can automate testing and carry out numerous web scraping operations. It has a JavaScript API that enables you to manage websites, control the browser, and take screenshots. In this section, we are going to have a glance at both of them.

Automation Testing

In automation testing, there are various advantages to using a headless browser like PhantomJS. Without a visual interface, it mimics actual browser behavior, accelerating test performance. It makes it easier to test complicated applications across platforms because it can render and interact with web pages just like a regular browser. Because of its command-line interface and lightweight design to efficiently minimize resource usage and improve performance, testing frameworks may be quickly integrated, resulting in scalable and reliable automation.

You can use frameworks like Selenium WebDriver to create Python and PhantomJS test cases. In the example, we will search for a query and get the resultant output by printing it in the terminal and taking a screenshot.

Here is an illustration of how to create a test case in Python and PhantomJS:

from selenium import webdriver
from selenium.webdriver.common.by import By

# Configure the PhantomJS WebDriver
driver = webdriver.PhantomJS()

# Navigate to a web page
driver.get('https://www.lambdatest.com/selenium-playground/table-sort-search-demo')
driver.set_window_size(1440, 550)

# Getting the search bar
search_bar = driver.find_element(By.XPATH,'//*[@id="example_filter"]/label/input')
search_bar.send_keys("London")

# Getting row of the table
rows = driver.find_elements(By.TAG_NAME,"tr")

# Capturing the screenshot of the window 
driver.save_screenshot('table_data.png')

# Printing the rows
for row in rows:
    data = row.text 
    print(data)
driver.quit()

Output:

Screenshot of the result:

Web Scraping

PhantomJS in Python for web scraping has several advantages, including enhanced resource management and faster execution. It can extract data more quickly as it doesn’t need to visually render web pages because it runs in a headless context. Additionally, it enables better resource management because it uses fewer system resources than a full browser with a GUI.

Python web scraping is effective when done using PhantomJS. You can programmatically control PhantomJS to browse websites, interact with components, and extract data with frameworks like Selenium or Puppeteer. Here is an illustration of how to scrape data of countries and their populations using Python and PhantomJS:

from selenium import webdriver
from selenium.webdriver.common.by import By

# Configure the PhantomJS WebDriver
driver = webdriver.PhantomJS()

# Navigate to a web page
driver.get('https://www.worldometers.info/geography/alphabetical-list-of-countries/')

rows = driver.find_elements(By.TAG_NAME,"tr")

# Initialize a list to store the countries and populations
countries_populations = []

# Iterate over the rows of the table
for row in rows:
    data = row.text.split(" ") # splitting the data 

   # Add the country and population to the list
    countries_populations.append((data))

for countries_population in countries_populations:
    if len(countries_population) == 5:
        print(f"{countries_population[1]}: {countries_population[2]}")

# Close the WebDriver
driver.quit()

Output:

Discover 31 top software testing tools in 2023 for powerful and efficient testing solutions.

Advantages of PhantomJS

There are various benefits of using PhantomJS which are going to cover in this section like Headless nature, Cross-platform compatibility, and many more.

Headless nature: PhantomJS is suitable for server-side automation and tasks where a visible browser window is not required due to its headless nature, which functions without a GUI.
Cross-platform compatibility: PhantomJS supports Windows, macOS, Linux, and FreeBSD, making it cross-platform compatible and enabling developers to use it across numerous platforms.
Support for JavaScript: PhantomJS comes with a JavaScript engine that allows it to handle and run JavaScript scripts on web pages. This makes it practical for scraping web pages that use a lot of JavaScript or dynamic information.
Network monitoring: Network traffic analysis is possible with PhantomJS, which is useful for performance evaluation, troubleshooting, or recording HTTP requests and responses for site scraping.
Screenshot capture: PhantomJS can render websites and take screenshots, making it handy for visual testing, creating thumbnails for websites, or storing screenshots of websites when scraping.

Disadvantages of PhantomJS

Some drawbacks related to Development discontinued, Performance, limited ecosystem support, and Data technology are going to cover in this section in detail.

Development discontinued: PhantomJS’s development has been stopped since 2017; hence there may not be any further security patches or bug fixes. The reason behind its depreciation is that the community needs to maintain it actively. Compatibility problems with more modern web technologies such as HTML5, CSS3, ES6+ JavaScript, and potential security holes also resulted in discontinuation.
Performance: PhantomJS executed rather quickly, although its page rendering and execution times might be slower than those of other headless browsers like Chrome or Firefox. This may affect the general effectiveness of automated tasks. We are going to see this in the next section.
**Limited ecosystem support: **The ecosystem and community support for PhantomJS may be significantly smaller than those for competing headless browsers. As a result, developers will have less access to tools, guides, and community-driven upgrades.
Dated technology: PhantomJS may not support more recent web standards, APIs, or functionality because development on it has ceased. This may reduce its ability to work with current websites and web applications.

Comparing Test Time Execution of Different Headless Browsers

There are various options available, like Chrome, Firefox, and Microsoft Edge, which provide a better performance, development tools, and many more. In this section, we will see how to use them and compare the execution speed of the script with the execution speed of PhantomJS.

For the execution speed comparison, we will run the below test on different browsers.

def scrape_table_data():
    start = time.time()
    driver.get('https://www.lambdatest.com/selenium-playground/table-sort-search-demo')

    search_bar = driver.find_element(By.XPATH, '//*[@id="example_filter"]/label/input')
    search_bar.send_keys("London")

    rows = driver.find_elements(By.TAG_NAME, "tr")

    data_list = []
    for row in rows:
        data = row.text
        data_list.append(data)
    driver.quit()
    end = time.time()

    execution_time = (end - start) * 10 ** 3
    return execution_time

PhantomJS

The below code will run the test on PhantomJS and the functions return the execution time which will be printed in milliseconds.

from selenium import webdriver
driver = webdriver.PhantomJS()
execution_time = scrape_table_data()
print(f"Time taken PhantomJS Browser: {execution_time:.03f}ms")

Note: we are using Selenium 3.8.0 to use the PhantomJS browser.

Output:

This article sheds light on the best cross browser testing tools to look for in 2023.

Chrome

To run the test on the Chrome browser in the headless mode you are required to add –headless=new using the add_argument() method in the Options.

Note: It is recommended to use the latest version of the Selenium. While executing the code Selenium 3.11.2 is used.

The below code will run the test on the Chrome browser in headless mode.

from selenium import webdriver
from selenium.webdriver.chrome.options import Options as ChromeOptions
options = ChromeOptions()
options.add_argument('--headless=new')
driver = webdriver.Chrome(options=options)
execution_time = scrape_table_data()
print(f"Time taken Chrome Browser: {execution_time:.03f}ms")

Output:

Firefox

To run the test on the Firefox browser in the headless mode you are required to add –headless using the add_argument() method in the Options.

Note: It is recommended to use the latest version of the Selenium. While executing the code Selenium 3.11.2 is used.

The below code will run the test on the Firefox browser in headless mode.

from selenium import webdriver
from selenium.webdriver.firefox.options import Options as FirefoxOptions
options = FirefoxOptions()
options.add_argument('--headless')
driver = webdriver.Firefox(options=options)
execution_time = scrape_table_data()
print(f"Time taken Firefox Browser: {execution_time:.03f}ms")

Output:

Discover the top 20 mobile testing tools for 2023 in this informative blog post. Stay up-to-date with the latest advancements and choose the perfect tool to meet your app testing requirements.

Microsoft Edge

To run the test on the Microsoft Edge browser in the headless mode you are required to add –headless using the add_argument() method in the Options.

Note: It is recommended to use the latest version of the Selenium. While executing the code **Selenium 3.11.2 **is used.

The below code will run the test on the Microsoft Edge browser in headless mode.

from selenium import webdriver
from selenium.webdriver.edge.options import Options as EdgeOptions
options = EdgeOptions()
options.add_argument('--headless')
driver = webdriver.Edge(options=options)
execution_time = scrape_table_data()
print(f"Time taken Edge Browser: {execution_time:.03f}ms")

Output:

From the above table, it can be concluded that the Chrome browser executes the test script most efficiently. Though Firefox is not left behind in the race it equally performs well.

Clone the PhantomJSInPython GitHub repository to run the Python files mentioned in the blog.

Apart from headless browsers, several cloud grid options offer the functionality to run tests without utilizing the system’s resources. One prominent platform in this domain is LambdaTest, an AI-Powered Test Orchestration platform, set up as a cloud infrastructure. It provides a comprehensive suite of features, including Automation Testing, Smart UI Testing, and Screenshots, all delivered optimally.

Discover 31 top QA automation tools in 2023 for powerful and efficient testing solutions.

The main advantages of using Cloud Grid like LambdaTest are as follows:

Ease of Use: LambdaTest is simple to use and has a user-friendly interface, making it simple to prepare and run tests without complicated settings or technical expertise.
Cloud Grid architecture: LambdaTest offers a cloud-based grid architecture for executing tests across numerous operating systems, browsers, and versions, ensuring thorough test coverage.
**Scalability: **LambdaTest makes it simple to scale testing efforts by giving users access to a huge variety of real browsers and devices, allowing for parallel testing, and shortening testing times in general.
Optimized Testing: By utilizing cloud resources, LambdaTest does not require the installation or upkeep of local browsers, conserving system resources and guaranteeing consistent test findings in various circumstances.
Smart UI Testing: A consistent user experience is ensured by LambdaTest’s Smart UI Testing features, which allow for the identification of visual regression problems and design inconsistencies by comparing screenshots of applications across browsers and devices.
Hassle-free Integration: Through its API, CLI, and browser extensions, LambdaTest easily interfaces with well-known automation frameworks and CI/CD technologies, making it simple to include in current development and testing workflows.

Conclusion

PhantomJS is a powerful headless browser that offers a range of capabilities for automation testing and web scraping. Integrating PhantomJS with Python allows developers to harness its functionalities and leverage its potential in their projects.

We saw How to install PhantomJS in Python in a step-by-step process, including the installation of Python itself, the setup of PhantomJS, and the installation of Selenium for Python. Verifying the installation can be done through a simple Python script that utilizes PhantomJS.

PhantomJS provides numerous advantages, such as its ability to execute automated tests, scrape websites, and support various programming languages. However, it also has some disadvantages, including its declining support and limited compatibility with newer web technologies.

Moreover, PhantomJS can be employed in real-world scenarios, such as data extraction and mining, as well as web application testing. These applications showcase the practical use cases and examples of utilizing PhantomJS effectively.

In conclusion, PhantomJS, when integrated with Python, provides a robust solution for automation testing and web scraping. While it has its advantages and disadvantages, understanding its features and optimizing its performance can lead to successful implementation. By leveraging the capabilities of PhantomJS, developers can enhance their projects and achieve their desired outcomes.

Frequently Asked Questions (FAQs)

Is there any community or support available for PhantomJS?

There was support and an active community for PhantomJS. PhantomJS has been deprecated since 2017, and its development has been halted since then. The official PhantomJS website urges users to switch to other programs like Puppeteer or Headless Chrome.

Are there any performance considerations when using PhantomJS in Python?

There are a few performance issues to remember when using PhantomJS in Python. PhantomJS executes in a separate process, adding extra overhead. Additionally, there may be some latency while using PhantomJS’s API. Consider alternatives like Puppeteer or Headless Chrome, which offer greater performance and more active development, if performance is a crucial concern.

Can PhantomJS be used with many OS systems?

Yes, a variety of operating systems can use PhantomJS. It is intended to work with various operating systems, including Windows, macOS, Linux, and FreeBSD. Depending on their requirements and preferences, customers can use PhantomJS on various operating systems thanks to this cross-platform support.

Can I run PhantomJS in parallel or distributed setups?

Yes, PhantomJS may be used in distributed or parallel configurations. You can accomplish this by simultaneously coordinating the execution of several PhantomJS instances deployed across several computers or virtual environments. This enables improved performance and scalability when managing complicated operations or answering numerous requests simultaneously. The use of alternative tools with built-in support for parallel or distributed execution, such as Puppeteer or Headless Chrome, is advised because PhantomJS has been deprecated and replaced.

Looking to manage your tests more efficiently? In this blog, explore a variety of test management tools and select the one that suits your team’s needs.

Dive Into Headless Chrome: Your Key to Efficient Web Development

jainishpatel78 — Mon, 18 Dec 2023 06:41:16 +0000

Numerous technologies and tools have been developed as a result of the ever-expanding internet that is intended to make web creation and test automation less complicated. Headless Chrome is one of the most noteworthy tools in this area. This headless version of the popular Google Chrome browser works without a visible window and gives programmers and developers a superb way to interact with web information programmatically.

In this article, we will explore key topics concerning Headless Chrome, such as detecting the headless mode, taking screenshots of the browser window, working with userAgent, printing DOM, handling cookies, and running tests on Cloud Grid.

Streamline mobile app development across iOS, Android, Windows, and macOS with Visual Studio App Center. It’s an all-in-one platform that consolidates services like building, testing, distribution, monitoring, diagnostics, and more into a unified cloud solution. Plus, it simplifies the app upload process, reducing manual effort.

What is Headless Chrome?

The Google Chrome web browser has a headless (or non-graphical) variant called Headless Chrome or Chrome Headless. It was launched with Chrome 59. Running Chrome essentially without Chrome! It enables programmatic and automated usage of Chrome’s features without presenting a Graphical User Interface (GUI). It provides to the command line all of the contemporary web platform capabilities offered by Chromium and the Blink rendering engine.

Common Use Cases of Headless Chrome?

Headless Chrome is mostly used for data extraction, data scraping, software testing, and web automation. It ensures functioning and spots regressions. In automation testing, it serves a dual purpose: ensuring the correctness of operations and promptly identifying any regressions.

It also provides programmers with the means to engage in web scraping, effectively extracting data from websites. Simultaneously, it simplifies activities like form-filling and the generation of PDFs through web automation.

Cross browser testing is made easier by assisting developers to ensure browser compatibility. Remote access to Chrome DevTools allows for debugging. Apart from this, it is used by security experts to examine the security of web apps, and web performance. It is ideally suited for server-side and automated processes because it can carry out these functions without the need for a graphical user interface.

Different Versions of Headless Chrome

A new mode was added in Chrome version 96 along with the established **— headless *option. From versions 96 to 108, this fresh mode was known as *— headless=chrome, and from versions 109 onwards, it was known as ***— headless=new*. Users have complete access to all browser features in this mode, including the ability to use extensions. This improvement goes above and beyond what the standard headless mode can do, giving users and developers additional freedom to use Chrome’s robust features while operating in a headless environment.

Discover 31 Top Software Testing Tools in 2023 for powerful and efficient testing solutions.

Working of Chrome — headless=new Mode

Before we delve into understanding — — *headless=new* mode, it’s essential to grasp the working of — headless. From the implementation of Headless Chrome, it can easily be misinterpreted that the Headless Chrome and regular Chrome browser are the same. Interestingly, this was only partially true.

The codebase of Headless Chrome didn’t share any of the Chrome browser code located in //chrome. However, Headless mode introduced some unique bugs and features that are not found in regular Chrome. This sometimes creates a scenario where a test case is working correctly in one mode but fails in the other. It also slowed automation testing performance involving browser extensions or other browser-level functions without dedicated Headless support.

In 2021, the Chrome development team finally integrated the Headless and Headful modes in version 112. In this mode, Chrome starts but doesn’t display any platform windows. However, all existing functionalities and future updates are available with no restrictions.

How to Run Headless Chrome in Selenium?

In this section, we are going to cover the installation process for Chrome Headless.

There are a few prerequisites for using Headless Chrome:

The latest version of Python 3.
Chrome Browser.
Newer version of Selenium 4.

You can follow a detailed blog on Python Selenium On Chrome if you are facing difficulties in installing any of the prerequisites.

Headless Chrome is enabled using ChromeOptions().

Discover the game-changing power of 31 leading QA automation tools in 2023 — optimize your testing for unparalleled efficiency and effectiveness.

Basic Headless Chrome Techniques

This section covers some basic functionalities like detecting Headless Chrome, getting the title of the website, taking screenshots, and many more.

How to Detect Headless Chrome in Selenium

You can use userAgent to detect whether Chrome is running in Headless mode or not. A web browser or other client software transmits a userAgent string to a web server with each request to identify itself and its capabilities. The user agent string includes details that can be used to determine the client, such as the operating system, device type, and browser version.

The below code shows the use of userAgent for detection.

from selenium import webdriver
from selenium.webdriver.chrome.options import Options as ChromeOptions

options = ChromeOptions()
options.add_argument("--headless=new")

driver = webdriver.Chrome(options=options)

isHeadless = driver.execute_script("return navigator.userAgent;")
substr = "HeadlessChrome"

print(isHeadless)
if substr.lower() in isHeadless.lower():
    print("Headless Chrome detected")
else:
    print("Non-Headless Chrome detected")

driver.quit()

Console Output:

Getting the Title of the Website

The title of the website can be used to verify that the website is accessible to users and also verify that the website is loading the correct page.

The below code shows how to get the title and check it using assertion.

from selenium import webdriver
from selenium.webdriver.chrome.options import Options as ChromeOptions

# Create a new ChromeOptions object.
options = ChromeOptions()

# Set the headless mode.
options.add_argument("--headless=new")

# Create a new ChromeDriver object, passing in the ChromeOptions object.
driver = webdriver.Chrome(options=options)

# Navigate to the desired website.
driver.get("https://ecommerce-playground.lambdatest.io/")

# Print the title of the page.
print("Title of the page is: " + driver.title)

# Assert that the title of the page contains the text "Your Store".
assert "Your Store" in driver.title

# Close the browser.
driver.quit()

Console Output:

The title method is used to get the title of the website. Assert is used for verifying or validating the website title. Learn How To Use Assert and Verify in Selenium WebDriver for accurate validation of website titles.

Taking a Screenshot of a Window

The screenshot ensures that while performing automation tests, we can have a visual record of the page’s state in case of any errors.

The below code shows how to use the save_screenshot() method.

from selenium import webdriver
from selenium.webdriver.chrome.options import Options as ChromeOptions

options = ChromeOptions()
options.add_argument("--headless=new")

driver = webdriver.Chrome(options=options)

driver.get("https://ecommerce-playground.lambdatest.io/")

# Save a screenshot of the current page to the file "screenshot.png".
driver.save_screenshot("screenshot.png")
driver.quit()

Taken Screenshot:

Selenium by default provides a method to record screenshots via the save_screenshot() method.

Advanced Headless Chrome Techniques

For developers, testers, and QAs, advanced techniques in Headless Chrome open up a world of customization possibilities. We’ll look at some of the methods in this section, like customizing userAgent, handling cookies, printing Document Object Model (DOM), generating PDFs, and working with ActionChains.

Customizing userAgent

Customizing the userAgent serves multiple purposes, such as debugging web applications, testing web applications across various browser environments, enabling web scraping without triggering anti-scraping measures, and accessing websites that may be restricted based on geographical location or device types.

from selenium import webdriver
from selenium.webdriver.chrome.options import Options as ChromeOptions

# Define your custom user agent
custom_user_agent = "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/116.0.0.0 Safari/537.36"

options = ChromeOptions()
options.add_argument("--headless=new")
options.add_argument(f"--user-agent={custom_user_agent}")  # Set custom user agent

driver = webdriver.Chrome(options=options)

user_agent = driver.execute_script("return navigator.userAgent;")
# Print the user agent string.
print(user_agent)
driver.quit()

Console Output:

This guide offers a complete overview of the various web automation tools available and how to choose the right tool for your specific testing needs.

Handling Authentication and Cookies

While performing web testing, it becomes crucial to handle and use cookies related to authentication or maintaining session data. Selenium provides various functions like add_cookie(), get_cookie(), get_cookies() and delete_cookie() to work with cookies.

from selenium import webdriver
from selenium.webdriver.chrome.options import Options as ChromeOptions

options = ChromeOptions()
options.add_argument("--headless=new")
driver = webdriver.Chrome(options=options)

driver.get("https://ecommerce-playground.lambdatest.io/")

driver.add_cookie({"name": "rohan", "value": "user1234"})
print(driver.get_cookie("rohan"))
driver.quit()

Console Output:

Capturing Screenshots

Taking snapshots of different HTML elements like div, span, a, and many more while performing tests helps to figure out issues related to UI and rendering data.

from selenium import webdriver
from selenium.webdriver.chrome.options import Options as ChromeOptions

options = ChromeOptions()
options.add_argument("--headless=new")
driver = webdriver.Chrome(options=options)

driver.get("https://ecommerce-playground.lambdatest.io/")

element = driver.find_element(By.XPATH,'//*[@id="mz-carousel-213240"]/div/div[1]/a/img')

element.screenshot('screenshot.png')

driver.quit()

Taken Screenshot:

Printing the DOM

Printing Document Object Model (DOM) helps evaluate a web page’s structure, which is essential for debugging and guaranteeing data consistency. You can inspect the page’s structure and spot any irregularities in the data retrieving process by printing the DOM.

from selenium import webdriver
from selenium.webdriver.chrome.options import Options as ChromeOptions
from selenium.webdriver.common.by import By
import time
options = ChromeOptions()
options.add_argument("--headless=new")
driver = webdriver.Chrome(options=options)

driver.get("https://ecommerce-playground.lambdatest.io/")

print(driver.page_source)

# or

html = driver.execute_script("return document.documentElement.outerHTML;")

# Save the DOM to a file.
with open("dom.html", "w") as f:
    f.write(html)
driver.quit()

Console Output:

File Output:

Looking to manage your tests more efficiently? In this blog, explore a variety of test management tools and select the one that suits your team’s needs.

Generating PDFs

Generating PDFs is a handy method for archiving web content and verifying that web pages are printable. There are various ways to generate a PDF:

Using the Selenium print_page() method:
The print_page() prints the current page within the browser. Read more about it from the official docs of selenium.

from selenium import webdriver
from selenium.webdriver.chrome.options import Options as ChromeOptions
from selenium.webdriver.common.print_page_options import PrintOptions
import base64
print_options = PrintOptions()
print_options.page_ranges = ['1-2']

options = ChromeOptions()
options.add_argument('--headless=new')

driver = webdriver.Chrome(options=options)
driver.get("https://ecommerce-playground.lambdatest.io/")

base64code = driver.print_page(print_options)
# decode the base64 string to bytes
pdf_bytes = base64.b64decode(base64code)

# write the bytes to a file with a .pdf extension
with open('output.pdf', 'wb') as f:
    f.write(pdf_bytes)
driver.quit()

Console Output:

Using Chrome DevTools Protocols:

Chrome DevTools Protocols (CDP commands) allow you to interact with and control Google Chrome or Chromium-based browsers programmatically. You can use CDP commands to perform various tasks, such as navigating to web pages, inspecting the DOM, capturing screenshots, and more. Here’s an overview of how to use CDP commands with the help of the execute_cdp_cmd() method provided by Selenium:

from selenium import webdriver
from selenium.webdriver.chrome.options import Options as ChromeOptions
import base64


options = ChromeOptions()
options.add_argument('--headless=new')

driver = webdriver.Chrome(options=options)
driver.get("https://ecommerce-playground.lambdatest.io/")

# Execute the CDP command to print the page to PDF.
pdf_data = driver.execute_cdp_cmd("Page.printToPDF", {"path": 'html-page.pdf', "format": 'A4'})

# Decode the PDF data and write it to a file.
with open('cdp_output.pdf', 'wb') as f:
    f.write(base64.b64decode(pdf_data['data']))

driver.quit()

Taken Screenshot:

Capture, Create, and Recreate Actions

Automating user interactions with Headless Chrome allows you to create end-to-end test scripts, replicating real user behavior.

Capture: Use ActionChains to record user actions like mouse movements and clicks; call the perform() method to execute them.
Create: Make a new ActionChains object, add user actions, and call the perform() method to execute them.
Recreate: To repeat captured user actions, create new ActionChains, add recorded actions, and call perform().

from selenium import webdriver
from selenium.webdriver.chrome.options import Options as ChromeOptions
from selenium.webdriver.common.by import By
from selenium.webdriver.common.action_chains import ActionChains

options = ChromeOptions()
options.add_argument('--headless=new')

driver = webdriver.Chrome(options=options)

driver.get("https://ecommerce-playground.lambdatest.io/")

Set the window size to 1200x760

driver.set_window_size(1200, 760)

Find the element to interact with

element = driver.find_element(By.CSS_SELECTOR, ".active > .d-block > .d-block")

Create a new ActionChains object

actions = ActionChains(driver)

Move the mouse to the element and click and hold

actions.move_to_element(element).click_and_hold().perform()

driver.save_screenshot('action1.png')

Release the mouse click

actions.move_to_element(element).release().perform()

Click on the dropdown toggle

driver.find_element(By.CSS_SELECTOR, ".dropdown-toggle > .info:nth-child(2) > .title").click()

driver.save_screenshot('action2.png')

driver.quit()

Action1 Image Output:

Action 2 Image Output:

Retrieving Performance Logs from Chrome Headless

It can be retrieved using execute_cdp_cmd() in Selenium. Performance logs are essential because they provide information about how a web application is being used, and help locate and fix performance issues, resulting in faster loading of web applications and ultimately better user experiences.

from selenium import webdriver
from selenium.webdriver.chrome.options import Options as ChromeOptions

options = ChromeOptions()
options.add_argument('--headless=new')

driver = webdriver.Chrome(options=options)
driver.get("https://ecommerce-playground.lambdatest.io/")

driver.execute_cdp_cmd('Performance.enable', {})
t = driver.execute_cdp_cmd('Performance.getMetrics', {})
print(t)

driver.quit()

Console Output:

How to Debug in Headless Chrome?

The DevTools protocol is activated when Chrome is run with the — remote-debugging-port=9222 option. DevTools protocol allows Chrome and the headless browser to communicate with one another. Tools like Sublime, Visual Studio Code, and Node.js also use this protocol for remote debugging.

Since there isn’t a visible browser window in headless mode, you may see if it’s functioning by using another web browser and going to http://localhost:9222. This will provide a list of pages that you may examine to see how the headless browser renders different pages.

You can analyze, debug, and modify the web page using the same DevTools interface seen on this page, just as you would with a standard browser. This website also functions as a helpful debugging tool for viewing all the underlying commands that the DevTools protocol is delivering to the browser if you’re utilizing headless mode in your code.

Using Headless Chrome on Cloud Grid

When it comes to running a series of test scripts to guarantee website compatibility across multiple versions of Chrome, a problem arises. In our current environment, where Chrome version 118 is the only version available, the implementation of test scripts for Chrome version 114 would lead to code errors. This is where the use of Cloud Grid comes into play.

A cloud grid gives users access to various real and virtual machines, each with its configuration, operating system, and web browser. This guarantees accurate testing in various situations, an essential component of reliable software development. Additionally, Cloud providers typically offer high availability and redundancy, reducing the likelihood of downtime during testing.

LambdaTest is a Selenium based cloud Grid and cross browser testing platform for web and mobile applications. It supports tests across 3000+ combinations of browsers, real devices, and OS to offer cross-browser compatibility. This empowers you to establish testing procedures that not only meet your specific requirements but also expand in scale as your project evolves.

In this section, we will run a test on LambdaTest. We will run our test on the macOS Sonoma operating system with Chrome version 118.0.

Before Running a Python test on LambdaTest, follow the simple steps:

Create an account on LambdaTest and complete all the required processes.
Go to the dashboard by clicking the dashboard button on the top right corner or Follow this LambdaTest Dashboard.
Get your credentials from the profile icon located in the top right corner, and then select the profile option it will redirect to the profile screen. Select Password & Security here; you can find your Username and Access Key and save it for future use. Or use this Profile link.

The below test uses the same code as the Capture, Create, and Recreate Actions section.

from selenium import webdriver
from selenium.webdriver.chrome.options import Options as ChromeOptions
from selenium.webdriver.common.by import By
from selenium.webdriver.common.action_chains import ActionChains

username = "username"
access_key = "access_key"


# Capabilities define the OS, Browser name, and other necessary details
lt_options = {
    "user": username,
    "accessKey": access_key,
    "build": "Build for Headless Chrome",
    "name": "Test 1",
    "platformName": "macOS Sonoma",
    "headless": True, # Headless Chrome
    "video": True,
    "w3c": True,  
    "browserName": "Chrome",
    "browserVersion": "118.0",
}

def test_for_headless_on_cloud_gird():
    # To run the test on the platform
    remote_url = "@hub.lambdatest.com/wd/hub">http://{}:{}@hub.lambdatest.com/wd/hub".format(username, access_key)
    browser_options = ChromeOptions()

    # Adding the capability to the chrome
    browser_options.set_capability('LT:Options', lt_options)

    driver = webdriver.Remote(command_executor=remote_url, options=browser_options)
    driver.get("https://ecommerce-playground.lambdatest.io/")

    element = driver.find_element(By.CSS_SELECTOR, ".active > .d-block > .d-block")

    actions = ActionChains(driver)

    actions.move_to_element(element).click_and_hold().perform()

    actions.move_to_element(element).release().perform()

    driver.find_element(By.CSS_SELECTOR, ".dropdown-toggle > .info:nth-child(2) > .title").click()

    driver.quit()

if __name__ == "__main__":
    test_for_headless_on_cloud_gird()

Understanding the code step by step:

Add your credentials here as it will help the lambdaTest to run tests on your account:

Get your desired capabilities that can be generated from the capabilities generator.

Find details of your test case under Automation>Builds click on the First test to get details of the implementation.

Explore various available test details to get a better idea of the platform.

In this tutorial, learn what is Regression testing, its importance, types, and how to perform it.

Conclusion

Headless Chrome is a powerful instrument for web scraping and automation testing. It comes in a variety of configurations that are tailored to different use cases, and Selenium integration is like the cherry on the cake.

Essential functionalities are provided by simple methods, including detecting Headless Chrome, getting web page title, and taking screenshots. Customizing userAgent, managing cookies for authentication or managing sessions, generating website PDFs, and obtaining performance logs all present a multitude of opportunities for more complex use cases in test automation.

Headless Chrome may simplify your workflows and assist you in achieving your objectives, regardless of whether you are a Developer, Tester, or QA. When successfully utilized, its adaptability and capabilities can greatly improve your web automation initiatives.

Frequently Asked Questions (FAQs)

Why to use Headless Chrome?

Headless Chrome is lightweight, it uses fewer resources, making it perfect for server-side operations.

How to detect Headless Chrome?

One common method to identify Headless Chrome is by examining its userAgent string. Headless Chrome userAgent strings typically include the word “Headless” and may also mention the Chrome version. For example HeadlessChrome/XXX.0.0.0.

How to run Chrome Headless in Selenium?

You just need two things installed on your machine Chrome browser and Selenium WebDriver. After that you use the headless chrome by the “–headless=new” as an argument in chrome options.

How fast is Chrome Headless?

Since it doesn’t have to show the content visually, Chrome Headless is often 1.5x to 2x faster than a standard, real Chrome browser. However, speed can vary according to the unique use case and the effectiveness of the scripts or tools being used with Chrome Headless.

Does Headless Chrome use regular browser cookies?

Yes, just like Chrome’s standard version, Headless Chrome uses conventional browser cookies. This enables a flawless browsing experience while using automated scripts or performing web testing because when you interact with a website using Headless Chrome, it may save and transfer cookies to retain your session and user-specific data.

Forem: jainishpatel78

Use Cases of XPath Python In Selenium With Examples

What is XPath and Why is it used with Selenium?

How to Install and Setup Python XPath

Install Selenium

Install lxml Library

Verifying the Selenium Installation

Finding an Element Using Python XPath in Selenium

Using Python XPath to Locate Elements

Retrieving attribute values

Extracting Text Content and Manipulating Using Python

Handling multiple matches and filtering results

Types of XPath in Selenium

Absolute XPath

Relative XPath

Breakdown of XPath Expressions

Using Python XPath with Functions, Mathematical and logical Operators

Functions in Python XPath

String Functions

Numeric functions

Node Functions

Boolean functions

Mathematical and logical operators

Mathematical Operators

Logical Operators

Custom functions and extensions

Advanced XPath Techniques

Handling dynamic content

XPath- Multiple Attribute

Using XPath variables and conditions

Conclusion

Frequently Asked Questions (FAQs)

Does XPath work with HTML in Python?

How to check if the XPath element exists in Selenium Python?

How to find XPath in Selenium Python?

Mastering Web Automation With Python Selenium On Chrome

Python Selenium Chrome Driver Setup

Installation and setup process of ChromeDriver for different operating systems

Windows Operating System

macOS Operating System

Linux Operating System

How to Check if the Installation is Successful

Installation of ChromeDriver using webdriver-manager

Opening a Website in Chrome Browser with the help of Python Selenium Script

Creating a virtual environment

Running Python Selenium using ChromeDriver

Creating Python Selenium Test Cases

find_element method

Adding value to the input field

Triggering Button

Controlling Dropdown

Clearing Edit Box or Text Input

Advanced Python Selenium Techniques in Chrome

Drag and Drop

Switching between windows and frames

Scroll a web application

Popups and Alerts

Navigating through history and location

Adding Cookies

Adding Waits

Taking Screenshots

Chrome Options and Preferences

Page Load Strategy

Headless Mode

Proxy Setting

Tips and tricks to improve the Chrome browsing experience for web automation

How to Integrate Selenium with Python Testing Libraries

Unittest

Pytest

Challenges of Testing on the Local Grid

Running Testcase on a cloud-based testing grid

Import required modules

username and access_key is important to run your test on LambdaTest

Capabilities define the OS, Browser name, and other necessary details

Conclusion

Frequently Asked Questions (FAQs)

What are the significant changes from Selenium 3 to Selenium 4?

What is the difference between ChromeOptions and Desired Capabilities?

What are the Python Selenium Chrome best practices?

How To Setup And Install PhantomJS In Python: All You Need To Know