Forem: jasson99

Setup and run tests in robot framework

jasson99 — Mon, 06 Apr 2020 10:30:50 +0000

Robot Framework

Robot framework is a generic open source test automation framework for acceptance testing. It is a keyword-driven testing framework that uses tabular test data syntax. It has easy syntax, using human-readable keywords. This framework is independent of operating system and application. The core framework is implemented using Python and also runs on Jython(JVM) and IronPython(.NET). This framework provides support for external libraries, tools which are open source and can be used for test automation.

The test data is in simple tabular format. When started, the robot framework processes the data, executes the test cases, and generates logs and reports.

Installation and setup of robot framework

Preconditions:
- Python installation
- Pip
- make virtual environment
Installing robot framework with pip
- python -m pip install robotframework
- python3 -m pip install robotframework

After the successful installation, you should be able to execute the created runner scripts (robot and rebot) with --version option and get both Robot Framework and interpreter versions as a result:

    $ robot --version
    Robot Framework 3.0 (Python 2.7.10 on linux2)

    $ rebot --version
    Rebot 3.0 (Python 2.7.10 on linux2)

Upgrade[optional]
- pip install --upgrade robotframework
Using robot and rebot scripts

Starting from Robot Framework 3.0, tests are executed using the robot script and results post-processed with the rebot script:

    robot tests.robot
    rebot output.xml

Installing required libraries
- pip install requests robotframework-selenium2library robotframework-pageobjectlibrary
Installation of IDE
- Install pycharm or any preferred IDE
Import required libraries in the IDE. In pycharm,
- To add the project interpreter, Go to Files => Settings => Project: <your_Project> => Project Interpreter and then add the python version available through your virtual environment.
- Also add the required libraries such as : Selenium, robotframework-seleniumLibrary, robotframework-pageObjectLibrary.
Make a robot file inside a test folder eg: robotBDDExample/test/login.robot

Test Data Sections

The data used is defined in different sections in robot framework. These sections are often called tables.

Settings: This section is used for importing test libraries, resource files and variable files.
Variables: The variables that can be used in other parts of the test data are defined in this section.
Test Cases: In this section, test cases are created from available keywords.
Tasks: In this section, tasks are created using available keywords.
Keywords: User keywords are created using existing lower-level keywords.
Comments: This section contains additional comments or data which are ignored by robot framework.

These sections are recognized by their header row. The recommended header format is *** Settings ***. However, the header is case-insensitive, the surrounding spaces are optional, and the number of asterisk characters can vary as long as there is one asterisk at the beginning.

Robot framework test cases can be written in space separated plain text format. In a space separated format, two or more spaces are used as a separator between different data items.

*** Settings ***
Documentation    Example using the space separated plain text format.
Library          OperatingSystem

*** Variables ***
${MESSAGE}       Hello, i am robo!

*** Test Cases ***
My Test
    [Documentation]    Example robot test
    Log    ${MESSAGE}
    My keyword    /lorem.txt

Another Test
    Should Be Equal    ${MESSAGE}    Hello, i am robo!

*** Keywords ***
My Keyword
    [Arguments]    ${path}
    File Should Exist    ${path}

In order to make the separations more clear, pipe separated format can be used.

| *** Settings ***   |
| Documentation      | Example using the pipe separated plain text format.
| Library            | OperatingSystem

| *** Variables ***  |
| ${MESSAGE}         | Hello, i am robo!

| *** Test Cases *** |                 |                   |
| My Test            | [Documentation] | Example robo test |
|                    | Log             | ${MESSAGE}        |
|                    | My Keyword      | /lorem.txt        |
| Another Test       | Should Be Equal | ${MESSAGE}        | Hello, i am robo!

| *** Keywords ***   |                    |         |
| My Keyword         | [Arguments]        | ${path} |
|                    | File Should Exist  | ${path} |

From the above examples, it is clear that the test cases are created in the test case tables from the available keywords. Keywords can be imported from the test libraries or resource files, or created in the keyword table of the test case itself.

Let us now see a practical example where a user logs in.

 *** Settings *** 
 Documentation       Test for user login
 Library             SeleniumLibrary
 Library             PageObjectLibrary


Test Setup           Open test browser
Test Teardown        Close all test browsers

 *** Variables ***  
 ${SERVER}           localhost:8080
 ${ROOT}             http://${SERVER}/myApp
 ${BROWSER}          chrome
 ${SELENIUM_URL}     http://localhost:4444
 ${PLATFORM}         linux
 ${VERSION}          latest
 ${USERNAME}         admin
 ${PASSWORD}         password
 ${username_field}   id=usernametext
 ${password_field}   id=passwordtext
 ${signIn_Button}    id=submit
 ${LOGIN_URL}        http://${SERVER}/myApp/login
 ${WELCOME_URL}      http://${SERVER}/myApp/welcome.html

 *** Test Cases ***                       
 Valid Login      
    [Documentation]       Test valid login  
    Open Login Page      
    Input Username        ${USERNAME} 
    Input Password        ${PASSWORD} 
    Submit Credentials   
    Welcome page is open 

*** Keywords ***
Open test browser
   Open browser  ${ROOT}  ${BROWSER}
    ...  remote_url=${SELENIUM_URL}/wd/hub
    ...  desired_capabilities=browserName:${BROWSER},version:${VERSION},platform:${PLATFORM}

Open Login Page
   Go To        ${LOGIN_URL}
   Login Page Should Be Open

Login Page Should Be Open
   Title Should Be    Login Page

Input Username
   [Arguments]    ${USERNAME}
   Input Text    ${username_field}    ${USERNAME}

Input Password
   [Arguments]    ${PASSWORD}
   Input Text    ${password_field}    ${PASSWORD}

Submit Credentials
   Click Button    ${signIn_Button}   

Welcome Page Is Open
   Location Should Be    ${WELCOME_URL}
   Title Should Be    Welcome Page

Close all test browsers
    Close all browsers

Now, let us understand the above example. The test case has a scenario for valid user login. This test case uses a number of keywords, which can be either the predefined keywords imported from the libraries, or can be self created using available low-level keywords. In the keyword section, all the self created keywords are defined using available keywords. For example: 'Input Username' is a user created keyword which uses an existing keyword 'Input Text'. Similarly, 'Welcome Page Is Open' is a user created keyword which is created using pre-existing keywords 'Location Should Be' and 'Title Should Be'.

The keyword 'Title Should Be' uses the page title such as 'Welcome Page' and 'Login Page' as in example above. These page titles are defined in the page object classes, which will be discussed below.

Using page objects in robot framework

Page objects provide an additional layer of abstraction for test case creation. Using page objects results in easier maintenance of the tests.

PageObjectLibrary is a Robot Framework keyword library that makes it possible to use the Page Object pattern when testing web pages with the keyword based approach of robot framework. Page Object classes are implemented as standard robot keyword libraries. When you use PageObjectLibrary keywords to go to a page or assert you are on a specific page, the keyword will automatically load the library for that page and put it at the front of the library search order, guaranteeing that the Page Object keywords are available to your test case.

Writing a page object class

Page Objects are simple python classes that inherit from PageObjectLibrary.PageObject. There are only a couple of requirements for the class:

The class should define a variable named PAGE_TITLE
The class should define a variable named PAGE_URL which is a URI relative to the site root.

By inheriting from PageObjectLibrary.PageObject, methods have access to the following special object attributes:

self.selib : a reference to an instance of SeleniumLibrary. With this you can call any of the SeleniumLibrary keywords via their python method names (eg: self.selib.input_text)
self.browser : a reference to the webdriver object created when a browser was opened by SeleniumLibrary. With this you can bypass SeleniumLibrary and directly call all of the functions provided by the core selenium library.
self.locator : a wrapper around the _locators dictionary of the page. This dictionary can contain all of the locators used by the Page Object keywords. self.locators adds the ability to access the locators with dot notation rather than the slightly more verbose dictionary syntax (eg: self.locator.username vs self._locators["username"].

The above tests can be easy to understand when written in gherkin format. Let's get familiar with gherkin syntax and how the test can be written in this format.

Gherkin Syntax

Gherkin format focuses on describing a feature to be implemented using the "Given", "When", "Then", "And", and "But" keywords. Writing requirements in this manner makes tests easier to understand, specially for non-technical people, as natural language is used to describe the test cases. Moreover, this format focuses on a clear separation between test-setup, test-actions, and test-results.

Given < some initial state or preconditions >

When < the action taken which triggers the scenario >

Then < the expected outcome >

for e.g.

 *** Settings *** 
 Documentation       Test for user login
 Library             SeleniumLibrary
 Library             PageObjectLibrary

Test Setup           Open test browser
Test Teardown        Close all test browsers

 *** Variables ***  
 ${SERVER}           localhost:8080
 ${ROOT}             http://${SERVER}/myApp
 ${BROWSER}          chrome
 ${SELENIUM_URL}     http://localhost:4444
 ${PLATFORM}         linux
 ${VERSION}          latest
 ${USERNAME}         admin
 ${PASSWORD}         password
 ${LOGIN_URL}        http://${SERVER}/myApp/login
 ${WELCOME_URL}      http://${SERVER}/myApp/welcome.html

 *** Test Cases ***                       
 Valid Login      
    [Documentation]                                     Test valid login  
    Given the user has browsed to the login page      
    When the user enters the username                   ${USERNAME} 
    And the user enters the password                    ${PASSWORD} 
    And the user submits the credentials  
    Then the current page should be  FilesPage

For clear separations and abstraction layers, we make context files for the test steps. Let us now create a contexts directory where all the context files are located. For the login step, let us create a login context file contexts/LoginContext.py.

Note that: Prefixes Given, When, Then, And and But are dropped when matching keywords are searched, if no match with the full name is found. This works for both user keywords and library keywords

from robot.api.deco import keyword

from LoginPage import LoginPage


class LoginContext:
    loginPage = LoginPage()

    @keyword(name="the user enters the username '${USERNAME}'")
    def enter_username(self, username):
        self.loginPage.enter_username(username)

    @keyword(name="the user enters the password '${PASSWORD}'")
    def enter_password(self, password):
        self.loginPage.enter_password(password)    

    @keyword(name="the user has browsed to the login page")
    def browse_to_login_page(self):
        self.loginPage.browse_to_page()

    @keyword(name="the user submits the credentials")  
    def submit_credentials(self):
        self.loginPage.submit_credentials()

The login context file uses the methods of the login page. So, we must write every needed functions in the login page. Let us create a page objects directory 'page_objects' and then a login page 'page_objects/LoginPage'.

from PageObjectLibrary import PageObject, PageObjectLibraryKeywords


class LoginPage(PageObject):
    PAGE_URL = "/myApp/login"
    PAGE_TITLE = "myApp"

    _locators = {
        "username": "id=user",
        "password": "id=password",
        "form_id": "login",
        "login_form_xpath": "//form[@name='%s']"
    }

    keywords = PageObjectLibraryKeywords()

    def browse_to_page(self):
        self.keywords.go_to_page('LoginPage')

    def enter_username(self, username):
        """Type the given text into the username field """
        self.selib.input_text(self.locator.username, username)

    def enter_password(self, password):
        """Type the given text into the password field"""
        self.selib.input_text(self.locator.password, password)

    def submit_credentials(self):
        """Clicks the submit button on the form
        For illustrative purposes, this uses the low level selenium
        functions for submitting the form
        """
        form = self.driver.find_element_by_xpath( self.locator.login_form_xpath % self.locator.form_id)
        form.submit()

We create all other context files and page objects as per requirements. All the methods and elements related to a certain page are included in a specific page object and its own context file. This makes tests easier to understand and maintain.

FilesPage can be created in a similar manner as follows:

from PageObjectLibrary import PageObject


class FilesPage(PageObject):
    PAGE_URL = "/index.php/apps/files/"
    PAGE_TITLE = "Files - myApp"

Example Code:

https://github.com/JankariTech/robotBDDExample

Datatables used in a gherkin feature file

jasson99 — Thu, 20 Feb 2020 08:49:31 +0000

As we are familiar with the basic gherkin syntax such as feature, scenario, Scenario Outline, background, given, when and then steps already, let us discuss about the table or tablenodes used in the steps of a gherkin feature file.

Tables

Tables or tablenodes or Datatables are used for specifying a larger data set as an argument in the steps of a scenario in a feature file. These tables are handy for passing a list of values to a step definition. The data tables are passed to the step definition of a specific step as its last argument. Let us understand with the help of an example below.

A scenario for logging in without using a data table

 Scenario: login with valid credentials  
   Given a user has been created with email "user@email.com" and username "user" and password "password"
   And the user has browsed to the login page
   When the user enters email "user@email.com" and username "user" and password "password" in the login form
   And the user logs in
   Then the user should be redirected to the homepage

There are cases where a large amount of data needs to be set in the steps. It can be done using a number of "ands" in the step text as shown in the example above. However, good practice would be to use data tables to provide the data.

A scenario for logging in using a data table

  Scenario: login with valid credentials
    Given a user has been created with the following details:
       | email          | username  | password |
       | user@email.com | user      | password |
    And the user has browsed to the login page
    When the user enters the following details in the login form:
       | email          | username  | password |
       | user@email.com | user      | password |
    And the user logs in
    Then the user should be redirected to the homepage

Please do not confuse data tables with the example table used in a scenario outline. Even if they look alike, they have completely different purposes. Please refer to this blog to learn more about the Scenario Outline: https://dev.to/jankaritech/scenario-outline-in-gherkin-feature-file-16jh

Data tables

Unlike the examples table, a table node provides all of the data in the data table at once, in the specific step where the table is provided.

Let us look at another example of a data table. To be more clear about its significance, let us first write a scenario without using a data table as below.

Scenario: view and delete files
  Given user "testuser" has been created
  And user "testuser" has logged in
  When the user browses to the files page
  Then file "myCollegeProject" should be listed
  And file "interviewQuestions" should be listed
  And file "picnicCollections" should be listed
  And file "presentation" should be listed
  When the user deletes file "myCollegeProject"
  And the user deletes file "interviewQuestions"
  Then file "myCollegeProject" should not be listed
  And file "interviewQuestions" should not be listed
  But file "picnicCollection" should be listed
  And file "presentation" should be listed

When observed closely, we can see that the listing of several files, continuous deletion of several files, non-listing of deleted files and listing of non-deleted files has been written using several repetitive steps. The steps seem to be difficult to maintain this way. Moreover, more effort is required to understand the scenario when reading it. To improve this, we can use the data tables. The example below illustrates the use of data tables in the steps where multiple data are to be passed at once. For example, multiple files are to be listed, multiple files are to be deleted, and again multiple files are expected not to be listed and to be listed respectively.

 Scenario: view and delete files
   Given user "testuser" has been created
   And user "testuser" has logged in
   When the user browses to the files page
   Then the following files should be listed:
      | files              |
      | myCollegeProject   |
      | interviewQuestions |
      | picnicCollections  |
      | presentation       |
   When the user deletes the following files
      | files              |
      | myCollegeProject   |
      | interviewQuestions |
   Then the following files should not be listed:
      | files              |
      | myCollegeProject   |
      | interviewQuestions |
   But the following files should be listed:
      | files              |
      | picnicCollections  |
      | presentation       |

The data set provided using the data table should be hashed in the step definition of the particular steps in order to transform the cucumber data table into either an array of data or object.

for example:
In nightwatch, to transform a cucumber data table to an array, we use hashes

   When('the user deletes the following files', (dataTable) {
    const filesToDelete = dataTable.hashes()
    console.log(filesToDelete);
    /* It returns : 
    [ {files: 'myCollegeProject'},
      {files: 'interviewQuestions'} ]
    */

    for (const file in filesToDelete) {
        this.delete(file)
   }
   })

Now, to transform a cucumber data table to an object, we use rowsHash. However, rowsHash can only be called on a data table where all rows have exactly two columns.

for example:

When the user sets the price for the following items
   | book      | 500 |
   | sharpener | 30  |
   | pencil    | 15  |

The step definition for this step can parse the data table using rowsHash as follows:

When('the user sets the price for the following items', (dataTable) {
  const priceOfItems = dataTable.rowsHash()
  console.log(priceOfItems)
    /* It returns : 
    { book : '500',
      sharpener : '30',
      pencil : '15' }
    */
})

Let us see what happens if we use hashes for this example.

When('the user sets the price for the following items', (dataTable) {
  const priceOfItems = dataTable.hashes()
  console.log(priceOfItems)
   /* It returns : 
   [ { book : 'sharpener', '500' : '30' },
     { book : 'pencil', '500' : '15'} ]
   */ 
})

However, if we provide data in the steps as follows, then hashes can be used. for example:

When the user sets the price for the following items
   | item      | price |
   | book      | 500   |
   | sharpener | 30    |
   | pencil    | 15    |


  When('the user sets the price for the following items', (dataTable) {
  const priceOfItems = dataTable.hashes()
  console.log(priceOfItems)
   /* It returns : 
   [ { item : 'book', price : '500' },
     { item : 'sharpener', price : '30' },
     { item : 'pencil', price : '15'} ]
   */ 
})

### Combination
It is also possible to combine data tables with a scenario outline as well when needed. Let us see an example below:

   Scenario Outline: User tries to signup with improper combination of password
     Given the user has browsed to the signup page
     When the user tries to signup entering the following details
        | email           |  william@xyz.com   |
        | password        | <Password>         |
        | confirmPassword | <ConfirmPassword>  |
     Then an error message "<validation>" should be shown above the "password" field
     Examples:
        | Password    | ConfirmPassword  | validation                         |
        | 234567569   | 234567569        | This password is entirely numeric  |
        | 123456789   | 123456789        | This password is too common.       |
        | abcde       | abcde            | This password is too short.        |

The values in the examples table get substituted into the steps which require the parameter values. The substitution takes place one row at a time. The values from the data table are passed to its step definition at once. For the above example, since there are 3 data rows in the examples table, this scenario runs thrice with a specific combination of data values as provided in the example table. Let us be clear about how the above example executes.

In the first scenario, a set of data is used from the first row of the examples table, as below.

Given the user has browsed to the signup page
When the user tries to signup entering the following details
    | email           | william@xyz.com    |
    | password        | 234567569          |
    | confirmPassword | 234567569          |
Then an error message "This password is entirely numeric" should be shown above the "password" field

For the step definition of when step, the data table is parsed as follows:

When('the user tries to signup entering the following details, (dataTable) {
const userDetails = dataTable.rowsHash()
  /* It returns: 
        { email : 'william@xyz.com',
          password : '234567569',
          confirmPassword : '234567569' }
  */
  return client.setValue(emailField, userDetails['email'])
               .setValue(passwordField, userDetails['password'])
               .setValue(confirmPasswordField, userDetail['confirmPassword'])
               .click(signupButton)
})

In the next run of the scenario, the next single row of data is used as follows.

Given the user has browsed to the signup page
When the user tries to signup entering the following details
   | email           | william@xyz.com    |
   | password        | 123456789          |
   | confirmPassword | 123456789          |
Then an error message "This password is too common" should be shown above the "password" field

Similarly, the third run of the scenario uses the third row of the examples table as follows.

Given the user has browsed to the signup page
When the user tries to signup entering the following details
   | email           | william@xyz.com    |
   | password        | abcde              |
   | confirmPassword | abcde              |
Then an error message "This password is too short" should be shown above the "password" field

Scenario Outline In Gherkin Feature File

jasson99 — Thu, 20 Feb 2020 06:26:20 +0000

As we are familiar with the basic gherkin syntax such as feature, scenario, background, given, when and then steps already, let us discuss about the Scenario Outline used in a Gherkin feature file.

Scenario Outline

Let us start with a very simple feature where the remaining candies should be calculated based on the total candies and the candies consumed. However, there can be different scenarios for the different combinations of total candies, consumed candies, and the remaining candies.

 Scenario: calculate remaining candies when all candies are consumed
   Given I have 10 candies
   When I eat 10 candies
   Then I should have 0 candies remaining


 Scenario: calculate remaining candies when some candies are consumed
   Given I have 60 candies
   When I eat 23 candies
   Then I should have 37 candies remaining


 Scenario: calculate remaining candies when no candies are consumed
   Given I have 93 candies
   When I eat 0 candies
   Then I should have 93 candies remaining

It can be very time consuming to write repetitive scenarios with different permutations of data values. It can also be difficult for the reader to understand many separate but similar scenarios. This can be improved by using the scenario outline and its example table, thus combining similar scenarios with different combinations of similar data values into one Scenario Outline. A Scenario Outline is always followed by an example table: Examples. The example table in scenario outline is used to combine multiple similar scenarios into a single scenario in the feature file. It provides one set of data per scenario. Each new row of the example table is run as a different scenario. The data values in a single row of data are passed to the step definition at the run time of a scenario.

A scenario outline replaces variables with the value from the examples table. Each row in the examples table is considered to be a scenario.

In the scenario outline, the data values do not need to be hardcoded in the step definition. Rather the values are replaced with the name of the parameter itself.

In the above example of candies, if we observe closely, all three scenarios have the same statements, only the parameter value (total/consumed/remaining number of candies) is changing. This is where the significance of the scenario outline comes into the picture.

When we define a scenario using a scenario outline, we can specify a single test scenario for all the similar scenarios and then provide an example table at the end of the test scenario. The test scenario is always specified by Scenario Outline and is always followed by a table: Examples. This scenario will thus be executed as many times as the number of data inputs (data rows) provided in the examples table

 Scenario Outline: calculate remaining candies
   Given I have <total> candies
   When I eat <eaten> candies
   Then I should have <remaining> candies
   Examples:
     | total | eaten | remaining  |
     | 10    | 10    | 0          |
     | 60    | 23    | 37         |
     | 93    | 0     | 93         |

Now, let us continue with more realistic scenarios. The scenarios of user login with valid credentials (login for different valid users) can be implemented using Scenario Outline as follows:

 Scenario Outline: login with valid credentials
   Given a user has been created with email "<email>" and password "<password>"
   And the user has browsed to the login page
   When the user enters email "<email>" and password "<password>" in the login form
   And the user logs in
   Then the user should be redirected to the homepage
   Examples:
     | email            | password     |
     | user1@email.com  | firstpass    |
     | second@email.com | password12   |
     | test@gmail.com   | testpassword |

Here, each row of the example table provides the email and password used in the given and when steps. Even if this scenario looks like a single scenario, it runs 3 times in this example with different email and password set in the step definition, one at a time. The first scenario takes the first row of email and password and completes the test (hopefully passed). Then the 2nd scenario takes the data in the second row independent of other data present in other rows. After this is completed, it's turn for the 3rd row of data sets.

Let us be more clear. The above scenario outline consists of 3 similar scenarios which run uniquely with their own set of data. Let us see how it happens.

The first row of data acts as first scenario and runs as :

  Given a user has been created with email "user1@email.com" and password "firstpass"
  And the user has browsed to the login page
  When the user enters email "user1@email.com" and password "firstpass" in the login form
  And the user logs in
  Then the user should be redirected to the homepage

Similarly, the next two scenarios run as below respectively.

  Given a user has been created with email "second@email.com" and password "password12"
  And the user has browsed to the login page
  When the user enters email "second@email.com" and password "password12" in the login form
  And the user logs in
  Then the user should be redirected to the homepage     


  Given a user has been created with email "test@gmail.com" and password "testpassword"
  And the user has browsed to the login page
  When the user enters email "test@gmail.com" and password "testpassword" in the login form
  And the user logs in
  Then the user should be redirected to the homepage

To sum up, when there are cases where the scenarios are similar with same statements but with varying data values as parameters, it is advisable to use Scenario Outline with different sets of values provided through the examples table.