Forem: Sadia Islam

Automating CSV Data for Reliable API Testing

Sadia Islam — Thu, 26 Jun 2025 07:47:19 +0000

Recently, I tackled a practical challenge in my API automation work and wanted to share both the problem and my solution — in case it helps someone out there in a similar boat!

🚧 The Problem
In my current project, one of our APIs is responsible for onboarding users through the upload of CSV files. This API checks the uploaded data against existing backend records and rejects the entire upload if it detects any duplicates

Now, here comes the tricky part: My test needs to run repeatedly (automated), but each time it runs with the same CSV file, the API fails because duplicate entries already exist in the system.

Two workarounds were initially proposed:

Manually upload a new CSV file every time (not scalable)
Reset the database daily (risky and not practical for CI/CD)

I knew we could do better.

💡 My Thought Process
Instead of changing the whole database or managing tons of files manually… Why not generate unique data programmatically for every test run?

That way:

The API sees fresh data every time
The test runs on schedule (hands-free!)
No need to reset the DB or manage uploads manually

🛠️ The Solution I Built
I created an R&D project to build an automated solution that:

Create a dummy API locally that mimics the real endpoint behavior (CSV upload, duplicate checks). Later, I deployed it on a dummy server to test the flow in a CI/CD environment.
Used faker.js to dynamically generate unique user data (ID, Name, etc..)
Write that data into a temp.csv file using Node.js
Use Postman to send a form-data request with the CSV
Executed the API request via Newman to upload the CSV file.
Runs in GitHub Actions every 30 minutes or is triggered manually as needed.
Sends me the test result report in Telegram, with a full HTML report with pass/fail status

⚙️ Technologies Used:

Node.js for scripting
faker.js for generating fake (but unique) data
Newman to run Postman tests in CI
GitHub Actions for automation
Telegram Bot API for real-time notifications

The Result

✅ Fully automated test
📂 Always fresh and valid data
🧪 No duplicates = no failed tests due to data
📲 Real-time result delivery
💥 100% hands-off CI/CD experience

My Key Learnings

Automating dynamic test data prevents flaky API test failures due to duplicates.
Manual work, like resetting databases or uploading new files daily, isn't scalable
Real-time Telegram alerts provide quick visibility into test outcomes [you can use Gmail, Slack, or any other platform to get the real-time update]
A simple R&D initiative can lead to valuable production-ready solutions.

What’s Next?
This R&D setup is now complete and validated. I’ll be integrating it into our production test automation pipeline soon.

LIVE API + Repository Link
☑️API Endpoint: https://playing-with-api.onrender.com/docs/

📂Github Repo: https://github.com/Sisadia/playing-with-api

Why I'm Sharing
This might seem like a small challenge, but it's a real-world pain point in test automation — and solving it with code gave me the flexibility to focus on testing logic rather than data cleanup.

Happy testing! ✨

Why Your API Call is Hitting Twice: Understanding Preflight & XHR

Sadia Islam — Thu, 26 Jun 2025 07:41:30 +0000

I often notice in the Network tab of DevTools that the same API is being called twice. Even though it’s the same API, the types of requests are different — one is preflight, and the other is xhr. Curious about why this happens, I found out that this usually occurs due to CORS (Cross-Origin Resource Sharing). Let’s try to understand what these things are one by one.

What is XHR?

XHR stands for XMLHttpRequest, which is the actual API call that sends the data request. This is the main API call where the client requests data from the server.

What is a Preflight Request?

When an API call is made that isn’t considered simple, meaning it doesn't meet certain conditions, the browser first sends a small request called a preflight or permission check request. This is typically an OPTIONS request that asks:

Will this API accept a request from this origin?

If the server responds with the appropriate Access-Control-Allow-* headers indicating permission, saying “Yes, you can call me with those headers/credentials” — then the browser proceeds to send the actual xhr call.

How does it work?

The browser first checks “Does this API call require a preflight?”
If needed, it sends an OPTIONS (preflight) request first.
If the server replies "Okay, go ahead", then the browser sends the XHR request.
If the server denies permission, the XHR request is never sent — the browser blocks it.

I believe this is important for QAs to know because:

It helps them understand that this is not a bug.
It aids in understanding security or permission checks: Preflight helps verify whether CORS permissions are set up correctly.
It allows for testing headers: If QAs know how preflight works, they can check whether the server is correctly responding with headers like Access-Control-Allow-Origin, Access-Control-Allow-Headers, etc.

The Common Struggle of Automation Testing: Programming Knowledge

Sadia Islam — Tue, 04 Jun 2024 14:10:28 +0000

You’ve probably been thinking about exploring the field of automation testing for a long time. You even started working on it according to your thoughts. But when you started working, you got into trouble. This problem has been faced by more or less everyone, including me, you, and us. So let’s share some of my limited knowledge about this familiar problem with you.

As an automation tester, having a strong foundation in programming is very important. This strong foundation will help you design, write, and maintain automation scripts beautifully. You will need to know some Core Concepts of programming. These are:

1. Syntax and Familiarity with data types, variables, operators, and expressions.
2. Conditional statements (if-else).
3. Looping constructs (for, while).
4. Functions and Methods -Function Invocation, Parameters and Return Values, local and global scope
5. Arrays and Lists
6. Object-Oriented Programming (OOP) — Classes and Objects, Inheritance, Polymorphism, Encapsulation, Abstraction
7. Exception handling mechanisms (try-catch/finally blocks/Custom Exception).

If you have a good understanding of these Core Concepts, you can write automation scripts using any programming language. If you want, you can also take ideas about Advanced Concepts.

Host your Automation Allure Report on GitHub Pages with GitHub Actions.

Sadia Islam — Sat, 27 May 2023 05:07:52 +0000

We all have heard of or used Allure Report in our day-to-day life. Additionally, GitHub Actions and GitHub Pages are also commonly used by many people in their daily lives. Will integrate all of them and learn something new. First of all, let's recap what these three are.

Allure Report

An Allure report is an open-source framework designed to create interactive and visually appealing test reports. These reports are generated automatically by test automation frameworks such as Selenium, Appium, and TestNG.

GitHub Pages

GitHub Pages is a static site hosting service offered by GitHub that allows developers to publish web content directly from a GitHub repository. It takes HTML, CSS, and JavaScript files.

GitHub Actions

GitHub Actions is a platform that allows developers to automate software workflows related to building, testing, and deploying their code. It can be used to automate the process of generating and publishing Allure reports on GitHub Pages.

In this document, you will learn how to host your automation Allure report on GitHub Pages using GitHub Actions.

When using GitHub Pages, you receive a subdomain in the format of github.io for hosting your website. In order to host your site, the first requirement is to have a GitHub account and a repository containing the automation code to execute the test cases. Your site will be named as <username>.github.io/<repository name>

It's important to note that the repository needs to be public in order to host a site with GitHub Pages.

Now, let's talk about how it can help us. As an automation tester, we know how frustrating it is to share our daily progress reports with management. Usually, we use tools like Extent Report or Allure Report and send these reports by email to show how our automation work is going, whether it's happening every day or every week.

But there's a problem with using Allure Report. We have to send a whole folder called "allure-results" and run a command to see it, which is a lot of extra work. But what if we didn't have to go through all that trouble? We can use GitHub Pages instead. We can put our automation reports there and just give management a link. So whenever they want to see our latest progress, they can easily click on the link and check it out. Isn't that amazing?

Now, let's get started and explore it further.

In this demonstration, I will present a project that utilizes the following technical stack along with the use of GitHub Actions.

Now, let's take a look at the entire process step by step.

Step 1:

Open up a IDE of your choice and setup a Maven Project.
Add all the required dependencies as mentioned above in your pom.xml file.
Write you tests in a Java Class.

4.Run your test file.

After successfully running your test, you will see a folder named allure-results.

Step 2:

After you finish writing your code, you should upload it to GitHub. If you don't have a GitHub account, you need to create one first. Once you have an account, create a new repository and then upload your code to GitHub.

Step 3:

1.Go to Action Tab. You have two options: either create a workflow using the suggested one, or set up your own workflow. Since you're using the Maven build tool, if you choose the suggested option, make sure to select Java with Maven.

2.You can choose to click on the setup workflow either by yourself or using Java with Maven. Change the file name to whatever you want (I changed it to ci.yml) and copy and paste the provided code.

As you see above what we are doing is whenever there is a new push or Pull request to main branch this workflow will get execute. In the workflow we are setting up JDK 11 which will run on ubuntu and then execute our test script with the help of maven.

Ensure that the branch you are using matches the one where you uploaded your code. I uploaded my code to the main branch, so the displayed value is "main." If the branches don't match, it won't function properly and the build will fail.

3.Once you've pasted the code, commit the changes. Then, navigate to the action tab where you will find the workflow you created. You'll notice that the workflow is set to run automatically. Once it finishes running, you will receive an error message. If you select the workflow with the error, you will be able to view the error message or information it is displaying.

Okay, How can I resolve this problem?

Navigate to the Repository Settings, Action → General → Check Read and Write Permissions option, and save the changes.

4.Next, navigate back to the Action tab and select the workflow. Then, click on it to rerun the job that previously failed. You will be able to observe the build process completing successfully.

5.Now, when you open the code tab of your repository, you will find two branches. Initially, there was only one branch. The second branch is for the GitHub pages of the report, where it stores the report and its history. To proceed further, go to the settings of your repository and then click on "Pages." Choose the gh-pages branch and save your selection by clicking on the save button.

6.If you go to the action tab, you will find that the "pages build and deployment" workflow is currently running. Once the deployment is successful, click on the workflow, and you will find the link to your GitHub pages where the allure report is available. By clicking on that link, you will be able to view the report.

If you now add test cases and push the new code to your repository, after the workflow finishes running, you will be able to see the updated report at the same link.