Forem: Charanraj

JSON schema validation in Postman using external JSON files

Charanraj — Mon, 05 Aug 2019 14:03:39 +0000

Postman is an excellent tool for testing APIs. The response for an API call can be validated against a JSON schema to ensure that the values are valid in terms of type and format. In this article, we will explore how external JSON files can be loaded into Postman to avoid duplication of schema, in turn, making the tests more maintainable.

Postman does not (as of v7.3.4) support the ability to reference external libraries and files. Which means that we will have to get the schema loaded in the environment variable to parse and use it.

Primer on JSON schema & Tiny Validator library

Before we begin, it would be wise to have some understanding of JSON schema and Tiny validator library which is used for the schema validation. For the purpose of demonstration, we have four entities Campaign, Media, Sponsor, and MediaType. Campaign requires an object of Media and optionally a Sponsor. Sponsor requires an object of Media.

The following are the schemas for the above-listed entities.

common-schema.json — Contains the schema for common elements that are to be reused across the entire application
advertisement-schema.json — Contains schema for all elements corresponding to a module called advertisement in the application.

Figure-1 common-schema.json

View on GitHub: https://gist.github.com/charanraj-golla/82bd2e51733cec512aab8f2d1e2713e0

Figure-2 advertisement-schema.json

View on GitHub: https://gist.github.com/charanraj-golla/1823c488895c177c4c449c4cbc11c407

Following methods from tiny Validator can be used to validate the results from an API response:

tv4.validate()
tv4.validateResult()
tv4.validateMultiple()

These methods accept four parameters:

data
schema
checkRecursion
banUnknownProperties

The first two are mandatory without which the validation won’t work. Passing true for checkRecursion will recursively validate the nested schema. Setting banUnknownProperties to true will result in a schema error if there are any additional properties apart from those which were defined in the schema. This is the bare minimum about this library that we must know for writing the tests.

Next, we need to have some idea about $ref and $id keywords in JSON Schema. $ref gets logically replaced with the thing that it points to. This referenced section can be within the same file or in an external file. The value of $ref is a URI, and the part after # sign (pound sign) is “the fragment” or “named anchor” and is in a format called JSON Pointer.

Next in line is the $id property. The value of $id is a URI that serves two purposes:

It declares a unique identifier for the schema. (Figure-4)
It declares a base URI against which $ref URIs are resolved. (Figure-5)

$id given to a subschema, in this case, the address provides a way to refer to subschema without using JSON Pointer. This means you can refer to them by a unique name as shown in the below example (Figure-4).

Figure-4 Unique identifier in a schema.

In the case where $id is used to provide the base URI, a subschema present in the same file can be referenced as "$ref" : "#/definitions/sponsor" which essentially means :

Go to the root of this document
Find value with the key definition
Within that object find the value of the key sponsor

Figure-6 Using JSON pointer

Going back to our example, Media property in Campaign object expects an array of objects of the type Media. The schema for the array item must be specified against the items property. The schema for media is present in a separate file named common-schema.json (Figure-1). Here we are using absolute URI along with JSON pointer. The intent of using an absolute URI will be clear once we start writing the tests. Note that Postman does not make any network calls to the given URI.

Even though the value of a $ref is a URI, it is not a network locator, only an identifier. This means that the schema doesn’t need to be accessible at that URI, but it may be. It is basically up to the validator implementation how external schema URIs will be handled, but one should not assume the validator will fetch network resources indicated in $ref values.

Setup for validation

The schema needs to be loaded into Postman’s environment variable before running tests for the collection. Include all schemas in the solution and add a controller for reading the schema files.

Figure-7 Visual Studio solution setup

Add a folder to the collection right before any other requests are made and add requests to GetSchema action method.

Figure-8 Test setup in Postman

In test tab for each request in the TestSetup folder, save the response to a unique environment variable. Paste the following code in the tests tab:

This setup gives you full IDE support for editing the JSON files and saves the time you would have spent trying to edit the JSON string in the single-line text box every time there were changes are made to the schema.

Figure-9 Environment variable set after GET request was executed

Finally, in order to validate the schema, all the required schemas which are currently available in the environment variables (stored as a string) must be parsed to JSON and registered in the Tiny Validator. This can be done using tv4.addSchema(). It accepts two parameters URI and schema object. An important point to note is that the URI used for registering the schema overrides the value specified for $id key in the JSON file while registering the schema. This means that the URI you use to register the schema in tv4.addSchema() function must be used while specifying the JSON pointer. When all the required schemas are registered using the tv4.addSchema() function, any references in the schema will be automatically resolved.

A response body similar to the one listed below would successfully pass the test.

tv4.validate() will stop at first error and return a false value. It would be ideal to have a list of all errors for a request. tv4.validateMultiple() returns an array of validation result which can be iterated over to find the error details for a property. Remember to keep the console (Ctrl + Alt + C) window open when you are running the test.

To verify if the schema validation really works as expected we can alter the response making it invalid. Value for alternateName in sponsor → media is changed to a number instead of a string and an unknown property called aBadField is added as well.

The validation fails and the postman console logs the exact path and the cause of the error.

Test fails due to invalid property values

Exact path of error logged in Postman console

Hope this post saves you some time while testing your APIs. I would appreciate your constructive feedback in improving this post. Got any handy tips on validating the APIs? Let's discuss :)

This article was originally published on SkillHive Blog

Programming for absolute beginners

Charanraj — Sat, 14 Oct 2017 10:50:53 +0000

Wanted to learn computer programming but not sure where to start? In this video series, I try to introduce you to basic concepts that you would find in almost any programming languages.
There is no pre-requisite for this course. This course is independent of any programming languages and focuses purely on building up the required foundation to get you started with almost any programming language.
We will be covering the following topics in this playlist:

What is a computer program & how does it work?
Abstract programming concepts — Pseudo code & Flowcharts
Syntax — The rules of programming languages
Working with data
Program flow
Introduction to Object Oriented Programming [Article]

Introduction

What is a computer program & how does it work?

Abstract programming concepts — Pseudo code & Flowcharts

Syntax — The rules of programming languages

Working with data

Program flow

This article was originally published on Skill Hive Blog

Beginner's guide - Object Oriented Programming

Charanraj — Fri, 04 Aug 2017 17:09:26 +0000

Complex real life requirements need to be model into instructions for the computer to understand. We have a few paradigms like Procedural Programming, Object oriented programming, Functional Programming etc. which can help in modeling the business requirements into computer programs. We will be learning about one such paradigm called Object oriented programming. OOP is a philosophy and is language neutral.

We will be taking a look at the four concepts which act as the pillars for Object Oriented Programming.

Abstraction
Encapsulation
Inheritance
Polymorphism

Skip video and continue reading article

What is Object Oriented Programming?

Older programming languages like COBOL and C followed the Procedural Programming approach. The program written using these languages used to be a series of step by step instructions. They made use of procedures/subroutines for making the program modular. This programming paradigm focused on logic more than data and the program used to combine both of them together.

Modern programming languages like Java, C# etc. follow the Object Oriented approach. In object oriented programming, importance is given to data rather than just writing instructions to complete a task. An object is a thing or idea that you want to model in your program. An object can be anything, example, employee, bank account, car etc.

Class, Object.. what's that?

For getting started with object oriented programming we would have to know what is a class and object and the difference between them. A class is a blueprint for creating an object. It is similar to the blue print of a house.

A class defines attributes and behavior. If we are to model a car in our application then we could define attributes of the car like, model, fuel, make and behaviors like start, break, accelerate etc.. If you notice, the attributes and behavior that we are specifying are not specific to just one model of car. We are trying to generalize a car by stating that the car which we are going to model in our program will have these number of attributes and behavior.There might be others as well but our scope and interest for the business requirement are limited to these attributes. This helps us create a blue print of the car and later while we use this class for creating objects we create car objects with specific details.
Example, using the same class "Car" we can create different objects having variation in model, fuel type and make year while having the same common behavior.

Object 1
Model	Volkswagen Polo
Fuel	Petrol
Make	2017
Start() Break() Accelerate()

Object 2
Model	Volkswagen Vento
Fuel	Diesel
Make	2017
Start() Break() Accelerate()

In this way, Object oriented programming allows you to easily model real world complex system behavior. With OOP, data and functions (attributes and methods) are bundled together within the object. This prevents the need for any shared or global data with OOP, which is a core difference between the object oriented and procedural approaches.

Abstraction

Abstraction lets you focus on what the object does instead of how it is done. The idea behind abstraction is knowing a thing on a high level. Abstraction helps in building independent modules which can interact with each other by some means. Independent modules also ease maintenance.

Abstraction means to represent the essential feature without detailing the background implementation or internal working detail.

We try to selectively focus on only those things that matter to us or in the case of programming, to our module. Modifying one independent module does not impact the other modules. The only knowledge one needs to know is what a module gives you. The person who uses that module does not need to bother about how the task is achieved or what exactly is happening in the background.

Abstraction is everywhere. Everyday objects that we use has abstractions applied at various levels. One example is applying breaks in your car or bike. The breaking system is abstracted and you are provided with a paddle for stopping your vehicle. Making changes to acceleration system does not affect the braking system they are independent. You also do not have to bother about the internal working of the brakes, you only have to press the brake pedal and be it disc brake or drum brake, the vehicle stops.

Encapsulation

The second concept Encapsulation is closely related to Abstraction. Encapsulation is all about exposing a solution to a problem without requiring the consumer to fully understand the problem domain.Encapsulation is binding the data and behaviors together in a single unit. This prevents the client or the user of the module from knowing about the inside view where the behavior of the abstraction is implemented.
The data is not accessed directly. It is accessed through the exposed functions. Hiding the internalsÂ of the objectÂ protects its integrityÂ by preventing users from setting the internal data of the component into an invalid or inconsistent state.

Encapsulation isÂ not just about hiding complexity but conversely exposing complexity in a fail-safe manner.

Inheritance

Inheritance is a powerful feature of Object oriented programming languages.Inheritance helps in organizing classes into a hierarchy and enabling these classes to inherit attributes and behavior from classes above in the hierarchy.
Inheritance describes an “IS A relationship. This is how we talk in the real world. Example. A Parrot is a bird. US Dollar is a type of currency. But the phrase, Bank is a bank account is not correct. This relation is obvious when you try to describe some entity in the given business/problem statement.

With inheritance, you can define a common implementation/behavior and later for specialized classes you can alter or change it to something specialized.
Inheritance does not work backward. The parent won’t have properties of the derived class.

Inheritance is a mechanism for code reuse and can help in reducing duplication of code.

Do not force inheritance. You would just be writing unnecessary code. It is important to note that while trying to model the requirement don’t go adding multiple levels of inheritance. This is not needed. You need to try to identify common attributes and behavior in the entities you modeled and based on that you can go ahead re-factoring the code defining a suitable parent class. Common implementation can then be moved to this class.

Polymorphism

Polymorphism is the concept that there can be many different implementations of an executable unit and the difference happen all behind the scene without the caller awareness.
PolymorphismÂ allows computer systems to be extended with new specialized objects being created while allowing current part of the system to interact with a new object without concern for specific properties of the new objects.

For example, if you needed to write a message on a piece of paper, you could use a pen, pencil, marker or even a crayon. You only require that the item you use can fit in your hand and can make a mark when pressed against the paper. So the action of writing would help you make a mark on the paper and what marking or writing instrument to use is a matter of decision. Another example is a plane and space shuttle both can be termed as Flying objects. But the way both fly are different I.e. there is a difference in implementation. But from a viewer’s perspective both the objects can fly.

Inheritance is one means of achieving polymorphism where behavior defined in the inherited class can be overridden by writing a custom implementation of the method. This is called method overriding also known as Run Time polymorphism.

There is also one more form of polymorphism called method overloading where inheritance does not come into the picture. The method name is same but the arguments in the method differ.

Need help in understanding any technical topics? Send in your requests using the following link and I will try to make a post/video for it.
http://bit.ly/RequestCourse

References:

https://stackoverflow.com/questions/16014290/simple-way-to-understand-encapsulation-and-abstraction
http://www.adobe.com/devnet/actionscript/learning/oop-concepts/polymorphism-and-interfaces.html
https://stackoverflow.com/questions/3322318/explain-polymorphism?rq=1

Credit:
Icons from http://www.freepik.com/

Learning from the top 10 mistakes developers make

Charanraj — Thu, 18 May 2017 18:10:48 +0000

I present to you a list of a few common mistakes new devs (and sometimes even experienced) usually do. I believe that learning from these mistakes would be of some help to you. So, here is my pick of the Top 10 mistakes.

10. Trusting user’s input

Never trust the inputs provided by the end user. Also don’t blame them, mostly they do it unintentionally. We all make ~~mistake~~ mistakes. This might not be of great concern to them but a piece of bad data could potentially bring down the application or cause compliance related issues. Apart from naive non-technical users we also have users with a certain level of technical know-how which they could potentially use to circumvent the system and get their work done. These are the users who pose a threat to your application and application needs to be safeguarded against the malicious activities.

The only option to save from landing into trouble at a later stage is to build a strong validation with self-explanatory and human understandable error messages. Cryptic messages hardly help the naÃ¯ve user accomplish any task. Validation should be done in both user interface and database level just to be sure that even if by some means front end validation was turned off the application still does not accept trash values.

9. Manual unit tests

Code changes that you do, needs to undergo unit testing and further rounds of Quality Assurance testing. All tests or at least a few minimum number of tests needs to be done to ensure that the changes you made in one unit of code do not have side effects on the others. This is a necessary evil. Having automated unit tests and integration tests save you a lot of manual efforts in validating side effects of the changes.

8. Skipping documentation

The evilest thing that you can do is skipping documentation. Off course you can read the source code and try to understand the functionality. Reverse engineering the functionality from the code is a daunting task. This is good enough for a tiny functionality but definitely not a go for the entire application.

"Requirement changes, code changes and more frequent - members of development team change."

Knowledge about the application might not be transferred 100%. Sometimes people just forget. You need not create dozens of documents, create only a few like requirement specification and technical document and make sure to keep them updated, this should be a part of a stringent process which needs to be followed at any cost and must be accommodated in the project planning. This would help at later stages when the application is in support/maintenance phase.

7. Forgetting about Audit and Error logging

Not all end user/stakeholder would be bothered about cross cutting concerns like audit logs and error logging. A stakeholder would simply come with a one liner requirement which you would have to drill down till it becomes good enough to work on and deliver a solution addressing the stakeholder’s problem.

Data change audit and error logging might never be a part of user’s requirement but is implicit that it needs to be maintained. One fine day user would simply approach support team and ask them to provide data related to access and roles or some business critical data required for compliance or audit purpose and then you would have nothing to share.

Error logging is also crucial, you might do defensive coding but you never know what might fail in such cases effective error logging comes in handy. Monitoring error logs and trying to work on resolving recurring error would help in making your application more stable.

6. Careless use of privileged access

First and foremost, full control and access should never be provided. The system cannot protect itself from malicious activities if you leak the superuser or privileged account passwords. Risks involved in the reckless usage of privileged access should be a part of learning for inexperienced new members of development or support team. There might be times when you would have to make a modification to production database and having privileged access can lead to many issues.

“With great power comes great responsibility”

I remember once unintentionally deleting data from the master table and the query was auto-committed. I went cold but soon realized I had taken backup of the data, phew! It is always good to provide limited access for data modification via an interface which keeps data backup. One should also note that this tool should also be capable of doing bulk operations, else you will find yourself doing manual updates for a very long time.

5. Configuration menace

Scattered configurations in different files and database is a menace. Change in a configuration which affects multiple applications would have to be replicated in many files and databases. This probably is not much of an issue if you have a handful of applications but would soon become an overwhelming task when you have hundreds of application in an enterprise environment. Moreover, naked passwords in config files are yet another threat. If configurations are database driven then it reduces the later hazels also safeguards sensitive configuration entries.

4. Hard-coded time bomb

Hard coding values in the program are definitely a bad practice. Strange issues could occur because of the hardcoded value which silently sits there in your source code. One such incident is when you make use of hard-coded values for some maximum number, assuming that the maximum will not be hit. Such cases could take months or years to raise an error. Combined with no error logging and you will be in a very bad situation. Try to avoid hard-coded values. Move them to configuration variables if needed or try to completely avoid situations which would require them.

3. Prevent Burnouts

Working continuously without any breaks will just lead to burnouts. You should always take breaks at regular intervals. When you are overworked even doing simple things take a whole lot of time.

Set small achievable milestones for yourself and don’t forget to reward yourself for it. Chocolates, cakes, quick games just about anything that makes you happy and feel free. Learning how to prevent burnouts will make you more productive.

2. Code / Suggest for building resume

Having a strong resume with a listing of all new technologies is great but don’t just do it for inflating the resume. Learn them but use them only if you see the benefit and high maintainability over the existing things. Sometimes adding new tech options would not even be required.

Increasing the complexity should at least pay off in providing ease of maintainability. So I strongly believe that when you suggest some solution, check if it makes the application easy to maintain and makes everybody’s life easy.

1. Finding comfort zone in just one technology

Being a ninja in one technology is awesome and is always beneficial. But don’t stay limited to one technology. Try to expand your knowledge, sure there is a learning curve but challenging yourself helps you become a better technologist – a person who uses various technologies as effective tools and provides business enabling solution.

Times change, technology may become obsolete but your learning should never stop. Always try to explore new technologies and try weighing the pros and cons before using it as a solution for solving a problem.

I would like to hear about your views on these points. If you have any experiences to share, go ahead and post a comment. Remember…

The only real mistake is the one from which we learn nothing. ~ John Powell

This article was originally published on SkillHive Blog