Forem: Veer Abheek Singh

Anyone ever made JSON RPC from NODE?

Veer Abheek Singh — Wed, 14 Oct 2020 13:18:49 +0000

Current options to deploy a Serverless blog

Veer Abheek Singh — Sun, 31 Mar 2019 09:30:41 +0000

Original Post: https://www.crumpledpapr.com/post/easy-to-deploy-serverless-blog

I love technology and learn whenever I can, on the job or not.

Went to work last evening for some usual stuff to be handled. The founder and I got into the discussion of our architecture and how we are trying to move away from spending time on IT management and concentrating on more time to serve the customers by using a fully serverless architecture which is run on AWS (Lambdas, S3, API Gateways and Step Functions). We talked about a couple of benefits of serverless for an hour or so. Talking about it kind of enticed me to come home and look for a serverless deployment of blog I could setup. So, I started scrounging the internet.

WordPress + Shifter + AWS S3
The first search result that comes up is by theburningmonk aka Yan Cui, an AWS Serverless HERO. He talks about serving WP pages using AWS S3, which is the usual suspect. It is amazingly fast but some of the features won't work, as suggested by Yan. To quote: "As you can imagine, some WordPress features don’t work anymore since the site is statically generated – comments, contact forms, search, etc." In this case, you would have to look for alternate options such as Google Forms, Disqus, etc. This kind of tears me down since it limits the basic capabilities of the blog such as commenting, etc.

Why not use service providers such as Shifter or any other service? I wanted to make the blog service independent. Earlier last year I used ServerPilot to host Grav, which was lightweight but not that great with the user experience for me. I was using ServerPilot to host and manage Grav but they removed the free plan and grandfathered the account, which for some reason stopped working a couple of weeks ago.

That being said, let move on to important things.

OpenSource Project + Zappa + AWS Lambda
The second option uses an opensource project based on the flask framework which sounds interesting but I am just a novice in python and at a total loss for flask framework. The downside is no theme support, which means I would have to do a lot of stuff to improve and add more functionality.

Features:

Integration with Markdown Editor

Ability to upload images for use in blog pages

Incorporate math formulas in LaTeX format

Integrates with authentication to allow multiple users

Plugin framework to easily extend and add new features

This option uses Zappa(handles horizontal scaling automatically), which looks like a rather new, to deploy the code to AWS Lambda with less effort. Could be used for other stuff. Making a note.

Might come back later if nothing works out.

HEXO (Open Source) + AWS S3
Next option found was on a post of Hackernoon, Hexo which looks like a good option. It's Open Source, well supported and a ton of plugins & themes to work with. Even though it is based on Nodejs(no hands on experience of moi).

It doesn’t look difficult to implement because the blog provides all the instructions required. A plus side is a number of themes which is handy because I care how my blog looks.

Dead Options
Stumbled upon an open source project aws-lambda-blog which turns me off when I read the first lines, it is not maintained anymore and the code is difficult to edit. Moving on. It refers to another project, Awly which is in Alpha, by the same person.

Chalice + AWS Lambda
Next comes Chalice framework which is python based, again which I say could be doable. This would require me to work with the code, even though the blog post provides most of it.

The way it works sounds interesting. It provisions services as and when needed. It talks about the MVC model which kind of makes me go to sleep because I am looking for an easy to setup system. After reading a bit more, I found that it is someone's project, so not a way to go since, no support for the blog, everything would have to be done manually. I am not looking to manage code at this point.

While reading through the article I remember I need to buy a monitor for my MacBook Air so I can try to multitask. Other option would be to get another laptop, inspired by Mark Zuckerberg's portrayal in the social network. Enough wandering, back to it.

Hasura + Next.js + Zeit
Next on the list was a blog by Hasura employing their Instant Realtime GraphQL on Postgres, Next.js 8(React Framework) and ZEIT(A Serverless Platform) which looks like a solution provided by Hasura. #marketingalert. Hasura might be used for variety of applications too.

Not sure about this because most of these solutions might not be that helpful or optimum to use. Since, it is a marketing thing. Again we are looking for something easy to implement, use and extendible if possible. Nextjs does look interesting though. Their traffic seems to be increasing as well. Might learn Nextjs but would require to learn JS first so….

FOR DEVELOPERS
Came across this very specific guide, Serverless Stack - Learn to Build Full-Stack Apps with Serverless and React on AWS, for Full Stack Devs. Does look good but can't comment. Do add feedback if anyone has used or feels it is of help.

Serverless Django with Zappa and Aurora Serverless

This looks very detailed but we’ll know once we start working on this.

On a side note:

Me doing this is a bad idea since, I have an active project which I am putting on hold for this weekend because I am tired… Probably not the best thing to do this weekend but tired from the Sunburn Music festival featuring DJ Snake. So… Continuing.

Picked Up Hexo option and ran into some issues. Made a noob mistake of providing all files with 777 access and MacOS fucked me over. I will pick this up later as this seems the best option which matches my needs. Will definitely update once it is out.

For now, I have failed at this. So, setting up on Wix for now but will keep on exploring more options. Feel free to suggest more options which are for novice programmers. I can work with infrastructure better than I can code. So open to options.

How effective is visualizations for monitoring infra on AWS?

Veer Abheek Singh — Thu, 19 Apr 2018 12:21:22 +0000

People going crazy over visualization on Reddit.

Can visual infra monitoring be effective?

Pros and cons?

How to Easily Delete Large S3 Buckets?

Veer Abheek Singh — Wed, 11 Apr 2018 13:55:43 +0000

Original post

Deleting an Amazon S3 bucket can be a difficult task. It is even a bigger task if it has a large number of objects in it because you cannot delete non-empty buckets.

Automated solutions such as ‘S3cmd’ or ‘S3nukem’ would either break or take weeks to delete an S3 bucket depending on its size and the number of objects in it.

The best approach is to use AWS, which comes with the benefit of a self-service platform and provides an option of setting lifecycle policy for S3 objects.

AWS lifecycle policies help in defining actions on Amazon S3 during an object’s lifetime. For example, transitioning objects to another storage class, archiving them, or deleting them after a specified period.

Using a shared prefix (that is, objects that have names that begin with a common string), you can define the lifecycle policy for all objects or a subset of objects in the S3 bucket. And using a lifecycle policy, you can define actions specific to current and non-current object versions.

Note: The objects inside the AWS S3 would be gone forever after a day.

This solution is not instant but it takes a day to execute, which is faster than other solutions.

Here’s How to Set Lifecycle of an S3 Bucket

Log in to the AWS Management Console and open the Amazon S3 console at https://console.aws.amazon.com/s3/.
To create a lifecycle policy for a bucket, choose the name of the bucket from the Bucket name list.
Choose the Management tab, and then choose Add lifecycle rule.
Give the rule a name and click Next if you want to delete the whole bucket. Also, you can set the lifecycle rule for objects with a specified name prefix (that is, objects with names that begin with a common string), limit the lifecycle rule scope to one or more object tags, and combine a prefix and several tags.
Click Next on the Transitions tab without any changes.
Select the options as shown in the image below.
Verify the settings for your rule under Review.
If you need to make changes, choose Previous. Otherwise, click Save.

From now on, AWS does the dirty work for you. S3 will do its own housekeeping, and after a day, everything inside your bucket will be deleted.

This approach can be used if you regularly create stacks with CloudFormation or Terraform, and the buckets won’t delete if there are objects in them.

Instead of writing scripts and adding to the workload you can easily setup lifecycle rules to take action on the AWS S3.

There are a number of things that can be accomplished using lifecycle rules. You can read more about them below.

Object Lifecycle Management

How Do I Create a Lifecycle Policy for an S3 Bucket?

5 Mistakes Startups Make While Buying & Managing AWS Computing Resources

Veer Abheek Singh — Thu, 29 Mar 2018 21:23:18 +0000

A majority of today’s startups or born-in-cloud businesses rely on public cloud, like AWS, instead of on-premises IT infrastructure. However, due to lack of in-house IT teams, they are coerced to take the DIY approach to provisioning and managing AWS cloud resources for their deployment and operational needs.

With no proper skill set, they need to learn and decipher several levels of abstraction that cloud brings with it and also familiarize themselves with siloed monitoring tools on their own. To top it, they are overwhelmed by the amount of information they need to soak in, analyze and act. It’s a natural human tendency to rely on what we already know or experienced in our past projects when we are learning a new technology or a tool.

Startups and newbies apply the same analogy of managing on-premise infrastructure management to cloud management as well and ultimately bite the bullet and shell out more money than expected. It’s because, they end up assuming too many things (mostly wrong ones).

This post discusses two primary and three common mistakes startups and newbies make while buying AWS compute resources:

Not using Payer’s Account concept: When a team creates an account and they must avoid creating all their resources in a single account. One of the best practices is to create three AWS accounts — one account each for development, QA, and production. By doing so, the team can provision the resources accordingly. Another best practice is to create a fourth account and make it as payer account. While a team can provision resources to other three accounts, it is not recommended to create any resources in payer account, but link all the three account to this account. With this, the team will get a good discount and there will be separation of resources as well.
Choosing the wrong instance model: An enterprise should know whether to launch an instance with On-Demand or RI or spot instance model. Check the need and calculate accordingly.
- If you don’t know how long you will need the server up, On-Demand instances are the best, because On-demand instances are charged every hour and can be discarded anytime as required. Here, you have the freedom to use whenever or how much ever you want to use, but cost incurred are high. Basically, a server for one year would cost you $1*24*365= $8760 USD.
- On the contrary, if you know how long you need a server, you can opt for Reserved instances, because you get it for discounted price compared to on-demand, however, if you want to discard the server after certain period of usage, you cannot get rid of it as it’s already been paid for and the cloud provider will not take it back.
- If you are in need of a server for a very short duration of time, or really want to test something, Spot instances are ideal as the prices are typically one tenth of the “on-demand server” cost and one fifth of the “reserved server cost.” However, the catch is you need to bid to buy it, say 0.1$ per hour. Hence, the server is available until someone else bids for it at a higher price. In that case, you have a window of 2 minutes to act before it’s gone.
Not monitoring AWS compute costs using CloudWatch alarms: Several newbie AWS users are not aware of CloudWatch alarms feature yet. They still are under the impression that they need to look into AWS cost explorer console manually. With CloudWatch service, a user can enable an alert for a specific instance to quickly know when cost touches a threshold point. One caveat here is, a user must know all the limitations associated with CloudWatch service like its ability to process huge amounts of custom log data, react to events in near real time, etc.
Not having Disaster Recovery (DR) plan: DR plays a vital role in business continuity. Outages even from AWS can sometimes affect a business’ services, not just disasters or other calamities. Let’s recall February 2017 S3 service outage where approximately 148,213 websites, and 121,761 unique domains were down. One of the best solutions to such outages is buying instances for DR. Not many startups and newbies are aware that cross-region replication and well-designed AWS CloudFormation template can go a long way in continuous supply of computing resources for development and operational needs. Additionally, using region level DR rather than AZ-level DR can be the most useful in all scenarios.
Not using a good cloud visibility tool and just relying on Trusted Advisor: To start with, many startups use Trusted Advisor feature to find loopholes in their AWS account, reduce cost, increase performance, and improve security. But, this feature does not provide complete visibility into resources usage and other metrics necessary for cloud optimization. Even though Trusted Advisor provides real-time guidance to help provision resources as per AWS best practices, it does not allow users to monitor, analyze, remediate and right-size AWS cloud with contextual insights and operational capabilities. A cloud visibility tool, like TotalCloud, which helps users to understand complete cloud usage contextually from a single pane view, will only help.

Are you a startup or a newbie still trying to get hold of your AWS usage. Let those mistakes do not come back to bite your business. Try https://TotalCloud.io, an interactive and immersive visual platform offering complete cloud visibility.

5 Mistakes Startups Make While Buying & Managing AWS Computing Resources

Veer Abheek Singh — Thu, 29 Mar 2018 21:21:59 +0000

This post discusses two primary and three common mistakes startups and newbies make while buying AWS compute resources:

1 Not using Payer’s Account concept: When a team creates an account and they must avoid creating all their resources in a single account. One of the best practices is to create three AWS accounts — one account each for development, QA, and production. By doing so, the team can provision the resources accordingly. Another best practice is to create a fourth account and make it as payer account. While a team can provision resources to other three accounts, it is not recommended to create any resources in payer account, but link all the three account to this account. With this, the team will get a good discount and there will be separation of resources as well.

2 Choosing the wrong instance model: An enterprise should know whether to launch an instance with On-Demand or RI or spot instance model. Check the need and calculate accordingly.

a) If you don’t know how long you will need the server up, On-Demand instances are the best, because On-demand instances are charged every hour and can be discarded anytime as required. Here, you have the freedom to use whenever or how much ever you want to use, but cost incurred are high. Basically, a server for one year would cost you $1*24*365= $8760 USD.

b) On the contrary, if you know how long you need a server, you can opt for Reserved instances, because you get it for discounted price compared to on-demand, however, if you want to discard the server after certain period of usage, you cannot get rid of it as it’s already been paid for and the cloud provider will not take it back.

c) If you are in need of a server for a very short duration of time, or really want to test something, Spot instances are ideal as the prices are typically one tenth of the “on-demand server” cost and one fifth of the “reserved server cost.” However, the catch is you need to bid  to buy it, say 0.1$ per hour. Hence, the server is available until someone else bids for it at a higher price. In that case, you have a window of 2 minutes to act before it’s gone.

3 Not monitoring AWS compute costs using CloudWatch alarms: Several newbie AWS users are not aware of CloudWatch alarms feature yet. They still are under the impression that they need to look into AWS cost explorer console manually. With CloudWatch service, a user can enable an alert for a specific instance to quickly know when cost touches a threshold point. One caveat here is, a user must know all the limitations associated with CloudWatch service like its ability to process huge amounts of custom log data, react to events in near real time, etc.

4 Not having Disaster Recovery (DR) plan: DR plays a vital role in business continuity. Outages even from AWS can sometimes affect a business’ services, not just disasters or other calamities. Let’s recall February 2017 S3 service outage where approximately 148,213 websites, and 121,761 unique domains were down. One of the best solutions to such outages is buying instances for DR. Not many startups and newbies are aware that cross-region replication and well-designed AWS CloudFormation template can go a long way in continuous supply of computing resources for development and operational needs. Additionally, using region level DR rather than AZ-level DR can be the most useful in all scenarios.

5 Not using a good cloud visibility tool and just relying on Trusted Advisor: To start with, many startups use Trusted Advisor feature to find loopholes in their AWS account, reduce cost, increase performance, and improve security. But, this feature does not provide complete visibility into resources usage and other metrics necessary for cloud optimization. Even though Trusted Advisor provides real-time guidance to help provision resources as per AWS best practices, it does not allow users to monitor, analyze, remediate and right-size AWS cloud with contextual insights and operational capabilities. A cloud visibility tool, like TotalCloud, which helps users to understand complete cloud usage contextually from a single pane view, will only help.

AWS S3 or the Pandora's box

Veer Abheek Singh — Wed, 21 Mar 2018 08:52:30 +0000

Source

Visualize AWS Flow Logs in a Visual Environment (closed beta)

Veer Abheek Singh — Sun, 11 Mar 2018 11:05:25 +0000

What are you using VPC for?

BETA

AWS Well-Architected Review – Should I or not?

Veer Abheek Singh — Fri, 09 Mar 2018 14:37:36 +0000

AWS Well-Architected Review Framework is a document/white-paper which enables you to review and improve your cloud-based architectures and better understand the business impact of your design decisions. It essentially addresses the general design principles as well as specific best practices and guidance in five conceptual areas defined as the pillars of the Well-Architected Framework.

Original Post

AWS Cloud Cost Monitoring & Management Report

Veer Abheek Singh — Fri, 09 Mar 2018 14:35:09 +0000

I did this survey in January and am looking for feedback on it.

Report

Minecraft your AWS

Veer Abheek Singh — Wed, 28 Feb 2018 13:18:42 +0000

Remember Minecraft? It was one of the games that exploded due to the new interface and the way it allowed you to build and create anything and everything imaginable.

We have the same vision for TotalCloud — to let you operate anything and everything imaginable for your AWS cloud infrastructure in a couple of clicks.

TotalCloud presently works only with AWS.

How does it work?

In Minecraft, you can mine resources from the existing world and create a new world. Similarly, on TotalCloud, you can disconnect (manipulate) your cloud resources from your IT infrastructure with just two clicks (just like mining in Minecraft). Sounds simple, right? It is!

Once you have the resources in your inventory, you can add, delete, or do anything and everything imaginable. This is one of the reasons TotalCloud is made on a gaming engine — providing a Minecraft experience for your AWS.

Original post

How can you convince your manager to start embracing DevOps?

Veer Abheek Singh — Thu, 08 Feb 2018 12:56:24 +0000

You have recently joined a software-powered company where they don't practice DevOps practices. Your manager and management are of old-fashioned and still follow the old methodologies. So, in this case, How do you convince your manager that DevOps is essential and should be embraced to survive in this competitive environment?