Forem: Ali Khajeh-Hosseini

June 2021 update - HashiCorp partnership, Env0 and Spacelift!

Ali Khajeh-Hosseini — Tue, 06 Jul 2021 10:00:28 +0000

In June we focused on integrations and adding more resource coverage. You can upgrade to the latest version (v0.9.2) to pickup the new features. If you are using v0.8 please follow the v0.9 migration guide.

⚙️ Integrations

Infracost already has CI/CD integrations with GitHub Actions, GitLab CI, Atlantis, Azure DevOps, CircleCI, Bitbucket Pipelines and Jenkins. This list keeps growing, so please let us know what you'd like to see added next. Infracost can now be used with the following infra-as-code management platforms too:

Terraform Cloud: we have partnered with HashiCorp to bring cloud cost estimates into Terraform Cloud's new RunChecks (currently in beta). Armon Dadgar, co-founder and CTO of HashiCorp, announced the partnership during HashiConf EU; screenshot below! If your company uses Terraform Cloud and would like to work with us and HashiCorp together on this integration, please reply to this email.
Env0: cloud cost estimates can easily be enabled in Env0. Our CEO Hassan, Env0's CEO Ohad Maislish, and Tim Davis are doing a webinar about cloud costs shifting left on 14th July. Signup to listen in.
Spacelift: cloud cost estimates can also be enabled in Spacelift.

⛅ New cloud resources

Infracost now supports over 200 Terraform resources across AWS, Azure and Google. Over 500 free resources have also been identified; these are not shown in the CLI output since they are free.

We added support for the following cloud resources:

AWS: Backup, EFS One Zone, Kinesis Firehose & Data Analytics, Neptune.
Google: BigQuery, Load Balancer, VPN tunnel.
Azure: Application Gateway, Application Insights, Automation, Cognitive Search, Event Hubs, Kubernetes Load Balancer & HTTP Application Routing, Load Balancer, Redis Cache.

⌨️ Shell completion

Run infracost completion --help to see how you can generate shell completion scripts for Bash, Zsh, fish and PowerShell. Once enabled, you can use the usual double tab to see autocomplete and help text for commands and flags.

Thanks for being part of the community! We look forward to hearing your feedback via GitHub issues or our community Slack chat.

Announcement: Azure cloud cost estimates in pull requests

Ali Khajeh-Hosseini — Mon, 14 Jun 2021 13:56:51 +0000

Thanks to our awesome community, I'm very excited to announce that you can now use Infracost to get cloud cost estimates for Microsoft Azure in pull requests. Try it now, it's free and open source!

Cloud costs for engineering teams

Cloud costs have become so complex that the industry (ourselves included) started addressing issues around breached budgets after the bill arrived. This is not the way it should be. It is like going shopping and having no idea how much things cost till after your card has been charged.

We are on a mission to empower engineering teams to use cloud infrastructure economically and efficiently. We do this by fitting in the developer workflow via CI/CD integration, reading the Infrastructure-as-code project, picking up the parameters that have a price point, looking up the prices for the configurations and leaving a comment in the Pull Request like "This change will increase your cloud costs by 25%" with a detailed breakdown. This way, the whole team is aware of the cost implications of the change.

Today's announcement

Today we are announcing that in addition to AWS and Google Cloud Platform, we have added support for Microsoft Azure. Not only has Azure support been requested by over 60 of our community members (https://github.com/infracost/infracost/issues/64), but we have seen a lot of growth from Microsoft in terms of the number of resources offered and enterprise adoption.

We have added support for over 65 Azure resources (and another 70 resources which are free), with many more planned. Visit our GitHub issues page and put a thumbs up on the resources you'd like covered and we will prioritize them.

But there is one more thing! We have also added support for Microsoft Azure DevOps Pipelines. This is in addition to our current supported CI/CD integrations such as GitHub Actions, GitLab CI, CircleCI, Bitbucket Pipelines, Atlantis and Jenkins.

Get started!

We have made it super simple to get up and running:

1. brew install infracost # (docker, windows etc options available)
2. infracost register
3. az login # To set cloud creds, see note-1
4. infracost breakdown --path . # Run in your terraform directory. We also have an example Azure terraform file you can use to try it out.

For full details, see our Getting Started guide. From there, you can setup Azure DevOps Pipelines for the CI/CD integration.

note-1: Infracost does not need or access your cloud creds, however, Terraform needs this to create the plan file.

GitHub stars matter! Here is why

Ali Khajeh-Hosseini — Tue, 11 May 2021 20:59:11 +0000

As Infracost has hit 3,000 GitHub stars 🎉, I wanted to share some thoughts as to why GitHub stars matter.

Why do people star repos?

There are two main reasons why people star GitHub projects:

Bookmarks: some people star GitHub repos to bookmark them for later use. For example I can see the repos I've starred[1] and search within them for a keyword or sort them by how recently I starred them, or how active the project has been recently.
Show support or appreciation: others star repos to show support or appreciation, similar to how "likes" are used in social media sites. This is a social signal, and it's very important in the very early stages of open source projects, acting as a feedback loop for project creators. Knowing that other people have seen the project and cared enough to click on the Star button can create motivation for the creators to continue working on the project initially.

The latter is why I personally star projects. Regardless of whether I've used the project in the past, using it just now, plan to use it, or think it's a cool idea, I want the project creator to know that I like what they're doing. Terraform and Pulumi are projects that I recently starred to show support.

Benefits of repo stars

The main benefit of repo stars is creating confidence and a good first impression of the project. That in turn helps with the project getting users, and to a lesser extent contributors.

A 2018 academic research survey of over 700 developers found that "three out of four developers consider the number of stars before using or contributing to GitHub projects"[2]. GitHub stars are not the only metric that matters though. A project's activity level, for example its last release or commit, and its ease of use, for example the quality of its documentation, are also important factors in helping projects get users.

I say to a lesser extent as contributing, by creating a GitHub issue or submitting a pull request, requires significantly more effort than starring a repo. People who only star a repo are probably not yet active community members but they might become active in the future. This is why the Orbit Model classifies them as Observers[3], as they can act as the top-of-funnel for growing users and contributors. hugely popular

In addition to helping projects get users, GitHub stars can help the project creators meet investors who are familiar with open source. Early on in Infracost's journey, we were surprised to get cold emails from VCs congratulating us on our star count. After speaking with a few, it became clear that they either had systems in place to monitor stars[4], or had analysts who reviewed Trending Repos on GitHub for potential investment opportunities[5]. Some have gone even further. For example, the VC firm Runa Capital, who invested in Nginx and MariaDB, has started to track the fastest growing open source startups using GitHub stars and forks. Infracost was recently placed 5th on the ROSS Index[6].

Future of GitHub stars

A16Z's Martin Casado thinks that there is a big trend towards bottom-up strategies in business-to-business (B2B) software that will shape the entire B2B landscape in the next 10 years[7]. I wonder if in the same way that social media influencers are changing how products are marketed and sold, GitHub influencers (someone with many GitHub followers) will change how enterprise software is marketed and sold? Developer Advocates are currently using Twitter and LinkedIn, but GitHub has a "follow" and a "status update" feature too. Will those remain as a simple way to get updates on code-related activities? Or could they be extended to enable GitHub influencers to post their demos, talks and blogs into the GitHub activity feed? Will companies be able to buy ads on GitHub and promote their open source projects?

Over to you - what have you learnt about GitHub stars, and how do you think they'll change in the future? I hang out on Twitter...

https://github.com/alikhajeh1?tab=stars, this is a public page, so you can see the repos that any GitHub user has starred.
H. Borges and M. Tulio Valente, "What's in a GitHub Star? Understanding Repository Starring Practices in a Social Coding Platform," Journal of Systems and Software, vol. 146, pp. 112–129, 2018.
The Orbit Model is implemented via the Orbit product, which can be used to measure and grow open source communities.
Openbase helps developers choose the right JavaScript package with more languages coming soon. See the React page to get an idea of the kinds of metrics they collect.
https://github.com/trending, Infracost has hit the Go trending page a few times.
https://runacap.com/ross-index/, Infracost was placed 5th in the fastest-growing open-source startups in Q4 2020.
Growth, Sales, and a New Era of B2B

March 2021 update - new diff command and usage file automation!

Ali Khajeh-Hosseini — Wed, 07 Apr 2021 13:25:51 +0000

March was busy as we added major new features and had Y Combinator's demo day, where Hassan (our CEO) delivered an awesome 60 second pitch on a Zoom call with hundreds of investors!

You can upgrade to the latest version (v0.8.3) to pickup the new features. If you are using v0.7 (or older) please follow the v0.8 migration guide.

🗒️ See diffs in CLI

A highly requested feature was the ability to see the difference in cost between the current state and the planned state of Terraform projects in the CLI (we already have this feature in CI/CD integrations). Check it out by running infracost diff --help. We have also updated the CI output to make it easier to read.

⚙️ Automated usage-based resource definitions

Usage-based resources, such as AWS Lambda or Google Cloud Storage, require estimated usage data so Infracost can show costs in the output. You can define these in a YAML file, called a usage file, and use that to get cost estimates for such resources.

Previously you had to create this file manually. You can now use the --sync-usage-file option to generate a new usage file or update an existing one from your Terraform project. This option is a safe sync: it adds any missing resources (with zeros for the usage estimates), it does not overwrite any lines that you have changed in the YAML, and it deletes any resources that are not used in the Terraform project.

  > infracost breakdown --sync-usage-file --usage-file infracost-usage.yml --path /code
  [...]
  > cat infracost-usage.yml

  version: 0.1
  resource_usage:
    aws_lambda_function.hi:
      monthly_requests: 0 # Monthly requests to the Lambda function.
      request_duration_ms: 0 # Avg duration of each request in milliseconds.

😃 Simplified inputs, outputs and config file

We like it when things are made easy:

Inputs: a new path flag has been introduced to replace the various methods of running Infracost. You can now simply point Infracost to the path of a Terraform directory, plan binary file, or plan JSON file and it'll just work. This lays some of the groundwork for supporting other IaC tools in the future.
Outputs: the dashes (-) in the output have been replaced with price descriptions such as Cost depends on usage: $0.20 per 1M requests so you can understand the pricing structure of usage-based resources such as AWS Lambda or Google Cloud Storage.
Config file: the config file has been updated to support infra-as-code repos that have multiple workspaces and projects. This enables you to combine projects into the same breakdown or diff output. So if a Terraform module or variable is used across workspaces/projects, you can quickly see the cost impact of changing it.

🚀 New Pull request comment options

We've updated the CI/CD integrations to add a new post_condition option so you can decide when you'd like cost comments to be shown in pull requests. Options include: always leave a cost comment, only comment when there is a change to the cost, or only comment when a percentage threshold has been reached (e.g. more than 5% increase or decrease in costs).

⛅ New cloud resources

We are working on adding Microsoft Azure to Infracost. This has two steps: the first is to add the prices to the Cloud Pricing API, then to add the resources to the CLI. We completed adding around 300,000 prices from Microsoft Azure to the Cloud Pricing API (step one), and now we're looking for volunteers to add resources to the CLI (step two) before the initial release. Please email ali@infracost.io if you are an Azure user and would like to contribute (basic golang knowledge is required).

We also added support for the following cloud resources:

AWS: Elastic File System (EFS), EBS GP3 volumes, DX Connection, Route53 Health checks, RDS Serverless
Google: Memorystore Redis, Cloud Monitoring and Logging, Compute Images and Snapshots

Thanks for being part of the community! We are always looking forward to your feedback.

Infracost diff - "git diff" but for cloud costs

Ali Khajeh-Hosseini — Thu, 18 Mar 2021 13:35:44 +0000

Recently we released a new infracost diff command inspired by git diff. This shows a diff of monthly cloud cost estimates between the current and planned state of Terraform projects. At a high-level this might seems like a simple exercise of subtracting the current state's cost estimate from the planned state, but cloud costs are rarely that simple to deal with. Let's take a look at the following screenshot to understand some of the nuances.

The aws_instance is being changed, which reduces the cost by $125/month (from $743 to $618).
AWS EC2 has many different cost components, so to explain what caused the above change, we also flag the sub-resource ebs_block_device[0] that changed (the first attached block device). Underneath it, we show the cost component that caused the actual cost diff, Provisioned IOPS SSD Storage (io1); i.e. reducing the size of that volume can save $1500/year. For those who have done this in production, they know it's not a one-click change as you need to create a new EBS volume and copy over the data. What Infracost enables you to do is to quickly tell how much such a change would save you, then decide if it's worth it.
A new aws_lambda_function is being added. Since we don't know how much it's going to be used, we can't show a cost estimate. But we can still show you the prices you'll be charged for: $0.20 per 1M requests and a tiny amount per GB-second. This is a usage-based resource, so if you like you can create a yaml file to provide usage estimates and get a cost estimate. It's hard to think in GB-seconds, so we enable you to input the average request duration and we'll do the math to map that to GB-seconds based on the memory_size of your function and any rounding rules that AWS applies.
```
version: 0.1

resource_usage:
  aws_lambda_function.hello_world:
    monthly_requests: 100000000 # Monthly number of requests.
    request_duration_ms: 250 # Average duration of each request in milliseconds.
```
Finally we show a summary at the bottom: the EC2 instance change reduces the cost by 17%, and you can use the above yaml file to do simple what-if analysis on the Lambda costs.

The new infracost diff command is used by our CI/CD integrations and is open source alongside the rest of Infracost. We look forward to hearing what you do with it via GitHub issues or our community Slack!

Infracost Feb 2021 update: faster runs, new resources and Atlantis!

Ali Khajeh-Hosseini — Wed, 03 Mar 2021 10:27:49 +0000

Here's what we released in February - big thanks to the community contributors! You can upgrade to the latest version (v0.7.20) to pickup these goodies:

🚀 Speed improvements

The CLI now only runs terraform init if required since Terraform commands aren't the fastest in the world (init usually takes 20+ secs for me, but it depends on how many plugins you have). Furthermore, calls to the Cloud Pricing API have been switched from sequential to parallel. Infracost should run much faster than before.

⚙️ Config file

Depending on your Terraform workflow, you'll run Infracost with different options. Things can get complicated when you have multiple projects in a repo, each requiring their own Terraform variables. For example, if you have two workspaces and want to see their total cost estimate, you would run something like this:

terraform workspace select dev
infracost --terraform-dir code --format json \
          --terraform-plan-flags "-var-file=env.dev.tfvars" > dev.json

terraform workspace select prod
infracost --terraform-dir code --format json \
          --terraform-plan-flags "-var-file=env.prod.tfvars" > prod.json

infracost report --format table dev.json prod.json

You can now create an infracost.yml config file in your repo to describe your setup, then just run infracost --config-file infracost.yml.

🌎 Atlantis integration

Infracost now integrates with Atlantis, which is a popular CI/CD tool that enables Terraform pull request automation.

🗒️ Diff functionality in JSON output

You can now get the monthly cost diff from the Infracost JSON output, e.g. the following shows the monthly cost is going to be increased by $1530 if the Terraform plan is applied. You can also get totalHourlyCost, or add --no-color=true --log-level=warn if you don't want the spinners/logs/color.

infracost --terraform-dir=. --format=json | jq '[.projects[].diff.totalMonthlyCost | select (.!=null) | tonumber] | add'
"+1530"

⛅ New cloud resources

We also shipped support for the following cloud resources:

AWS: Config, ECS on EC2, EventBridge, Route 53 Resolver, CodeBuild
Google: Key Management Service (KMS), Google Cloud Functions
Azure: great progress is being made, stay tuned for exciting news soon

The usage file params for Google Cloud Functions are pretty cool; as shown below you can define 3 simple params and we'll estimate the cost for you, no need for you to decode how function memory maps to GHz-seconds and rounding.

google_cloudfunctions_function.my_function:
  request_duration_ms: 150 # milliseconds
  monthly_function_invocations: 10000000
  monthly_outbound_data_gb: 50

NAME                              MONTHLY QTY  UNIT         MONTHLY COST

google_cloudfunctions_function.hi
├─ CPU                                800,000  GHz-seconds  8.0000
├─ Memory                             500,000  GB-seconds   1.2500
├─ Invocations                     10,000,000  invocations  4.0000
└─ Outbound data transfer                  50  GB           6.0000
Total                                                       19.2500

As always, looking forward to your feedback (hello@infracost.io).