Forem: Gustav Ehrenborg

Home Assistant dev.to views sensor

Gustav Ehrenborg — Tue, 31 May 2022 19:21:37 +0000

Home Assistant (HA) is a popular home automation software. Is has tons of built-in features, for example something called RESTful Sensor. It let's you configure an API call and how to process the data to show the data you want on your Home Assistant dashboard.

This guide will show you how to set it up to show your dev.to total views count.

Getting started

Dev.to has an API that can be used to fetch user and article information. However, total post view count is not available in the API, but by adding the view count of each article together will produce this number.

Go to https://dev.to/settings/account and generate a new API key. It is used for authorization, since the view count is not publicly available.

The API endpoint that we're gonna use is documented here. It returns the published articles, including the attribute page_views_count.

curl 'https://dev.to/api/articles/me/published' \
-H 'api_key: <your API key>' \
-H 'per_page: 100'

will return

[
  {
    ...
    "page_views_count": 103,
    ...
  },
  {
    ...
    "page_views_count": 3245,
    ...
  },
  ...
]

Make sure to add the per_page: 100 (or more, up to 1000), since 30 is the default. Once you go over 30 posts, all views will not be counted.

Configure the sensor in HA

In the secrets.yml, add dev_to_api_key: <your API key>. Separating the secrets from the configuration.yml is a good practice. !secret dev_to_api_key will be substituted for the secret value.

In configuration.yml, add the following sensor code.

sensor:
  - platform: rest
    name: Dev to views
    resource: https://dev.to/api/articles/me/published
    headers:
      api_key: !secret dev_to_api_key
      per_page: 100
    scan_interval: 3600
    value_template: "{{ value_json | sum(attribute='page_views_count') }}"
    unit_of_measurement: views

The sensor attributes are pretty self-explanatory. Scan interval takes in the amount of seconds between each call (3600 seconds are one hour). The value template needs some explanation though.

The data received looks like in the snippet above. It is automatically parsed as json and put in the value_json variable. That is piped through the sum method that sums the specific attribute.

Stop and start HA to reload the configuration.

Add to the dashboard in HA

Add a new entity card and select your new Dev.to entity and a suitable icon.

This will be the result:

Good luck!

Using dotnet secrets

Gustav Ehrenborg — Wed, 20 Apr 2022 18:05:01 +0000

Instead of putting secret environment variables in the appsetting.json file, the dotnet CLI has functionality to add secrets to a project, without keeping them in the project folder.

This will decrease the risk of leaking the secrets, for example by adding them to a commit by mistake.

This link leads to Microsoft's own documentation.

The cheatsheet

dotnet user-secrets --help # See available commands

dotnet user-secrets init # Initiate secrets

dotnet user-secrets set "MY_API_KEY" "xyz" # Set a secret

dotnet user-secrets set "SOME_API.MY_API_KEY" "xyz" # Set a nested secret

cat secrets.json | dotnet user-secrets set # Set all secrets in a json file

dotnet user-secrets list # List the secrets

dotnet user-secrets clear # Delete all secrets in the project

The walkthrough

Initiating

The dotnet user-secrets init command enables storage of secrets. It adds a line with a GUID in the .csproj file. This GUID is used to identify which secrets belong to which project. The command only needs to be run once for every project, if you are sharing your code with other people, they don't need to rerun it.

<UserSecretsId>71ad533a-ed09-4780-9037-a5aafb01958b</UserSecretsId>

The command also creates a folder in ~/.microsoft/usersecrets (on a Mac), with a file contaning the secrets in json format.

~/.microsoft/usersecrets
|
└───71ad533a-ed09-4780-9037-a5aafb01958b
│   └──secrets.json
|
└───<some-other-GUID>
    └──secrets.json

The secrets are saved in plain text in this json file.

Setting secrets

dotnet user-secrets set "MY_API_KEY" "xyz"
dotnet user-secrets set "SOME_API.MY_API_KEY" "xyz"

The commands above are self explanatory and the resulting secrets.json file will look like this:

{
  "MY_API_KEY": "xyz",
  "SOME_API.MY_API_KEY": "xyz"
}

If you are given secrets in the same format as above, these can be imported all at once with the command:

$ cat secrets.json | dotnet user-secrets set
Successfully saved 2 secrets to the secret store.

Using dotnet tools

Gustav Ehrenborg — Tue, 12 Apr 2022 17:03:44 +0000

Dotnet tools are applications that can assist you with various tasks and functionality. There are tools for checking test coverage, deploying to certain cloud services, code formatting, etc... Nuget has a list here.

The following article will help you get started with the tools using the Swagger CLI, called Swashbuckle.AspNetCore.Cli, as an example.

The cheatsheet

dotnet new tool-manifest # Create a tool manifest file

dotnet tool list # List installed tools

dotnet tool install ToolName # Install a tool

dotnet tool restore # Install the tools in the manifest file

dotnet tool update ToolName # Update a tool

dotnet tool uninstall ToolName # Uninstall a tool

dotnet ToolName # Run a tool

The walkthrough

Local and global tools

Tools can be installed locally or globally. Locally installed tools only work for the specific project, but can leverage the tool manifest file which make the tool setup more portable.

Globally installed tools are available everywhere. The commands for global tools are the same, but the --global flag must be passed on.

The tool manifest file

The command dotnet new tool-manifest will create a file .config/dotnet-tools.json. It is a file that contains a list of the installed tools and which version. If you are familiar with the package.json file in npm, this file has the same purpose. The empty file looks like this:

{
  "version": 1,
  "isRoot": true,
  "tools": {}
}

When tools are installed, the tools object will be populated with data about the tools.

Install a tool

dotnet tool install Swashbuckle.AspNetCore.Cli will install the Swagger CLI and add it to the manifest file. If you want a specific version, add --version x.y.z. If you have multiple Nuget sources added, the installation might fail since the installer will look in the wrong source first. Add --ignore-failed-sources to circumvent this.

This is the manifest file with the tool installed:

  "tools": {
    "swashbuckle.aspnetcore.cli": {
      "version": "5.5.1",
      "commands": [
        "swagger"
      ]
    }
  }

Run a tool

From the manifest file, we can see that the Swagger CLI comes with the command swagger. This means it can be executed with dotnet swagger.

The Swagger CLI generates the Swagger documentation. It only has one command, tofile and takes two required arguments, the path to the compiled .dll file and which version of the API that should be exported.

dotnet swagger tofile bin/debug/net.../something.dll 1

List tools

Run dotnet tool list to see installed tools. This command is especially useful when listing global tools, since no manifest file exists for these.

➜ dotnet tool list
Package Id                   Version   Commands   Manifest
-------------------------------------------------------------------------------------
swashbuckle.aspnetcore.cli   5.5.1     swagger    ...tools/.config/dotnet-tools.json

Restore tools

Since the manifest file keeps track of the tools, they can easily be restored on a new machine by running dotnet tool restore.

Uninstall tools

dotnet tool uninstall swashbuckle.aspnetcore.cli will uninstall the Swagger CLI.

Summary

There you have it! Explore the available tools, there's probably a tool out there which will enhance your development experience.

Multiple ssh keys

Gustav Ehrenborg — Sat, 05 Feb 2022 14:18:41 +0000

Yesterday, I was gonna add my ssh key to Azure DevOps, to be able to contribute to a repository. I was presented with the following:

Apparently, DevOps does only support rsa keys, and not ed25519 keys, even though ed25519 is considered to be more secure. The ed25519 is also the default algorithm in GitHub's guide to setting up keys which many people follow.

Generating a new rsa key is the solution to this. However, having multiple ssh keys on the same machine presented new problems. I want to use my ed25519 key as much as possible, for example on repositories on GitHub, but how do I tell my computer to use the rsa key for the repository on Azure DevOps?

Apparently, there is a file, ~/.ssh/config, that solves these kind of troubles. You probably have a couple of lines in it, like this:

Host *
  AddKeysToAgent yes
  IdentityFile ~/.ssh/id_ed25519

These lines tells the ssh client to automatically add keys to the running agent and use the key in ~/.ssh/id_ed25519, for all hosts.

By adding these lines:

Host ssh.dev.azure.com
  IdentityFile ~/.ssh/id_rsa

We tell the ssh client to use the key ~/.ssh/id_rsa when connecting to ssh.dev.azure.com, which is the host of the repositories.

Simple as that!

Bonus 1

If you tired of writing the entire ssh username@some-host.com:xxxx when using ssh, aliases can be configured:

Host my-alias
    HostName some-host.com
    User username
    Port xxxx

Now ssh my-alias will be enough to connect.

Bonus 2

If you ever find yourself on a network were port 22 is blocked and you want to use ssh to connect GitHub, you can use ssh over port 443 instead. Just add this configuration.

Host github.com
  Hostname ssh.github.com
  Port 443

Firebase Function to populate Algolia

Gustav Ehrenborg — Thu, 03 Feb 2022 15:25:46 +0000

This is a guide for setting up a Firebase Function that will receive data and update an Algolia index. Algolia is a widely used search engine, and populating and updating its data using webhooks and serverless functions is a common approach to integrate with it.

Setup!

Firebase setup

npm install -g firebase-tools
firebase login
firebase init functions

This will result in a file, functions/src/index.js in which all code will be written.

Algolia setup

Create an account, create an index
Go to Settings and then API keys to get the application ID and admin API key. You will also need the name of the index.

Add environment variables

Exchange the placeholders in the following command:

firebase functions:config:set algolia.application_id="THE APPLICATION ID" algolia.api_key="THE API KEY" algolia.index_name=”THE INDEX NAME”

Use firebase functions:config:get to display the variables.

To run the function locally, you'll need to save the variables to a specfic file, using the following command:

firebase functions:config:get > .runtimeconfig.json

Reference the variables in code for easy access:

const ALGOLIA_APPLICATION_ID = functions.config().algolia.application_id;
const ALGOLIA_API_KEY = functions.config().algolia.api_key;
const ALGOLIA_INDEX_NAME = functions.config().algolia.index_name;

Install dependencies

npm install algoliasearch –save

...and import it with

const algoliasearch = require('algoliasearch');

Code!

Add the following function. The functions.https.onRequest will expose it and bind it to a specific endpoint. Multiple such function can exist in the same file, but we will only have one. Express is the underlying framework, and the request and response objects work similar to when using Express directly.

export const publishedEntry = functions.https.onRequest(
  async (request, response) => {
    const entry = request.body;
    const index = getAlgoliaIndex();

    if (entry.action === 'DELETED_ENTRY') {
      await index.deleteObject(entry.id);
    } else if (entry.type === 'BOOK') {
      const record = createAlgoliaBookRecord(entry);
      await index.saveObject(record);
    }

    response.send('ok');
  }
);

The data sent in contained in the request.body. The getAlgoliaIndex() method is presented below. The if-statement idenfies the action, or the type, and performs the right action.

Using the Algolia SDK, the following function will return a SearchIndex. It's on this object that inserts, updates, deletions, etc, of objects are made. The Algolia documentation has all the available methods.

const getAlgoliaIndex = () => {
  const client = algoliasearch(ALGOLIA_APPLICATION_ID, ALGOLIA_API_KEY);
  return client.initIndex(ALGOLIA_INDEX_NAME);
};

The createAlgoliaBookRecord() filters out just the attributes we're interested in having in our index. A book might have a price, a number of pages, etc, that we do not want to search in. The same result can be achieved by configuring the searchableAttributes in Algolia, however, the more data that is added to the index, the slower it will become.

Our book model has an ObjectID, that we pass along to Algolia. All Algolia records must have an ObjectId, and if it's not supplied, Algolia will create one.

const createAlgoliaBookRecord = (entry) => {
  const { id, type, name, isbn } = entry;
  return { objectID: id, type, name, isbn };
};

Deploy!

npm run deploy

The deploy script will run the linter, build the code, upload the code and environment variables and deploy it. The URL of the function(s) will be presented.

Try it out

POST the following data to the function URL...

{
    "action": "PUBLISHED",
    "type": "BOOK",
    "id": "61fbdf833dbc31f5935dea1b",
    "name": "Some book name",
    "author": "Some Author",
    "isbn": "123456789"
}

...and watch how it is added to Algolia.

To test the deletion, only the following data is needed in the POST:

{
    "action": "DELETED_ENTRY",
    "id": "61fbdf833dbc31f5935dea1b"
}

Congratulations! You now have a Firebase Function that is updating an Algolia index.

Webhook your weight with Withings API

Gustav Ehrenborg — Mon, 24 Jan 2022 14:18:01 +0000

This is a short walkthrough on how to authenticate with and use the Withings API. Its purpose is to explain how to get going with the code in this GitHub repo. The goal of the code is to be lightweight and have few dependencies, making it possible to run on weaker computers, often used in home automation.

Prerequisites

You'll need a Withings product. I use a scale. The webhook is the same for all products.
You'll also need a domain name pointing towards your server and an open port, or use ngrok. This will be your callback url.
I write cURL commands below, if you prefer to use Postman, it has an Import button to convert from cURL.

Withings account setup

Click here to create a partner account. You’ll receive a client id and a consumer secret, hereafter called client secret.

Withings authentication

The authentication flow contains quite many steps, and we'll cheat by running some requests manually, instead of implementing the whole process in code. This links leads to the authentication documentation.

Using your client id, client secret and callback url, complete the following URL and enter it in a browser.

https://account.withings.com/oauth2_user/authorize2?response_type=code&client_id=<your client id>&state=whatever&scope=user.info,user.metrics&redirect_uri=<your callback url>

Click Allow this app, which will redirect to your callback url with a query param called code. This is an authorization code and it is valid for 30 seconds.

Then, fill out your values in this cURL command and run it.

curl --data "action=requesttoken&grant_type=authorization_code&client_id=<your client id>&client_secret=<client secret>&code=<the authorization code from above>&redirect_uri=<your callback url>" 'https://wbsapi.withings.net/v2/oauth2'

You will receive a JSON response, containing a user id, an access token and a refresh token. The access token is needed to fetch data, and it's valid for three hours. The refresh token is used to get a new access token, and every time the access token is renewed, the refresh token is also renewed.

The webhook server

Persisting the user id and refresh tokens

As mentioned above, the refresh token is also renewed, thus demanding a way both to persist and update it. A database would be the best option, but this would require more dependencies. Instead, we'll persist the user id and refresh tokens in a file on disk.

Two really simple methods is all that is needed, one for reading and one for writing.

const getRefreshToken = async (userId) => fs.readFile(`./${userId}`, 'utf8');

const persistRefreshToken = async (userId, refreshToken) => fs.writeFile(`./${userId}`, refreshToken);

The last step is also to create a file for your user id and current refresh token. Use this command: echo '<refresh token>' >> <userid>, for example: echo 'xxyyzz112233' >> 1234.

Getting an access token (and a refresh token)

Axios is used to make the API request. An axios instance for communicating with the Withings API is made, setting the defaults of the API. Note that the data is not sent as JSON, it is form-data.

const withingsAxios = axios.create({
  baseURL: 'https://wbsapi.withings.net',
  headers: { 'Content-Type': 'application/x-www-form-urlencoded' },
});

The refreshAccessToken function fetches the current refresh token from file, uses it to obtain both an access and a new refresh token, and then persists the new refresh token. The Withings endpoint used is described more here.

const refreshAccessToken = async (userId) => {
  const currentRefreshToken = await getRefreshToken(userId);

  const data = {
    action: 'requesttoken',
    grant_type: 'refresh_token',
    client_id: WITHINGS_CLIENT_ID,
    client_secret: WITHINGS_CLIENT_SECRET,
    refresh_token: currentRefreshToken,
  };
  const body = qs.stringify(data);
  const response = await withingsAxios.post('/v2/oauth2', body);

  const newRefreshAccessToken = response.data.body.refresh_token;
  await persistRefreshToken(userId, newRefreshAccessToken);

  return response.data.body.access_token;
};

Fetching a measurement

The getMeasures function fetches mesurements from your Withings appliances. An access token must be supplied, as well as a start and an end date. The dates should be Unix timestamps. However, we don't have to worry about those, because the callback will include timestamps.

Again, the axios instance is used, the data is form-data. The access token is included in the header. This links to the documentation.

const getMeasures = async (accessToken, startDate, endDate) => {
  const data = {
    action: 'getmeas',
    meastype: 1,
    category: 1,
    startdate: startDate,
    enddate: endDate,
  };
  const body = qs.stringify(data);

  const response = await withingsAxios.post('/measure', body,
    { headers: { Authorization: `Bearer ${accessToken}` } },
  );

  return response.data.body.measuregrps;
};

Webhook server using Express

A simple Express endpoint is used to receive the webhook call. The call does not include any measurements, only timestamps, but your code will get a new access token and fetch the measurement data using the functions above, using the timestamps that were included in the webhook.

app.post('/withingsWebhook', async (request, response) => {
  response.send('ok');

  const userId = request.body.userid;
  const startDate = request.body.startdate;
  const endDate = request.body.enddate;

  const accessToken = await refreshAccessToken(userId);
  const measures = await getMeasures(accessToken, startDate, endDate);
  const weightInGrams = measures[0].measures[0].value;
  return weightInGrams;
});

Subscribing to the Withings webhook

The last step is to tell Withings to post to your webhook. We will do this manually as well.

curl --header "Authorization: Bearer <your access token>" --data "action=subscribe&callbackurl=<your callback url>&appli=1&comment=comment" 'https://wbsapi.withings.net/notify'

Finalizing this will have your webhook endpoint be called when a new measurement is made. There is, unfortunately, about 30 seconds delay from Withings.

The optional last steps

Add it to cron to start at system reboot
Publish your weight on mqtt
Have Home Assistant receive it
...and configure Google Home to say it out load!
Add error handling, and tests

For full code, head over here: https://github.com/GuTsaV/trumpet

My thoughts on the Contentful Certification

Gustav Ehrenborg — Fri, 24 Dec 2021 09:48:27 +0000

I recently passed the Contentful certification, and became a "certified professional". I want to share my thoughts about the exam and recommendations on what to study to pass the exam. Please remember that the following reflects the conditions when I took the exam (winter 2021) and things might have changed.

The Contentful Training Center

Contentful has its own training center, training.contentful.com. It gives you access to courses, consisting of text, interactive images, videos and knowledge checks. The content is good, short pieces of text describes features and concepts, and videos show how to work with them. There is some overlap in the different courses, but its easy skip through. However, it is quite difficult to get an overview of the available material in the training center. There are activities, paths and courses, and its difficult to find the way back to where you were.

I went throught an activity called Contentful Essentials. It has paths for developers, content modelers and content authors. Since I wanted to be prepared for the exam, I took all three paths. The courses in the paths are good, but sometimes too basic. Being a developer, I especially liked the developer courses, since they showcased specific Contentful features. There are excellent videos on how to setup approval flows, configure webhooks, etc.

Contentful Developer Documentation

There is also technical documentation available at contentful.com/developers/docs/. The documentation includes API references, videos showing features and more. Since the exam is said to be not only for developers, but also for project managers and content architects, I did not study the developer documentation as much as I would have wished. In retrospect, the exam contained quite many technical questions that are covered by these docs.

The exam

50 questions
75 minutes
70 % score needed to pass
"closed-book", no notes or google allowed

You are presented with one question at a time and you can't go back to change your answers! Your score is presented to you directly after submitting the last question.

The exam was more difficult than I anticipated. I thought the questions were gonna be as difficult as the knowledge checks in the training courses, or should I say as easy as the knowledge checks... I was wrong.

The exam does contain some easy questions, however, there are a lot of questions on details, for example on the capabilities of the Contentful CLI and the SLAs.

To make things even more difficult, there are a lot of questions where you should select all the correct answers out of the four alternatives, and the alternatives are slightly rephrased from what you have seen before.

Summary

Don't be fooled by the easy knowledge checks. Prepare for technical questions by spending more time reading the developer documentation, but make sure to go through some training courses, especially if you are lacking content model experience.