Forem: TnvMadhav⚡

Stocks Profits Tracker (+ Notion API Integration with Python)

TnvMadhav⚡ — Sun, 26 Nov 2023 14:54:21 +0000

You can track live US stock prices in your personal notion dashboard and a single Python script.

➚ Get it now

What will I get?

📊 A Notion Template to maintain a personal portfolio of US stock investments

🤯 A mechanism for live price tracking & profits calculation

📖 Python script to update the live prices & step-by-step setup guide to get it all working

😃 Easy to use

Integration Guide

1. Duplicating Template:

Once, you purchase the notion template, you’ll be given a notion template link. Click on the Duplicate button in the top right of the notion page.

ℹ️ Ignore if you’ve already completed this step. Notion may ask you to login into your notion account to successfully duplicate.

2. Creating Notion Integration:

What is a Notion Integration?

A notion integration allows a user to run automations using tools like programming languages like Python, Javascript, etc.

🤔 You don’t have to worry too much about this if you’re confused, this is just a simple explanation that will help you get a general idea.

Follow these steps to create a notion integration:

Click on this link → https://notion.so/my-integrations/ (once you open this link in your browser, the website may ask you to login to your notion account)
You’ll be presented with this webpage, click on + New Integration button (big black button)

You’ll then be presented with a form to fill in the Basic Information: , you can set the below values:
1. Type → Internal
2. Associated Workspace → Choose your notion account’s workspace
3. Name → Stocks Profits Tracker
4. Logo → No need, ignore this
Finally, Click on the Submit → (black button)
You’ll next be presented with a webpage called Secrets , you will have to click show and copy the notion integration secret token and keep it safely aside

Copy & Paste the above 🔑 Notion Integration Secret Key somewhere 🔒 safe because it will be used in the 🐍 python script.

3. Connect Notion Integration to Template:

Click on the three dots ... in the top right corner of the duplicate notion template
Click on + Add Connections at the end of the the drop down, search for and click the notion integration you’ve created in step 2 ( Stock Profits Tracker)

4. Populating Stock Prices in ‘Stock Profit Tracker’ Database:

You can add the following values for every stock symbol(tick) you want to add:

Name → Stock Symbol (like MSFT)
Amount Invested → Add the amount you bought the stock units with
Price/Unit → This value will be updated by the Notion Integration (add a default 0.1 so that the python script can update it)
Number of units bought → Add the number of stock units you bought

Leave the Status & Net Profit because when the Price/Unit gets updated by automation, these values will change automatically.

💡 The default notion database contains a few stock market symbols like MSFT, NFLX, TSLA, AAPL, HOOD, etc. (use them as example while adding new stock units)

🚨 Don’t change the column names or database name in the Stocks Profits Tracker database, doing so will make it impossible to update the columns using the python script.

5. Running Python Script:

The final part of the integration is about running a python script that will update the real stock prices for all the stock units you have in the database.

Clone this GitHub repository → https://github.com/tnvmadhav/notion-stocks-integration
Save the secret notion integration token in the my_variables.yml file
```
MY_NOTION_SECRET_TOKEN: paste-your-notion-integration-token-here
```
Install the python requirements and dependency libraries by running the command in your terminal
```
python3 -m pip3 install -r requirements.txt
```
Finally run this command in your terminal to start the live Price/Unit update process
```
python3 main.py
```

That’s it!

If all the steps are followed properly, your notion database should be getting live stock price and net profit updates when python server is running on your machine.

🚨 If something isn’t working or if you’re confused, contact me on Twitter/X DM and I’ll help you out 🙂

➚ Get it now

How to Generate Dynamic Sitemaps in Next.js

TnvMadhav⚡ — Sun, 30 Jul 2023 02:27:38 +0000

Introduction

Let's say you have folder containing markdown files which are used as individual blog posts in the Next.js blog website.

This is exactly what I did to generate sitemap.xml file for my Next.js website https://notionworkspaces.com/

Steps to generate sitemap.xml (only 3 steps)

Create a sitemap.xml.js file inside src/pages/ folder from the Next.js home directory
Add this code inside it

import fs from 'fs';
import matter from 'gray-matter';

const postsDirectory = path.join(process.cwd(), 'posts');

const URL = "https://yourwebsitedomain.com";

function generateSiteMap(posts) {
  return `<?xml version="1.0" encoding="UTF-8"?>
   <urlset xmlns="https://www.sitemaps.org/schemas/sitemap/0.9">
     <!-- Add the static URLs manually -->
     <url>
       <loc>${URL}</loc>
     </url>
     ${posts
       .map(({ id }) => {
         return `
           <url>
               <loc>${`${URL}/posts/${id}`}</loc>
               <lastmod>${new Date().toISOString()}</lastmod>
           </url>
         `;
       })
       .join("")}
   </urlset>
 `;
}

export function getSortedPostsData() {
  // Get file names under /posts
  const fileNames = fs.readdirSync(postsDirectory);
  const allPostsData = fileNames.map((fileName) => {
    // Remove ".md" from file name to get id
    const id = fileName.replace(/\.md$/, '');

    // Read markdown file as string
    const fullPath = path.join(postsDirectory, fileName);
    const fileContents = fs.readFileSync(fullPath, 'utf8');

    // Use gray-matter to parse the post metadata section
    const matterResult = matter(fileContents);

    // Combine the data with the id
    return {
      id,
      ...matterResult.data,
    };
  });
  // Filter out draft posts
  const livePosts = allPostsData.filter((postData) => {
    return postData.draft == false
  })
  // Sort posts by date
  return livePosts.sort((a, b) => {
    if (a.date < b.date) {
      return 1;
    } else {
      return -1;
    }
  });
}

export async function getServerSideProps({ res }) {
  const posts = getSortedPostsData();

  // Generate the XML sitemap with the blog data
  const sitemap = generateSiteMap(posts);

  res.setHeader("Content-Type", "text/xml");
  // Send the XML to the browser
  res.write(sitemap);
  res.end();

  return {
    props: {},
  };
}

export default function SiteMap() {}

Run npm run dev locally to verify that the http://localhost:3000/sitemap.xml is created successfully.

That's it!

Conclusion

You can deploy your Next.js app and verify the sitemap with all 'posts' links dynamically generated for you without any additional effort.

Your website's sitemap should have the same structure as mine.

Here is how my sitemap looks → https://notionworkspaces.com/sitemap.xml

You can also modify the above code to add/delete parts that are unique to your blog.

I hope you found this useful!

👋 -- @TnvMadhav

How to Generate Table of Contents From Markdown/Html Text in Next.js

TnvMadhav⚡ — Thu, 27 Jul 2023 03:24:34 +0000

Introduction

I wanted a programmatic way to generate and extract 'Table of Contents' HTML snippet from existing markdown text for my Next.js blogging website www.notionworkspaces.com.¹

- These are the benefits of this approach:

You don't have to have a Table of Contents section in all your markdown files or HTML files
You can cut-out/extract (or) keep the table of contents after generating them in the content HTML using cheerio

In this tutorial, I'll teach you about a programmatic way to generate and extract 'Table of Contents' HTML snippet from existing markdown text in Next.js

TLDR; working code snippet here

Original Snippet before modification

I already had a function that converted markdown text to html text using remark² library.

export async function getPostData(id) {
  const fullPath = path.join(postsDirectory, `${id}.md`);
  const fileContents = fs.readFileSync(fullPath, 'utf8');

  // Use gray-matter to parse the post metadata section
  const matterResult = matter(fileContents);

  // Use remark to convert markdown into HTML string
  const processedContent = await remark()
    .use(html)
    .process(matterResult.content);
  const contentHtml = processedContent.toString();

  // Combine the data with the id and contentHtml
  return {
    id,
    contentHtml,
    ...matterResult.data,
  };
}

This above snippet was taken from the Next.js's official getting started tutorial³.

- What I wanted exactly:

But, it didn't do everything that I wanted. It didn't generate table of contents based on the structure of markdown data.

I googled around and found an existing library called remark-toc⁴ but it didn't do exactly what I wanted.

It required a few conditions that I didn't want to entertain.

I later stumbled upon rehype⁵ library a more recent take on processing html (also markdown) in Next.js.

The Working Code Snippet

This is final code I use to generate and extract table of contents from my markdown content.

export async function getPostData(id) {
  const fullPath = path.join(postsDirectory, `${id}.md`);
  const fileContents = fs.readFileSync(fullPath, 'utf8');

  // Use gray-matter to parse the post metadata section
  const matterResult = matter(fileContents);

  const file = await unified()
  .use(remarkParse)
  .use(remarkRehype)
  .use(rehypeSlug)
  .use(rehypeDocument)
  .use(rehypeFormat)
  .use(rehypeTOC)
  .use(rehypeStringify)
  .process(matterResult.content)

  // Extract TOC dynamically
  const $ = cheerio.load(String(file));
  const contentTOC = $("nav.toc").html();
  $("nav.toc").remove();
  const contentHtml = $.html();

  // Combine the data with the id and contentHtml
  return {
    id,
    contentHtml,
    contentTOC,
    ...matterResult.data,
  };
}

I used the following imports to get it all working seamlessly,

The import requirements

import { unified } from 'unified'
import remarkParse from 'remark-parse'
import remarkRehype from 'remark-rehype'
import rehypeDocument from 'rehype-document'
import rehypeFormat from 'rehype-format'
import rehypeStringify from 'rehype-stringify'
import rehypeSlug from 'rehype-slug'
import rehypeTOC from "@jsdevtools/rehype-toc";
import * as cheerio from 'cheerio';

I used cheerio⁶ to build an DOM tree from html text for me to extract the TOC div component using the name nav.tov and use it as a Table of Contents snippet I used in my react components.

This is a screenshot of how I used this piece of code on www.notionworkspaces.com.

The dynamic table of contents section, in left section in the above screenshot.

I hope you found this useful!

👋 -- @TnvMadhav

References

How to Track Gumroad Sales in Notion Using Notion API and Python

TnvMadhav⚡ — Sun, 28 May 2023 04:24:56 +0000

Introduction

In this tutorial, you’ll learn how to track Gumroad¹ sales in real-time in Notion² using 🐍 Python.

You will also learn,

What are APIs?
How to use Gumroad API?
How to use Notion API?
How run a python script in Replit etc

What are APIs?

APIs stand for Application Programming Interfaces.

Let’s break this term down,

→ An Application is any website or app like Twitter, Facebook, Instagram, Reddit etc.

→ Programming Interfaces are a way to talk to these applications

APIs are way to for your code to read information from (or) write information to said applications in an instant.

Why use APIs?

APIs are a great way to automate tasks.

For example, if you want to publish a post to social media, you can use a scheduler like HootSuite³ (or) Buffer⁴.

If you have a social-media post scheduled for 6:00 PM, the scheduling software will write your post to the social media app using their APIs.

So APIs are a way to do things really quickly and really efficiently.

A way to talk to APIs is to use a programming language.

Let’s understand this using an example.

Understanding APIs using a real world analogy

Imagine a postal delivery system.

An API (Application Programming Interface) acts as a request receiver's letterbox in the postal delivery system analogy.

It is the gateway through which a sender (programming language) can send letter (the data) to the request receiver’s letter box (Application’s API endpoint).

Once the data is received by the API endpoint, it can be processed by the the Application, which then returns an acknowledgement along with the requested data if any.

🔑 Remember:

Just as a postal system has rules and constraints, an API system has a defined set of protocols, standards, and endpoints that determine how data can be transmitted between the sender and the receiver.

The sender (programming language) needs to conform to these rules to ensure that the data is correctly received and processed by the API.

In this tutorial, you’ll learn about two applications and talk to them via their respective APIs:

Notion
Gumroad

At the end of your tutorial, you’ll learn to use Notion API and Gumroad API using Python.

🔑 What is Python?

Python is a free and powerful programming language used by millions of software developers to build apps and websites. It’s also super easy to use.

Get started with Gumroad API

We will use Gumroad API to read your store products. This data will be used to update your Notion dashboard automatically using Notion API.

To read your store information, Gumroad needs to know it’s you who are requesting for it. To help gumroad understand this, we will be passing in a shared secret token along with the API request that only you and gumroad know about.

Mission: Acquire Gumroad API Secret Token

If you aren’t already aware, you can generate an API secret token by creating an application from the Settings → Advanced → Applications option from Gumroad’s dashboard.

Steps:

Provide an Application Name (Any name that you are comfortable with)
Redirect UI - we don’t really need this, so you can set the value http://localhost
Upload any icon you want (optional)
Click on Create application button
Once that’s done, click on a button called Generate access token in the next step. Access Token (your API secret key) will be generated. Copy the text and keep it aside safely.

🔑 Do not share your secret key with anyone.

You have now successfully acquired the Gumroad API secret key.

Now you can request to read all your Gumroad store products along with this secret key so gumroad can verify and validate that it’s really you who is making this call.

Mission: Request for your Gumroad store data using Python and your Gumroad secret token

Gumroad has an API documentation, a document that let’s developers know what all API endpoints (links) they can request to read or write information.

✨ If you want to go deeper into the world of Gumroad API, you can get started with this document → https://app.gumroad.com/api

For the sake of this tutorial, the API endpoint we need, to read your store products details is as follows:

https://api.gumroad.com/v2/products

Making a READ request to this API endpoint (url) along with your secret token will return you all your Gumroad store products in a JSON format.

🔑 What is JSON format?

Instead of giving you a unreadable paragraph of your store products and their information, JSON is a structured way of passing along data.

A simple json object has 2 things you need to know, key and value. They’re often called key-value pair.

For example, if you want to pass a question and answer to someone, you can pass it as a JSON object.

{
    "Which planet do we live in?": "Earth",
}

Here, the key is the question, and the value is the answer.

A JSON usually readable by programming languages and big software pass along complex JSON objects which can be read and processed very fast by programming languages under the hood.

Coming back to the Gumroad API, https://api.gumroad.com/v2/products returns the lists of products as a JSON notation.

Let’s understand the full flow here,

You send a request ot Gumroad to this API endpoint (https://api.gumroad.com/v2/products) along with a secret token

This is how we do it in Python:

import requests

url = "https://api.gumroad.com/v2/products"

payload='access_token=ACCESS_TOKEN'

response = requests.request("GET", url, headers=headers, data=payload)

print(response.text)

Gumroad will verify the request and return a list of your products in JSON format.

{
  "success": true,
  "products": [{
    "custom_permalink": null,
    "custom_receipt": null,
    "custom_summary": "You'll get one PSD file.",
    "custom_fields": [],
    "customizable_price": null,
    "description": "I made this for fun.",
    "deleted": false,
    "max_purchase_count": null,
    "name": "Pencil Icon PSD",
    "preview_url": null,
    "require_shipping": false,
    "subscription_duration": null,
    "published": true,
    "url": "http://sahillavingia.com/pencil.psd",
    "id": "A-m3CDDC5dlrSdKZp0RFhA==",
    "price": 100,
    "purchasing_power_parity_prices": {
      "US": 100,
      "IN": 50,
      "EC": 25
    },
    "currency": "usd",
    "short_url": "https://sahil.gumroad.com/l/pencil",
    "thumbnail_url": "https://public-files.gumroad.com/variants/72iaezqqthnj1350mdc618namqki/f2f9c6fc18a80b8bafa38f3562360c0e42507f1c0052dcb708593f7efa3bdab8",
    "tags": ["pencil", "icon"],
    "formatted_price": "$1",
    "file_info": {},
    "shown_on_profile": true,
    "sales_count": "0", # available with the 'view_sales' scope
    "sales_usd_cents": "0", # available with the 'view_sales' scope
    "is_tiered_membership": true,
    "recurrences": ["monthly"], # if is_tiered_membership is true, renders list of available subscription durations; otherwise null
    "variants": [
      {
        "title": "Tier",
        "options": [
          {
            "name": "First Tier",
            "price_difference": 0, # set for non-membership product options
            "is_pay_what_you_want": false,
            "recurrence_prices": { # present for membership products; otherwise null
              "monthly": {
                "price_cents": 300,
                "suggested_price_cents": null # may return number if is_pay_what_you_want is true
              }
            }
          }
        ]
      }
    ]
  }, {...}, {...}]
}

💡 JSON objects are usually complex and are harder to understand for humans but programming languages and computers prefer this over just paragraphs of text to fast processing.

Now, that you’ve understood how Gumroad API works, we will now learn how to upload the store data to Notion.

We can do this using the aforementioned, Notion API.

Getting started with Notion API

Notion API provides capabilities to read and write data into your Notion workspace using code.

Just like for Gumroad API, NotionAPI needs you to pass a secret token along with API request to verify it’s really you who is requesting an API endpoint.

Mission: Generating Notion Integration along with a secret key for NotionAPI

To generate Notion API secret key, follow these steps:

Go to https://www.notion.so/my-integrations/
Click on the + New Integration button
Give it a name
Upload a logo (optional)
Select your Notion Account in case you have multiple accounts
Click on Submit
You’ll be presented with a secret key. Copy this secret key and keep it aside for later use.

🔑 Do not share your secret key with anyone.

Mission: Building your Notion Database

Get the Notion Template

→ I’ve already built the Notion Template. You can grab it here with discount coupon: gumnotion to get 50% off.

If you wan to build one for youself, you can refer to the given python code for the schema design.

Mission: Connecting your Notion Database with the Notion Integration

To connect the dupliated notion database with the notion integration, follow these steps:

Click on the three dots on the top right of the database page
Click on the Add Connection option, search for and select the notion integration you created

🔑 Ensure the Notion integration isn’t connected to any other database (or) page to prevent unwanted problems.

Mission: Making a write request to store data into your Notion Workspace

Once you’ve duplicated the Notion template containing the database. You can now think of making API requests to Notion to write your product details into it.

If you’re interested to dig deeper into Notion API, you can start here → https://developers.notion.com/reference/intro

The core API endpoint we will be using to write data into Notion will be,

https://api.notion.com/v1/pages/

Let’s understand the full flow here,

You make a WRITE request to https://api.notion.com/v1/pages/ API endpoint

import requests
import json

url = "https://api.notion.com/v1/pages/"

payload = json.dumps({
  "cover": {
    "type": "external",
    "external": {
      "url": ""
    }
  },
  "parent": {
    "database_id": "<DATABASE-ID>"
  },
  "properties": {
    "Sales Count": {
      "type": "number",
      "number": 0
    },
    "Revenue": {
      "type": "number",
      "number": 0
    },
    "id": {
      "type": "rich_text",
      "rich_text": [
        {
          "type": "text",
          "text": {
            "content": "<product-id>",
            "link": None
          },
          "annotations": {
            "bold": False,
            "italic": False,
            "strikethrough": False,
            "underline": False,
            "code": False,
            "color": "default"
          },
          "plain_text": "<product-id>",
          "href": None
        }
      ]
    },
    "Link": {
      "type": "url",
      "url": "<product-url>"
    },
    "Price": {
      "type": "number",
      "number": 1000
    },
    "Name": {
      "id": "title",
      "type": "title",
      "title": [
        {
          "type": "text",
          "text": {
            "content": "<Your-Gumroad-Product-Name>",
            "link": None
          },
          "annotations": {
            "bold": False,
            "italic": False,
            "strikethrough": False,
            "underline": False,
            "code": False,
            "color": "default"
          },
          "plain_text": "<Your-Gumroad-Product-Name>",
          "href": None
        }
      ]
    }
  }
})
headers = {
  'Content-Type': 'application/json',
  'Notion-Version': '2022-02-22',
  'Authorization': 'Bearer <secret-token>',
}

response = requests.request("POST", url, headers=headers, data=payload)

print(response.text)

Notion validates and writes received product into your Notion database and returns a sucessful response as a JSON object. This for example:

{
  "object": "page",
  "id": "59833787-2cf9-4fdf-8782-e53db20768a5",
  "created_time": "2022-03-01T19:05:00.000Z",
  "last_edited_time": "2022-07-06T19:16:00.000Z",
  "created_by": {
    "object": "user",
    "id": "ee5f0f84-409a-440f-983a-a5315961c6e4"
  },
  "last_edited_by": {
    "object": "user",
    "id": "ee5f0f84-409a-440f-983a-a5315961c6e4"
  },
  "cover": {
    "type": "external",
    "external": {
      "url": "https://upload.wikimedia.org/wikipedia/commons/6/62/Tuscankale.jpg"
    }
  },
  "icon": {
    "type": "emoji",
    "emoji": "🥬"
  },
  "parent": {
    "type": "database_id",
    "database_id": "d9824bdc-8445-4327-be8b-5b47500af6ce"
  },
  "archived": false,
  "properties": {
    "Store availability": {
      "id": "%3AUPp"
    },
    "Food group": {
      "id": "A%40Hk"
    },
    "Price": {
      "id": "BJXS"
    },
    "Responsible Person": {
      "id": "Iowm"
    },
    "Last ordered": {
      "id": "Jsfb"
    },
    "Cost of next trip": {
      "id": "WOd%3B"
    },
    "Recipes": {
      "id": "YfIu"
    },
    "Description": {
      "id": "_Tc_"
    },
    "In stock": {
      "id": "%60%5Bq%3F"
    },
    "Number of meals": {
      "id": "zag~"
    },
    "Photo": {
      "id": "%7DF_L"
    },
    "Name": {
      "id": "title"
    }
  },
  "url": "https://www.notion.so/Tuscan-Kale-test"
}

🔑 The requests and responses look a bit intimidating because Notion is a complex data store. If everything is done right, you don’t have to personally deal with these responses.

Connecting both Notion and Gumroad APIs

Now that that you’ve gotten some idea on how APIs can be used, let’s try to combine the request and responses to do exactly what we need.

The following is the python script that I wrote ,that you can run to pull product information from gumroad save it to your notion database.

import json
import time
import requests
import traceback
import sys
import os

class MyIntegration:

    def __init__(self):
        """
        Gets required variable data from config yaml file.
        """
        self.my_variables_map = {
            "MY_NOTION_SECRET_TOKEN": os.getenv("MY_NOTION_SECRET_TOKEN"),
            "MY_GUMROAD_SECRET_TOKEN": os.getenv("MY_GUMROAD_SECRET_TOKEN"),
            "NOTION_ENTRIES": {},
        }
        self.fallback_cover_image_url = (
            "https://assets-global.website-files.com/6171b265e5c8aa59b42c3472/6195275a9e5f4655891de886_gum-coins.svg"  # noqa: E501
        )
        self.get_page_and_database_data()

    def get_page_and_database_data(self):
        url = "https://api.notion.com/v1/search"
        headers = {
            "Content-Type": "application/json",
            'Notion-Version': '2022-02-22',
            'Authorization':
                'Bearer ' + self.my_variables_map["MY_NOTION_SECRET_TOKEN"]
        }
        payload = json.dumps({
            "query": 'Store',
            "filter": {
                "value": "database",
                "property": "object"
            }
        })
        response = requests.request("POST", url, headers=headers, data=payload)
        self.my_variables_map["DATABASE_ID"] = \
            response.json()["results"][0]["id"]
        # Database Entries
        url = f"https://api.notion.com/v1/databases/"\
              f"{self.my_variables_map['DATABASE_ID']}/query"
        response = requests.request("POST", url, headers=headers)
        resp = response.json()
        for v in resp["results"]:
            self.my_variables_map["NOTION_ENTRIES"].update({
                v["properties"]["id"]["rich_text"][0]["plain_text"]: {
                    "Sales Count": v["properties"]["Sales Count"]["number"],
                    "Price": v["properties"]["Price"]["number"],
                    "Revenue": v["properties"]["Revenue"]["number"],
                    "Link": v["properties"]["Link"]["url"],
                    "Name": v["properties"]["Name"]["title"][0]["plain_text"],
                    "id": v["properties"]["id"]["rich_text"][0]["plain_text"],
                    "cover": v["cover"]["external"]["url"],
                    "page_id": v['id']
                }
            })

    def get_gumroad_products(self):
        url = "https://api.gumroad.com/v2/products/"
        headers = {
            'Authorization': 'Bearer ' +
            self.my_variables_map["MY_GUMROAD_SECRET_TOKEN"],
        }
        response = requests.request("GET", url, headers=headers)
        for i in response.json()["products"]:
            self.update_notion_entries(i)

    def update_notion_entries(self, data):
        or_data_page_id = self.my_variables_map["NOTION_ENTRIES"].get(
            data['id'], {}
        ).get('page_id', None)
        self.my_variables_map["NOTION_ENTRIES"].update({
            data["id"]: {
                "Name": data["name"],
                "Price": float(data["price"]) / 100,
                "Sales Count": data["sales_count"],
                "Revenue": float(data["sales_usd_cents"]) / 100,
                "Link": data["short_url"],
                "id": data["id"],
                "cover": data.get(
                    "preview_url", self.fallback_cover_image_url
                ) or self.fallback_cover_image_url,
                "page_id": or_data_page_id
            }
        })

    def update_notion_database(self, database_id, data):
        if data.get("page_id"):
            url = "https://api.notion.com/v1/pages/" + str(data.get("page_id"))
            method = "PATCH"
        else:
            url = "https://api.notion.com/v1/pages/"
            method = "POST"
        headers = {
            'Authorization':
                'Bearer ' + self.my_variables_map["MY_NOTION_SECRET_TOKEN"],
            'Notion-Version': '2022-02-22',
            'Content-Type': 'application/json'
        }
        payload = json.dumps({
            "cover": {
                "type": "external",
                "external": {
                    "url": data["cover"]
                }
            },
            "parent": {
                "database_id": database_id
            },
            "properties": {
                "Sales Count": {
                        "type": "number",
                        "number": float(data["Sales Count"])
                },
                "Revenue": {
                    "type": "number",
                    "number": float(data["Revenue"])
                },
                "id": {
                    "type": "rich_text",
                    "rich_text": [
                        {
                            "type": "text",
                            "text": {
                                "content": data["id"],
                                "link": None
                            },
                            "annotations": {
                                "bold": False,
                                "italic": False,
                                "strikethrough": False,
                                "underline": False,
                                "code": False,
                                "color": "default"
                            },
                            "plain_text": data["id"],
                            "href": None
                        }
                    ]
                },
                "Link": {
                    "type": "url",
                    "url": data["Link"]
                },
                "Price": {
                    "type": "number",
                    "number": float(data["Price"])
                },
                "Name": {
                    "id": "title",
                    "type": "title",
                    "title": [
                        {
                            "type": "text",
                            "text": {
                                "content": data["Name"],
                                "link": None
                            },
                            "annotations": {
                                "bold": False,
                                "italic": False,
                                "strikethrough": False,
                                "underline": False,
                                "code": False,
                                "color": "default"
                            },
                            "plain_text": data["Name"],
                            "href": None
                        }
                    ]
                }
            }
        })
        response = requests.request(method, url, headers=headers, data=payload)
        return response.json()["id"]

    def update_indefinitely(self):
        while True:
            try:
                self.get_gumroad_products()
                for _, data in self.my_variables_map["NOTION_ENTRIES"].items():
                    data["page_id"] = self.update_notion_database(
                        data=data,
                        database_id=self.my_variables_map["DATABASE_ID"]
                    )
                    time.sleep(5)
                time.sleep(10)
            except Exception:
                traceback.print_exception(*sys.exc_info())
                # Drop memory and rebuild from existing notion server state
                self.my_variables_map["NOTION_ENTRIES"] = {}
                self.get_page_and_database_data()

if __name__ == "__main__":
    # With 😴 sleeps to prevent rate limit from kicking in.
    MyIntegration().update_indefinitely()

Now, I’ll show you how to run this code on Replit, a popular cloud based code hosting and execution environment.

Running the Python script on Replit:

Create a free account on Replit and login,
Fork my REPL: https://replit.com/@tnvmadhav/Notion-Gumroad-Sync

🍴 Forking means making a copy of someone else’s public repository

After forking into your workspace, share the Gumroad and Notion API secret keys with the Python Script
1. Click on Secrets box in the Left Sidebar
2. Add the following variables & their respective tokens that were kept aside before into the form
  1. MY_NOTION_SECRET_TOKEN
  2. MY_GUMROAD_SECRET_TOKEN

Run the script from your browser!

That’s it!

If all went well, you should see your Gumroad Product information automatically update every 30 seconds.

🔑 You can keep the python script running if you like to keep up with live sales and revenue updates but Replit might charge money to keep your script running forever.

Thanks for reading.

If you liked this guide, you may also like a similar guide I wrote on Tracking Habits in Notion using Python API.

👋 -- @TnvMadhav

Reference

How to Take Screenshots of views in SwiftUI

TnvMadhav⚡ — Tue, 21 Mar 2023 05:13:03 +0000

Introduction

In SwiftUI, you can use ImageRenderer¹ to take screenshots or images as an UIImage object for any SwiftUI view.

Let’s consider a simple Text view in SwiftUI. The goal is to take a snapshot of it using ImageRenderer object.

import SwiftUI

struct ContentView: View {
    var body: some View {
            VStack {
                 Text("Hello, world!")
            }
    }
}

This is how this renders in the Xcode iOS device simulator

How to use ImageRenderer?

Now, let’s try to create a snapshot of this view using ImageRenderer feature.

import SwiftUI

struct ContentView: View {
    var body: some View {
        VStack {
            Spacer()
            VStack {
                Text("Hello, world!")
            }
            Spacer()
            snapshot().resizable().aspectRatio(contentMode: .fit)
                .frame(maxWidth: 100, maxHeight: 100).border(.red)
            Spacer()
        }
    }

    @MainActor func snapshot() -> Image {
        let imagerenderer = ImageRenderer(
            content: VStack {
                Text("Hello, world!")
            }.frame(maxWidth: 100, maxHeight: 100)
        )
        return Image(uiImage: imagerenderer.uiImage!)
    }
}

💡 If the view gets complex, which it inevitably does, it’s better to define it in a function or variable so that it can be reused in snapshot function without too much duplication.

I’ve abstracted the view to be captured, in it’s own variable to clean up duplicate code,

import SwiftUI

struct ContentView: View {

    var viewToBeSnapshotted: some View {
        VStack {
            Text("Hello, world!")
        }
    }

    var body: some View {
        VStack {
            Spacer()
            VStack {
                Spacer()
                Text("↓ The View").underline()
                Spacer()
                viewToBeSnapshotted
                Spacer()
            }
            Spacer()
            VStack {
                Spacer()
                Text("↓ The Snapshot").underline()
                Spacer()
                snapshot().resizable().aspectRatio(contentMode: .fit)
                    .frame(maxWidth: 100, maxHeight: 100).border(.red)
                Spacer()
            }

            Spacer()
        }
    }

    @MainActor func snapshot() -> Image {
        let imagerenderer = ImageRenderer(
            content: viewToBeSnapshotted.frame(maxWidth: 100, maxHeight: 100)
        )
        return Image(uiImage: imagerenderer.uiImage!)
    }
}

The modifiers and annotations around the snapshot image is so that the view and the related ‘snapshot’ are presented cleanly in the demonstration

This is the simplest implementation of ImageRenderer in action.

To understand how awesome this is, I would like to demonstrate a cool experiment below.

Code Snippet

Let’s say the view contains a variable that changes in real-time. We want the corresponding snapshot function to also capture the changes in real-time.

Say the the text contains a number variable that can increase/decrease on some trigger

Here’s the code:

import SwiftUI

struct ContentView: View {

    @State var number: CGFloat = 0

    var viewToBeSnapshotted: some View {
        VStack {
            Text("The number is \(number)")
        }
    }

    var body: some View {
        VStack {
            Spacer()
            VStack {
                Spacer()
                Text("↓ The View").underline()
                Spacer()
                viewToBeSnapshotted
                Spacer()
            }
            Spacer()
            VStack {
                Spacer()
                Text("↓ The Snapshot").underline()
                Spacer()
                snapshot().resizable().aspectRatio(contentMode: .fit)
                    .frame(height: 100).border(.red)
                Spacer()
            }
            Spacer()
            HStack {
                Spacer()
                Text("Slide to update number →")
                Slider(
                    value: $number,
                    in: 0...100,
                    step: 1,
                    label: {
                        Text("Number")
                    }
                )
                Spacer()
            }
        }
    }

I’ve added a simple slider² that updates a @State variable called number from 0 to 100. This number is added part of the Text view.

When the slider updates the number, the view and the corresponding snapshot update in real-time.

Demonstration

In this tutorial, you've learnt how to generate a snapshot of SwiftUI view in only a few lines of swift code.

👋 — @TnvMadhav

References

Daily Task Automator (+ Notion API with Python) - Tutorial

TnvMadhav⚡ — Sun, 08 Jan 2023 10:16:37 +0000

In this tutorial, I'll show you how to set your own Task Automation System with Notion and Python.

Brief Introduction:

We have to follow these two main steps:

Notion integration setup
Python code setup

The link between these two steps, lie in the magic of Notion API.

If you're new to Notion API and Notion Integration itself, I'll explain how you can get started with it in this tutorial.

Want the Notion Template? Get it here!

If you dont have access to the notion template yet, get it 👉🏻 here👈🏻

Now that you've gotten your hands on this template and duplicated into your workspace, let's get started:

The Notion integration setup

This step ensures the python script makes updates to the right notion database.

To ensure this, the python code needs to first 'identify' the notion database so that it can 'talk' to it.

Each notion database has a unique id that is part of the url link.

To make this process easier, we can create a 'secret token' that can be used to connect to your notion database.

Let's do just that,

1. Create notion integration secret token

If you haven't already, create a free Notion Account

Go to https://notion.so/my-integrations/
Click + New Integration button

Give name My Notion Task Automator and submit

)

Show and copy secret token (keep it aside safely)

Click Save Changes

Copy and keep the notion integration secret token so that you can later share it to your python script.

2. Connect it to the notion page

Click ... on the top right corner

Click on Add Connections and search and select the newly created Notion Integration

3. Create tasks to automatically populate everyday

Let’s start with a simple habit,

💪 “Doing 10 pushups at 10:30 AM — everyday of the week”

In Task Definition notion database, click + new, and create a new task:

Task Name Perform 10 Pushups
Date 10:30 AM (any date but time should be 10:30 AM)
Days of the week (Monday, Tuesday, Wednesday, Thursday, Friday, Saturday, Sunday)

Here is how it will look like,

Tasks into the Task Definition database](/2.png)

Woo-hoo! Look at you!

Now let's focus on running the python script. Jumping right in:

The Python code setup

To do this, you don't have to run python on your PC or laptop.

You can:

1. Create a free account on Replit and login,

2. Copy and paste the python code

Copy the below python code:

import requests
import json
from datetime import date
import calendar
import os
import dateutil.parser
import datetime


class MyNotionIntegration:

    def __init__(self):
      self.headers = {
        'Content-Type': 'application/json',
        'Notion-Version': '2021-05-13',
        'Authorization': f'Bearer {os.getenv("NOTION_DAILY_TASK_SCHEDULER_INTEGRATION_SECRET")}'
      }
      self.read_db = self.write_db = self.list_of_tasks = None

    @staticmethod
    def getSameTimeButToday(date_string):
      if not date_string:
        return
      iso_parser = dateutil.parser.isoparse(date_string)
      today_date_obj = iso_parser.replace(
        day=datetime.datetime.now().day,
        month=datetime.datetime.now().month,
        year=datetime.datetime.now().year,
      )
      return today_date_obj.isoformat('T')

    def getReadAndWriteDBs(self, name=None):
      if not name:
        name = "TASKS TEMPLATE"
      url = "https://api.notion.com/v1/search/"
      payload = json.dumps({
        "filter": {
          "property": "object",
          "value": "database"
        }
      })
      response = requests.request("POST", url, headers=self.headers, data=payload)
      databases = response.json().get('results')
      for database in databases:
        if database.get('title')[0].get('text').get('content').upper() == name:
          self.read_db = database
        else:
          self.write_db = database

    def getTodaysTasks(self):
      url = f"https://api.notion.com/v1/databases/{self.read_db.get('id')}/query"
      payload = json.dumps({
        "filter": {
          "property": "Day of the Week",
          "multi_select": {
             "contains": calendar.day_name[date.today().weekday()]
          }
        }
      })
      response = requests.request("POST", url, headers=self.headers, data=payload)
      self.list_of_tasks = response.json().get('results')

    def createTasks(self):
      url = "https://api.notion.com/v1/pages/"
      for task in self.list_of_tasks:
        print(task.get('properties').get('Date').get('date').get('start'))
        payload = json.dumps({
          "parent": {
            "type": "database_id",
            "database_id": self.write_db.get('id'),
          },
          "icon": task.get('icon'),
          "properties": {
            "Due For": {
              "type": "date",
              "date": {
                "start": self.getSameTimeButToday(
                  task.get('properties').get('Date').get('date').get('start')
                ),
                "end": self.getSameTimeButToday(
                  task.get('properties').get('Date').get('date').get('end')
                ),
              }
            },
            "Task": {
              "type": "title",
              "title": [
                {
                  "type": "text",
                  "text": {
                    "content": task.get('properties').get('Name').get('title')[0].get('plain_text'),
                  },
                  "plain_text": task.get('properties').get('Name').get('title')[0].get('plain_text'),
                }
              ]
            }
          }
        })
        response = requests.request("POST", url, headers=self.headers, data=payload)
        print(f"HTTP Status {response.status_code}", response.text)

    def RunWorkflow(self):
      self.getReadAndWriteDBs()
      self.getTodaysTasks()
      self.createTasks()

if __name__ == "__main__":
  MyNotionIntegration().RunWorkflow()

3. Share the notion integration secret key with the python script

4. Run the script from your browser!

After running the Python script on Replit, you can see that today’s tasks are auto-populated in Task Automator database.

🥳 Nice! You now have a automated todo-list!

Add any habits to the Task Definition database and run the python script on replit (automation) every day to auto generate the todo-list in the Task Automator database.

😆 Thank you

...for being part of this experiment and hope you use this to enhance your productivity 😃

If you dont have access to the notion template yet, get it 👉🏻 here👈🏻

If you liked this product, you'll probably love my other products which can be found 🔗 👉🏻 here👈🏻

Let's connect on Twitter @TnvMadhav if you don't wanna miss out on newer projects!

🚀 Save Tweets to Notion (w/ context)

TnvMadhav⚡ — Wed, 08 Dec 2021 04:24:28 +0000

TLDR;

casually surfs twitter...

finds a 🔥 thread...

it's too long to be read now...

bookmarks tweet...

~ THE END ~

Problem

Whenever I find a piece that I am interested in, I would switch to lean mode

After getting a fair amount of knowledge on the topic, I discard the links/ resources and jump back on search mode

I found this approach 'draining' due to the sheer frequency of context switches. Twitter doesn't help much with its lightning fast content feeds.

I was in search of a practice that isolates the knowledge collection and acquisition phases.

I decided to go ahead with the following approach —

"Schedule the knowledge collection step"

...and go consume them at a later point in time.

For example,

"I would collect interesting & new data during the week and learn them leisurely during the weekends"

Twitter has this nice Bookmarks feature.

But there lies a catch,

...if we were to save more than a couple of threads, we should have to doom-scroll our way into the right tweet.

On way to look at this is,

"What if we were to go through all the bookmarked tweets in order and subsequently get rid of them after knowledge acquisition"?

🤨 That's not how I intend to learn things.

What could I possibly do?

Existing Solutions

Twitter has a premium subscription service called 'TwitterBlue' where the user can create bookmark folders, much like how one does in a browser.

This is light years ahead of the normal bookmarks on twitter and not to mention it's also inexpensive... about $3 per month (+/-).

It also provides other features like 'thread-reader' which I would definitely benefit from.

Before I jumped into Twitter Blue, I had this thought.

"I use Notion as my knowledge management tool"

I could just export my bookmarks into a csv/excel file and import it into my notion workspace.

"Notion databases have this amazing feature of filtering records by attributes (columns)"

My Solution

If I were to import tweets into my notion, I would have to manually have to add author and context details.

That's when I got this 💡 moment.

API!

✅ Notion API
✅ Twitter API

I have prior experience with both APIs and they are not that hard to get started with.

I would read tweets from twitter servers and save them into my notion database in an instant... all using code.

Author information can be read from the tweet metadata.

Context is my input, #web3 for example is an indication that the context is web3. Hashtags are a supported metadata value in twitter API responses.

What if....

My code scrapes (searches) for a condition to save the respective tweets/threads into the notion database?

I would reply to a tweet with a command; for example:

@tnvmadhav save=Notion #web3

and the code searches for commands from these types for selected users (whitelist) and save the tweet into their respective notion databases?

✅ MVP

Well, I have built just that and have been using it for a month now.

Ever since, I have saved so much time and boosted my productivity.

I also had this thought of productising this into a tiny SaaS for other users as well.

The product is currently MVP complete and can be used by others here 👇🏻

Get Product Here

https://madhavtnv.gumroad.com/l/bookmark-tweets

Gumroad Sales Tracker (feat. Notion API)

TnvMadhav⚡ — Sun, 29 Aug 2021 11:59:47 +0000

Introduction

Notion is an incredible software that allows creators develop, publish, maintain content with little to no effort. Creator economy is the new deal. If you are a creator who sells ebooks, presets, software bundles like Notion Templates, you must've heard about Gumroad. Creators, like myself and many more use Gumroad on a day to day basis.

Gumroad Provides API that allows the creator to programmatically retrieve their products and user data for analysis/ maintenance purposes.

Notion also provides API support to read and write into the notion workspace with granular scopes w.r.t resources.

So this would naturally mean, one can read gumroad product data and write them into Notion Pages/ Databases using code.

I am here to present just that.

ICYAC: Link to the template at the end of article 👇🏻

👓 My Expectations

Open a Notion page,
Check my Product Sales,
Close the page

Process

[1] Notion Integration

In order to provide edit capabilities on my sales dashboard, I would want to provide access token with edit scope to my updation script (here python).

To get the access token (secret key), I would want to create an internal integration. The steps to create an integration for a database can be found here

Steps to setup an Integration: [Referenced from Official Notion Doc]

Go to https://www.notion.com/my-integrations.
Click the "+ New integration" button.
Give your integration a name, say "Gumroad-Notion-Integration"
Select the workspace where you want to install this integration.
Click "Submit" to create the integration.
Copy the "Internal Integration Token" on the next page and keep it aside

A user must share specific pages with an integration in order for those pages to be accessed using the API.

The script must update the Database as well as the Parent Page. Therefore, we would want to give access to both.

Bear in mind to give database access before parent page access

Share the integration with the Notion Database

Open 'My Products' Database as a Page & click on the Share button and use the selector to find your integration by its name, then click Invite. Your integration now has permission to read, write, and edit the new database.

Ensure that this created integration is not used by any other database

Share the integration with the Parent Page

Open the Gumroad Sales Page & click on the Share button and use the selector to find your integration by its name, then click Invite. Your integration now has permission to read, write, and edit the new database.

[2] Gumroad Integration

To get your gumroad product information, you need to get a gumroad account secret token that will be used by the python script, to pull your product data and push them to your notion database

Login to your gumroad account and visit https://app.gumroad.com/settings/advanced url in your browser
Scroll down to Create Application Section that looks like this,
Give the application a Icon (optional), Name and Redirect URI (Set redirect uri value to http://127.0.0.1 as our notion integration doesn't involve any oauth2 authorisation code flow)
Click on "Create application" to create the application. This will advance you to the next screen, showing the new application
Click Generate access token button to get your access token
Your access token will be displayed. Copy the Access token

[3] Setting up Code Environment

🔥 Leveraging Notion and Gumroad API

Now that we have completed creation of Notion API and Gumroad API Integration. You should have 2 Secret tokens pasted in your text editor safely :-

i. Notion API Secret Token
ii. Gumroad API Secret Token

These two will be used by the python script to :-

Read Product list from your Gumroad Store
Update Products to your Notion Page

Let's walk you through to get it done 😄

Clone to this GitHub Repository to your local device

GitHub - tnvmadhav/notion-gumroad-integration
Open my_variables.yml file using a code/ text editor
Paste the Notion and Gumroad secret tokens like this and save file,

[4] Running Code

After updating your notion and gumroad secret tokens into the my_variables.yml, we can now install the python dependencies to run the code

Running the following command to install python dependencies ,

pip3 install -r requirements.txt

# use `py -m pip install -r requirements.txt` (for Windows System)

Run the following command to start python server

python3 gumroad_notion_product.py   

# Use `py gumroad_notion_product.py` if you are on Windows!

🙌🏻 Watch your gumroad products appear / get updated one by one in 5 to 10 second time intervals!

Note:
If a product sale happens, the sales count will be updated immediately as long as the python script is running!

Conclusion

This was yet another fun experiment of mine and if you liked this, do follow me on Twitter and let me know 😁

You can find the Notion Template + Demo here :-

Gumroad Sales Tracker

# Crypto Profits Tracker (feat. Notion API) using Python

TnvMadhav⚡ — Sun, 04 Jul 2021 16:06:39 +0000

Crypto Boom of early '21

A crackle involving cryptocurrencies took off late 2020, and early 2021 saw a portion of it go mainstream. Cryptocurrency adoption was at all time high, with mighty bursts of bulk investing and selling.
The prices went from all time highs to dwindling states all in the matter of minutes. This was a result of non stop bidirectional flow of coins all over the globe. This forced the investors to keep up to date with extremely mutable prices toggling the state of the market.

Some wanted quick profits, some wanted to be a part of history and others went for the overhaul. Regardless of the intentions, almost all of the players had the trade sites running non-stop on all screens to catch the next dip.

Hours and hours of precious time were lost while keeping up to date with the graphs based profits (or losses) and the inconsistent nature of the the market didn't make the situation any better.

For people like me who waste a lot of time overthinking and make decisions on impulse, I wanted to try sorting this out.

🤔

The idea was to keep me updated without all the glitter and jam. I hated keeping all of the trade firms open multiple tabs and wanted to get this sorted out.

So I planned to use APIs to pull data to my console to check the prices without any other hassle.

ICYAC: Link to the template at the end of article 👇🏻

...

😲 Advent of Notion API

Then Notion launched API Integrations Beta. This immediately lit a spark in my head. Now I didn't have to leave my notion app (workspace).

👓 My Expectations

Open a Notion page,
Check my profits (or losses),
Close the page

👉🏻 "This should take only 1 minute of my time."

Design

I want my net profit but I don't want to leave my Notion Workspace.

I had to get my net profit,

where

n is number of coins I am currently 'hodling',

m is number of coins of type j that I currently have in my wallet

I just want check 'NetProfit' and peace out.

Dashboard Setup

"Notion is incredible with its DIY databases." - We can build wonders without worrying about the code that goes into maintaining it.

I quickly set up a dashboard with the crypto coin entries as a entities. The selected attributes were used to calculate the profit.

Considering that my API Integration is setup and my updation server is running, If I had to ever invest in a new currency, I would just add an entry to this database consisting of Name, Amount Invested and Number of Coins in wallet , I should automatically see my profits taking form in the Notion Database.

Notion Integration

In order to provide edit capabilities on my crypto dashboard, I would want to provide a access token with edit scope to my updation script (here python).

To get the access token (secret key), I would want to create an internal integration. The steps to create an integration for a database can be found here

Steps to setup an Integration: [Referenced from Official Notion Doc]

Go to https://www.notion.com/my-integrations.
Click the "+ New integration" button.
Give your integration a name, say "My-Crypto-Updates"
Select the workspace where you want to install this integration.
Click "Submit" to create the integration.
Copy the "Internal Integration Token" on the next page and keep it aside

Share a database with your integration

A user must share specific pages with an integration in order for those pages to be accessed using the API.

Go back to the 'Crypto Dashboard' & click on the Share button and use the selector to find your integration by its name, then click Invite. Your integration now has permission to read, write, and edit the new database.

Ensure that this created integration is not used by any other database

🔥 Leveraging notion-api

Once you have a database setup in your notion workspace, you would want to have a system where the dashboard is constantly updated with latest crypto currency prices.

😄 Let's walk you through to get it done, using the 💪🏻 power of Notion API,

I am using python as it is more readable and is familiar for most. The script essentially performs 3 actions:

Getting list of CryptoCoins that the user has added to the Notion database (GET~ POST)
Getting prices for the CryptoCoins in List (GET)
Updating the Prices to the Notion Database (PATCH)

The above functionality is implemented using python class methods.

How to use Notion API 💻

Clone the GitHub Repository
(After unzipping) You will find 3 files inside
cd into the directory where the 3 files are present using bash shell or terminal (command prompt for windows)

Install the required software dependencies by running the following command,


pip3 install -r requirements.txt

# use `py -m pip install -r requirements.txt` (for 
  Windows System)

Update and save the my_variables.yml file with your notion related information, (don't include the <> characters)
```
MY_NOTION_SECRET_TOKEN: <insert-your-notion-integration- 
secret-token> # (use the secret from prior step
```

Run the following command to execute python script,

python3 read.py   

# Use `py read.py` if you are on Windows!

🙌🏻 Watch your crypto dashboard update automatically! 🎉

Conclusion

This was another experiment of mine but it really helped me save a lot of time scrolling and clicking around Crypto Trading Firms.

If you liked this, do follow me on Twitter and let me know 😁

👉🏻 The Notion Template can be found here :~

Crypto Dashboard

👉🏻 If you want to offload running and maintaining code to me, you can subscribe to this service as a monthly subscription:

Crypto Profits Tracker App

Building a Time Tracking Progress Application

TnvMadhav⚡ — Sat, 14 Nov 2020 01:33:13 +0000

"Progress is being made..."

It would be really cool to stamp milestones for various types of progress. It could be anything from a projects, careers to vacations. Timestamps are often used to plot the beginning of something and the expected time for something yet to happen or develop. Timestamps can be put on anything to literally mark events into our calendar. It gives us humans clarity about what can and cannot be done in project/life management.

Breaking down processes into manageable tasks gives us a drive to reach each milestone. These milestones are referenced based on timestamps and this temporal labelling helps us manage better to improve the outcome(s).

Time flows until entropy is levelled and for us humans, time is something we use to tame ourselves by setting up calendars. But more often than not, we tend to underestimate the power of time.

"Time is always moving forward" ~ not John Doe

To better understand this power, I feel that being reminded about it helps us focus on the goals we set for ourselves.

With this thought, I tried to create a product that does just that.

Time Progress App

This product reminds a person about the progress made with respect to the current year. This product blurts out the % year progress that has been made until that point. This was created to remind and in turn possibly help one put themselves or their progress in perspective.

The program blurts out on Twitter at TimeProgressApp.

The Year has progressed:
[--------------------||----] 86%
— TimeProgressApp (@TimeProgressApp) November 10, 2020

Did you call me here to get preached?

No, Let's talk about what is happening inside.

The idea is to be as accurate as possible and that is,

Spit out the numbers with little to no latency at all.

I used Golang to manage information and Heroku for hosting

The core functionality is divided into the following steps:

Get the progress %
Is it something significant to be reminded about?

Yes? Pass along the number

No? Repeat

Get the progress %

The progress being made with a chosen time reference( example, a year) can be put as:

Progress % = (time taken to reach now) / (total time duration) x 100

This calculation is isolated into a function GetTimeProgress:

func GetTimeProgress() float64 {
    time_year_start := time.Date(time.Now().Year(), 1, 1, 0, 0, 0, 0, time.UTC) //Time progressed in Hours
    time_year_end := time.Date(time.Now().Year() + 1 , 1, 1, 0, 0, 0, 0, time.UTC) // Get total time in hours
    return (float64(time.Since(time_year_start).Round(time.Second))/ float64(time_year_end.Sub(time_year_start).Round(time.Second))) * 100 // Year Progress
}

Is it something significant to be reminded about?

Now the calling function w.r.t GetTimeProgress can give back any value percentage like 1.0304 (%) or 63.2323 (%). It would be cleaner to output whole numbers but we need the values to be as accurate as possible .

Therefore, we can get the expected behaviour by sending whole numbers when they are returned and we want to capture every whole number.

We are calculating the Progress % based on seconds metric, therefore, we can expect GetTimeProgress function to return whole numbers without significant misses.

So asserting on the whole numbers is the way to go, so something like

GetTimeProgress() % 10 == 0

would work with no problems...

Ok, you know what, I am paranoid, let me consider the possibility of some latency and we get back x.00000003 , then the above method wouldn't help.

So let's fix it, I am allowing the latency to be More than 0 or less than 0.00004

This will result in something like :

answer := GetTimeProgress()
latency := answer - math.Floor(answer)
if latency >= 0 && latency < 0.000004 {
            // Pass along the progress %
    }

We need to do this until the appropriate value is reached, therefore, we can loop this functionality in an indefinite for loop, we then get

for {
        answer := GetTimeProgress()
        latency := answer - math.Floor(answer)
        if latency > 0 && latency < 0.000004 {
            return answer
        }
    }

let us give the CPU some time to catch a breath, but we don't want to miss the intended progress % value, so let's add a single second (significant for a CPU) sleep time.

The latency consideration does in fact help as a second of sleep can possibly be the cause for missing the capture of whole number progress %

for {
        answer := GetTimeProgress()
        latency := answer - math.Floor(answer)
        if latency > 0 && latency < 0.000004 {
            return answer
        }
        time.Sleep(time.Second * 1)
    }

Let's wrap this in a function,

// Return the right data
func IsProgressed() float64 {
    for {
        answer := GetTimeProgress()
        latency := answer - math.Floor(answer)
        // When to return the data...
        if latency > 0 && latency < 0.000004 {
            return answer
        }
        time.Sleep(time.Second * 1)
    }
}

Now that we have isolated the logic for calculating time progress (GetTimeProgress) & check for the progress significance (IsProgressed) , we can rely on calling IsProgressed which will return a number when acceptable progress is made.

Design:

Now, we have two concerns

Make sure that the calling function for IsProgressed doesn't just exit after getting the number.
Tight coupling of the 'calculating functionality' with 'handling functionality'

Say the we are using a third party API to send the progress data, it is better to isolate the API call logic from the calculating logic. The calculating logic is time sensitive (to seconds accuracy) and API calls when are network based would lead to some time overhead.

We can isolate the two functionality with the help of channels and goroutines

Goroutines are light weight threads and channels are messaging pipes (here blocking) can be used for communication between (concurrently running) go routines. The system is best explained with the above diagram.

The handling go routine (main function) is configured at the receiving end of the channel that waits indefinitely until the channel returns the types value (Progress %)

A Channel is created using:

c := make(chan float64)

A channel can only take in values that are of a specific type (defined during declaration).

A goroutine can be created by just stating go keyword before a certain function call, this spawns a separate child routine which executes the function.

go Updated(c)

Here the child (calculation goroutine) is used to calculate and return the progress % when encountered and produce it into the channel passed as an argument.

// calculating goroutine:

updateChan <- IsProgressed()

When the channel is loaded with a value, at the receiving end, the main (handling goroutine) will read the value and move on to make the API call (if any)

// handling goroutine:

answer := <- c

// Handle the data received from the channel...
sampleApiCall(answer)

Once the data is consumed from the channel, we then call Twitter API to tweet the progress data.

The Final Code looks like:

package main

import (
    "fmt"
    "time"
    "math"
)

// Long running data producer
func Updated(updateChan chan float64) float64 {
    for {
        updateChan <- IsProgressed()
        // After milestone is reached, sleep for some time to prevent sending same progress data multiple times...
        time.Sleep(time.Hour * 8)
    }
}

// Return the right data
func IsProgressed() float64 {
    for {
        answer := GetTimeProgress()
        latency := answer - math.Floor(answer)
        // When to return the data...
        if latency > 0 && latency < 0.000004 {
            return answer
        }
        time.Sleep(time.Second * 1)
    }
}

// Just give me the damn number!
func GetTimeProgress() float64 {
    time_year_start := time.Date(time.Now().Year(), 1, 1, 0, 0, 0, 0, time.UTC) //Time progressed in Hours
    time_year_end := time.Date(time.Now().Year() + 1 , 1, 1, 0, 0, 0, 0, time.UTC) // Get total time in hours
    return (float64(time.Since(time_year_start).Round(time.Second))/ float64(time_year_end.Sub(time_year_start).Round(time.Second))) * 100 // Year Progress
}

// Used to create a simple progress bar using keyboard characters
// Example: [-----||-----]
func getBitBar() string {
    myString := "["
    var value string
    for i := 1; i <= 25; i++ {
        if i == int(temp_answer)/ 4 {
            value = "||"
        } else {
            value = "-"
        }
        myString += value
    }
    myString += "]"
    return myString
}

func main() {
    // Start log...
    fmt.Println("Time Progress App")
    // Create a channel
    c := make(chan float64)
    // Start a routine in parallel with following statements
    go Updated(c)
    // Get the result from the routine
    for {
        answer := <- c
        // Tweet the Result using Twitter API
        fmt.Println(tweet(fmt.Sprintf("The Year has progressed:\n %s %d", getBitBar(), int(answer)) + "%"))
    }
}

This is finally deployed on Heroku and is currently live at TimeProgressApp. So make sure to follow 😃 to receive time progress %.

In the end, this was a little experiment of mine and if you felt this was Insightful, do follow me on Twitter at TnvMadhav and let me know 😄