Forem: Pruthvi Kumar

Semantic Similarity for Personal Knowledge Management

Pruthvi Kumar — Sun, 02 Feb 2025 01:23:58 +0000

In today's digital world, we are all knowledge workers 🧑‍💻, constantly swimming in a sea of information 🌊. Whether it's code snippets 💻, research papers 📚, meeting notes 📝, blog posts 📰, or just our daily reflections 🤔 – we are bombarded with digital content. We rely heavily on search 🔍, often keyword-based, to navigate this information. But what if there was a way to go deeper 🤿, to find connections based on meaning, not just matching words?

What if your note-taking system could understand the meaning of your notes, not just the specific words you happened to use? 🤯

This question has been a driving force for me. Keyword search is undeniably useful 👍, but it has inherent limitations for effective Personal Knowledge Management (PKM) 🧠. Consider this 👇:

You might have notes on "improving focus and concentration" spread across different documents, using varied phrasing each time. Keyword search might miss these connections if you don't use consistent terminology. 😔
Related ideas might be expressed using synonyms or different language entirely, making it challenging to link them using simple keyword queries. This hinders our ability to build a truly interconnected understanding of our knowledge. 🔗

The Idea: Semantic Similarity for PKM ✨

This is where the concept of semantic similarity becomes incredibly valuable for Personal Knowledge Management. Instead of solely relying on matching keywords, semantic similarity aims to understand the underlying meaning of your notes and documents.

Think of it this way 👇:

Keyword search for PKM: It's like organizing your physical notes by only looking at a few keywords you wrote on the top of each page. You might miss the deeper connections and broader themes across your notes. 🙈
Semantic similarity for PKM: Imagine having a system that understands the concepts within your notes and can intelligently group together related information, even if you used different words to express those concepts over time. 💡

Semantic similarity for PKM means moving beyond surface-level word matching to capture the essence of your ideas 💫, allowing you to discover connections based on meaning and build a richer, more interconnected personal knowledge base. 🕸️

For example, consider these notes someone might have in their PKM system:

"Need to find strategies to minimize distractions during deep work sessions."
"Experimenting with time-blocking techniques to improve concentration."
"My focus is scattered; looking for ways to enhance attention span."

A semantic similarity approach can recognize that these are all related to the core concept of "improving focus and concentration" within a PKM system, even with varied phrasing. 🎯

Exploring Semantic PKM with Cipher 🛠️

This exploration of semantic similarity for PKM led me to build a tool called Cipher. It's a personal project, driven by my own need to better manage and understand the connections within my digital notes and reflections, with the goal of enhancing my personal knowledge management practices.

Cipher, at its core, is an experiment in applying semantic similarity to organize and analyze my journal entries and notes. It's not about replacing traditional search, but rather augmenting my PKM workflow by offering a new, meaning-based perspective on my information.

(Watch a quick demo of Cipher in action!)

How It Might Work ⚙️

Cipher uses what I call Contexts — groupings of thoughts that seem to naturally appear as I write. These contexts might emerge on their own as I’m writing, suggesting connections I might not have consciously made. It’s a bit like having a conversation where things you’ve thought about before might naturally come to mind. 🔄

These Contexts aren’t strict categories or folders, but more like flexible connections that seem to shift as my thinking evolves. As I write, Cipher looks for links between my entries, perhaps building a kind of map 🗺️ of my connected thoughts. The system seems to learn from what I write, noticing if certain ideas or themes come up repeatedly.

Sometimes, seeing these connected thoughts might offer a slightly different perspective on things I’ve experienced. Some of these observations might resonate immediately, while others might take longer to consider. There’s no pressure to do anything with these observations — they’re simply there as potential starting points for further thought, if I choose to explore them. 🤔

The goal isn’t to force a strict structure. Instead, Cipher offers a flexible way to see if there are natural rhythms and patterns in my own life, based on the idea that understanding might sometimes come from careful observation, rather than actively searching for it. It’s about creating a space where understanding could potentially arrive in its own time. 🧘

This idea of connected Contexts is vaguely similar to how our own brains work, right? We might create webs of connections between different thoughts and experiences. Our minds seem to constantly link things together, building a rich understanding of the world that perhaps changes as we learn. Cipher, in a small way, is inspired by this, hoping to help me (and maybe others) connect the dots in my own reflections. 🧩

My Own Experience 📝

Cipher started as something I built for myself, a digital way to keep a journal. I wanted to try and track not just my thoughts, but potentially how they might shift over time. Regular journals and note apps often felt either too structured or not structured enough for what I was looking for. I was hoping for something that could adapt as my understanding grew, maybe helping me see broader patterns in my daily thoughts.

What began as a simple journaling tool gradually became something that seemed more interesting. The patterns that appeared in my writing sometimes hinted at connections and insights I hadn’t consciously noticed. This somewhat natural process of discovery is what Cipher is about today. ✨

If This Resonates 🤝

This isn’t about widespread adoption or any commercial goals. It’s about sharing something that came from a personal need and has been quietly helpful in my own experience. If the idea of a subtle aid for noticing patterns in your own thoughts sounds potentially interesting, you’re welcome to explore Cipher through a small beta program - https://cipher.sysapp.dev. I’m keeping the group small to allow for thoughtful development and refinement based on genuine feedback. My hope is simply that it might be valuable to others who are curious about exploring their own thoughts in this way. 🙏

I Built a CLI Tool That Tells Me When I'm Over-Engineering (Again) 🤖

Pruthvi Kumar — Fri, 01 Nov 2024 07:43:05 +0000

Built this to catch myself when I'm 47 commits deep into "optimizing" a simple function. It's a pragmatic approach to detecting when you're over-engineering - using ML to analyze work patterns and suggest when to just ship it.

Some actual results from 3 months of usage:

31% reduction in context-switching
47% better task completion rate
Significantly fewer 3AM "this needs a complete rewrite" episodes

(And yes, it tells me when I'm over-engineering the over-engineering detector😁)

GitHub
Blog
LinkedIn

Serverless backend service that uses Typescript & Golang - Automated. Well, mostly

Pruthvi Kumar — Thu, 07 Apr 2022 07:12:00 +0000

https://gitlab.com/1x-eng/g-zippy

Sparkle - Master computer programming by spaced & repetitive learning.

Pruthvi Kumar — Sun, 22 Sep 2019 05:05:33 +0000

Perpetual Learning is the greatest constant in a Software Development career (at least if you do care about your career). The technology landscape changes too fast and we need to keep up. That’s just the way it is.

Software programming can be mastered by spaced repetition. And, flashcards are just meant to help you get there.

SPARKLE is free software that lets you create flashcards in respective sections and offers a whole lot more. Thanks for checking it out - https://sparkle.apricity.co.in

SPARKLE - Where you start as a novice; and, leave as a master!

Art of mastering computer programming - Sparkle

Pruthvi Kumar — Fri, 20 Sep 2019 07:17:04 +0000

Have you ever used flashcards? Sure you did! We’ve all been there and it seems like they are making a comeback. Programmers can remember programming knowledge they don’t use on a day to day basis using spaced repetition.

Sparkle, is a free to use software (web & mobile friendly) that lets you create your flashcards in neatly separated sections. Plus, there are a lot more goodies. Check it out - https://sparkle.apricity.co.in

Sparkle - where you start as a novice and end as a master!

ETHEReact: Build a Full Stack Decentralized E-Commerce App, Deploy to Ethereum Virtual Machine & Interact using ReactJS

Pruthvi Kumar — Fri, 26 Jul 2019 05:02:36 +0000

TL;DR

This is not an article explaining Blockchain or marketing Crypto.
This article focuses on facilitating any developer &/ blockchain enthusiasts build a prototype on blockchain; end-to-end. You’ll build a basic decentralized e-commerce platform by the end of this article.
In this article, you will write two contracts in Solidity (one for supplier & another for the customer), compile and migrate your contract to EVM, create a simple ReactJS frontend, connect your frontend to EVM via web3.js.

Alright, now that you are interested, let’s start coding! No fluff right!

Pre-requisites:

This article assumes you understand how to install NodeJS on development machines, how to download newer packages from npm and a basic understanding of ReactJS.
That you have the latest version of NodeJS installed on your development machine.
Install Truffle npm install -g truffle Truffle is a development and testing framework for Ethereum. With Truffle, you can compile, test and deploy smart contracts on Ethereum network.
Install TestRPC npm install -g ethereumjs-testrpc TestRPC is Node.js based Ethereum client for testing and make developing Ethereum applications much faster.
Install Create-React-App in some other folder. Probably in your root. npm install -g create-react-app
Create a folder for your app. May be call it MyDApp mkdir MyDApp & change to newly created folder cd MyDApp
npx create-react-app FrontEnd This will create the front-end. Production grade React built by issuing just one command! Can’t get any better! Right? For the sake of this demo, we will not build bundles; but, just run the app. npm start in the same folder; you should have ReactJS smooth and running on port 3000 (Unless it was directed to run on any other port)

Back-end:

With all the pre-requisites satisfied, lets concentrate on having a functional Blockchain back-end for our DApp.

Change to app’s root folder cd MyDApp
Create a dedicated structure for Back-end. mkdir BackEnd & cd BackEnd
Initialize Truffle by truffle init (In Windows 10, if this results in *Microsoft JScript runtime error, *change your command to truffle.cmd init) This will initialize the folder structure to create and deploy smart contracts.
Navigate to file named truffle.js and make sure it has this contents:

Inside contracts folder, create a new file named Supplier.sol Now, lets code some Solidity to define our Suppliers back-end logic.

The above solidity contract is comprised of:

State variables: Think of these as global variables available across the contract.
Transactions: These are functions that add &/ update &/ delete data in the Blockchain.
Getters: These are functions too; but, only read data from the Blockchain.

It’s important to understand that Transaction functions cost ‘GAS’ when dealing with Blockchain. That means, operations like storing new data, updating &/ deleting existing data from Blockchain is not free! Explaining GAS is not in the scope of this article. This stackexchange answer does a pretty good job in explaining this unit.

However, reading data from blockchain can be free! Getter functions help us achieve that.

The above contract empowers our backend to support ‘Suppliers’ in an e-commerce platform.

We need code for Customers. Inside contracts folder, create a file named Customer.sol and lets code some solidity again. It’s fun isn’t it?

Now, lets write some logic for migrating these contracts to Blockchain network.

Inside ./MyDApp/BackEnd/migrations folder, create a new file named 2_deploy_contract.js which should contain the following code

Congratulations! You just completed coding a Blockchain back-end for your cool decentralized e-commerce.

Now, lets get it working.

Solidity is a compiled language. Lets compile our code using truffle compile (on Windows10, you might want to do truffle.cmd compile )

No compile time errors? Congratulations. Now, we have to migrate our contracts to Ethereum network. Open a new command prompt / terminal and navigate to ./MyDApp/BackEnd/ . It’s time to get Ethereum running locally on your machine. Do that using testrpc -l 9000000000 -p 8484

-l 9000000000 indicate the gas limit of the network. On production systems, this is real money! (Crypto of-course). On your machine, you have the control! (Feels rich right?)
-p 8484 indicate the port on which your RPC server must listen.

Leave that cmd/terminal running. Open another one and change to ./MyDApp/BackEnd . We gotta migrate our contracts into the network that we just started. We do that using truffle migrate (Again, on Windows10, you might want to use truffle.cmd migrate )

Whilst your contracts get migrated to the network, notice activity on another terminal where you have your rpc server running. Blockchain has recorded your contracts into its system! Phew, we are flying!!!

Contract Address’ & other config

For your front end to communicate with Blockchain back-end, it needs a few config information. Lets grab that info and save it for future reference.

Grab address of each contracts deployed to network.
Open another cmd/terminal and navigate to ./MyDApp/BackEnd Get truffle console using truffle console (on Win10, truffle.cmd console )
Get Supplier contract address using Supplier.address Store this address. We will need this shortly.
Get Customer contract address using Customer.address Store this address. We will need this shortly.
For web3 to marry Ethereum to React, it needs contract ABI. Again, explaining ABI is out of scope for this article; but, this stackexchange answer does a good job.
Get ABI for Supplier contract using JSON.stringify(Supplier.abi) . Terminal spits a huge string which is essentially JSON once you remove start and end quotes. Use JSONlint to help you format the JSON. Store this JSON. We will need this shortly.
Similarly, get ABI for Customer contract using JSON.stringify(Customer.abi) Again, remove the first and last quote, format and save this for future reference.

That’s it. You have coded, compiled, migrated and obtained config for Blockchain back-end of your cool decentralized e-commerce app!

There is only front end remaining. Let’s get to it!

Front-end:

As a part of pre-requisites, we installed create-react-app globally & created a react app for ourselves. Lets change to that directory. cd ./MyDApp/FrontEnd

We need to provide config information for React to connect to Blockchain backend. Let’s do that using the following code:

Create a config file named EthereumSetup.js in /src directory. This is the time for you to grab hold of Supplier and Customer ABI’s obtained from previous steps.

Create front-end components for Suppliers to add products to marketplace & process purchase-orders. Within /src folder, create a file named Suppliers.js

Notice how transaction functions (line 115, 127, 140) require gas to complete a transaction. If you are wondering how I’ve pre-determined gas limit to be 20000; The answer is, It’s random! In test networks, you will not have to worry about gas limits. It is a serious consideration when you go to public networks because, we are talking real money then. Higher the gas available for a transaction, it will be ranker higher in priority for the network to complete the transaction.
Now, similar to Suppliers.js we need front end logic and components for Customers to view items available in the marketplace and place order(s). Under /src directory, create a file named Customers.js

Also, note the event listeners. It’s important to trigger these event listeners after the component has been loaded to DOM. You can choose to watch events from a particular block or choose to go from 0 to latest . Using latter, you will catch all events recorded in the Blockchain from the very first block!
With Supplier and Customer components now ready to go, we are one step away from completing our front-end code. Let’s nail it.
Find App.js and make sure it reflects this code:

That’s it. You’ve completed front-end code for your decentralized app! Pretty amazing! You’ve come a long way from having nothing to a full stack DApp! Congratulations!!
To bring the site alive, open a cmd/terminal and change to ./MyDApp/FrontEnd and issue this command — npm start
Whoa!!! Can you see activity on the blockchain backend? That is all your event watchers from front-end subscribing to event emitters from the back-end. For those who know pub-sub; this is sort of similar! You now have a functional Blockchain powered decentralized e-commerce app!
If you are confused about what next, check line 22 to 27 in App.js for instructions.

Conclusion

This article helps you create a full stack DApp from scratch. You will understand how to write smart contracts in Solidity, start your Blockchain back-end and expose it to HTTP via RPC server. You will then use Web3 on the front-end to marry Ethereum back-end to React front-end. Hence ETHEReact!

I will be excited to learn about newer products you build from your newly acquired skills. I’ll be more than happy to support you in this exciting journey. You can find the codebase for ETHEReact here.

For further questions/ feedback / comments, you could reach me @ pruthvikumar.123@gmail.com

Happy coding,

Pruthvi | http://www.apricity.co.in

PROTON - Pragmatic automagic python3 framework

Pruthvi Kumar — Thu, 25 Oct 2018 22:05:52 +0000

PROTON aims at easing server-side development for all Python enthusiasts. Essentially, by running a shell command, developer will auto generate necessary Model, Controller and APIs! All of this with connectivity to Transactional Databases (PROTON supports Postgresql, MySQL & SQL Server),caching (Redis middleware), Auto generated OpenAPI specs & descriptive logging! One command, to get a production ready server-side stack!

https://github.com/PruthviKumarBK/PROTON

Happy coding,
Pruthvi | http://www.apricity.co.in

PROTON - Automagic python3 framework | Python MVC containerised

Pruthvi Kumar — Wed, 17 Oct 2018 21:04:15 +0000

PROTON is a high-level Python framework that facilitates rapid server-side development with clean & pragmatic design. Thanks for checking it out!

https://github.com/PruthviKumarBK/PROTON

PS: If you are experienced &/ interested in opentracing, connect with me if you are interested to extend PROTON's core to support method level profiling.

ETHEReact - Decentralized E-Commerce using Ethereum & ReactJS

Pruthvi Kumar — Sun, 14 Oct 2018 07:48:17 +0000