Forem: silverboi

How to Add Your SSH Key to Your GitHub Account

silverboi — Wed, 13 Mar 2024 19:07:47 +0000

This is just a self reminder of how to do this and not have to google it every time I need to add a new one.
These steps are solid as they are, but might change over time to create more robust documentation on it.
These steps are all performed from the terminal.

Create your ssh key with your github email

ssh-keygen -t ed25519 -C "your_email@example.com"

Optional

Name your file name and route when prompted this:(use the site name you are creating the key for id purposes).

Enter file in which to save the key (/Users/dgplatac/.ssh/id_ed25519):

Enter a passphrase for added security when prompted this:

Enter passphrase (empty for no passphrase):

Add Your SSH Key to the ssh-agent

eval "$(ssh-agent -s)"
ssh-add ~/.ssh/id_ed25519

Copy your ssh key to the clipboard

cat ~/.ssh/ed25519.pub | pbcopy

Remember to use the correct file name and path.

Add Your SSH Key to Your GitHub Account

On GitHub, navigate to your settings by clicking your profile photo, then click Settings.
In the user settings sidebar, click SSH and GPG keys.
Click New SSH key or Add SSH key.
In the “Title” field, add a descriptive label. For example, if you’re using your personal laptop, you might call this key “Personal Laptop”.
Paste your key into the “Key” field.
Click Add SSH key.

Test Your SSH Connection

ssh -T git@github.com

The following message should display

The authenticity of host 'github.com (IP ADDRESS)' can't be established.
RSA key fingerprint is SHA256:xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx.
Are you sure you want to continue connecting (yes/no/[fingerprint])?

answer yes and the following should display:

Hi username! You've successfully authenticated, but GitHub does not provide shell access.

Silverboi

SQL Notebook I Wish I Had Crafted Sooner in My Coding Path

silverboi — Thu, 01 Feb 2024 19:27:13 +0000

BigQuery is very similar to SQL and is easy so here is my cheatsheet.

SELECT:

SELECT * FROM table;
SELECT ... FROM ... WHERE condition
SELECT ... FROM ... WHERE condition GROUP BY field;
SELECT ... FROM ... ORDER BY field1, field2 ASC;
SELECT ... FROM ... WHERE condition ORDER BY field1, field2 DESC;
SELECT ... FROM ... LIMIT 10;
SELECT DISTINCT field1 FROM ... table;
SELECT DISTINCT field1, field2 FROM ... table;

CONDITIONS:

field1 = value1
t1.field = t2.field
field1 <> value1
field1 LIKE 'value _ %'
field1 IS NULL
field1 IS NOT NULL
field1 IS IN (value1, value2)
field1 IS NOT IN (value1, value2)
condition1 AND condition2
condition1 OR condition2

AGGREGATION FUNCTIONS:

MAX()
MIN()
AVG()
CONCAT()
STRING_AGG()

COMMON TABLE EXPRESSIONS (CTE):

WITH CTE_table_name AS ( query );

JOINS:

Silverboy

Exploring Feature Stores: Personal Insights and Notes on Hopsworks pt.2

silverboi — Mon, 29 Jan 2024 01:16:49 +0000

More personal insights on Hopsworks and MLOps. Remember that these are random personal notes in no particular order:

Roles in Machine Learning Projects: For building a professional, robust, maintainable machine learning project you need at least least two people; someone to focus on the actual ML model and other to do MLOps (the way I see it this is data engineering + DevOps).
Hopsworks python client: This connection provides access to projects, feature groups, feature views, model registry and model serving. For using them is wise to have all of the former things in a class object (at least that's the way I'm building my project).
Feature Group Queries as Objects: Hopsworks queries are objects and work for querying from feature groups; read() method is for reading into data frame.
Model Serving: Hopsworks provides model serving capabilities but I haven't had time to mess around with this yet, will probably try next.
Model Registry: Per Hopsworks documentation

It enables developers to publish, test, monitor, govern and share models for collaboration with other teams. The model registry is where developers publish their models during the experimentation phase. The model registry can also be used to share models with the team and stakeholders.

So basically Hopsworks' model registry is for storing ML models through serializable objects like pickle files.

Silverboy

MLOps project setup: upload ML models to Hopsworks model registry

silverboi — Sun, 28 Jan 2024 22:59:26 +0000

Here is the previous entry in case you didn't see it. Also this is the code repo for the series and is also updated accordingly.

In this entry these things that are happening:

Create a ML model training pipeline
Upload the ML model into Hopsworks model registry

Notes on step 1

The model selection is entirely contingent on your data and the objectives within your project, some model examples are: Support Vector Machines (SVM), clustering and neural networks. Right now model serves primarily as a placeholder it's not particularly functional yet. However, the intention is for me to refine it over time creating of more than one version. This just serves as a starting point for the development of my MLOps pipeline.

Notes for step 2

This step detailed breakdown is in this gist lines 43-47:

get your project's model registry
create a model in your project's model registry
push using save() method

Silverboi

Exploring Feature Stores: Personal Insights and Notes on Hopsworks

silverboi — Sun, 21 Jan 2024 00:15:47 +0000

Taking a slight detour from my ongoing MLOps project to delve a bit deeper into Hopsworks.

These are my random notes in no particular order, since starting to work on the MLOps project:

The Purpose of Feature Stores: At heart, feature stores are dedicated spaces for storing features essential for machine learning models. These features have already undergone the process of feature engineering.
Exploring External Tables in Hopsworks: Hopsworks offers support for external tables such as JDBC-enabled source, Snowflake, Data Lake, Redshift, BigQuery, S3, and others as feature groups. This is something I've yet to try, but I'm keen to explore. Their JDBC connector is not great yet.
Feature Views: Hopsworks 'Feature Views' are essentially a composite of features drawn from various feature groups. I haven't had the chance to use feature views yet.
Easy Use with Python: Hopsworks has user-friendly interfaces, especially Python.
Feature Groups Versioning: Hopsworks allows versioning of feature groups. This is an advantage over using just a simple SQL table. It works for having a history track of your features and allows better management and control over ML model's data.

Silverboy

MLOps project setup: upload data to Hopsworks feature store

silverboi — Fri, 12 Jan 2024 03:41:33 +0000

This is how the project will be looking after completing this step. Go see previous entries one, two and three if you want to follow along.

You'll need a Hopsworks account for this step so create one.

This is the complete code repo with the code to push a feature group to Hopsworks but basically what is happening is:

retrieve data from Supabase database and load it into a pandas dataframe
upload features into Hopsworks feature store

In between this two steps you can perform the following:

feature engineer, this is an optional step which I plan on completing later.
reassure data quality, this can be achieved through great expectations library this is also optional and for now I'm skipping this step

Here is a gist only with code for uploading feature groups.

MLOps project setup: Airbyte - Supabase connection

silverboi — Tue, 02 Jan 2024 02:13:03 +0000

Access the Airbyte web interface with default creds airbyte password.

Create a new source using Sample Data (Faker).
Leave default values or customize it to your own taste.
Test and save.

Create a new Postgres destination.
Leave default values as they are and those blank text boxes fill 'em up with db info from supabase step.
You can find DB password text box in optional fields.
Test and save.

Setup the connection between the source and destination.
Wait for the job to run and the db to be populated.

Once it runs go to supabase dashboard and double check the inserted data.

Kudos man, now you have working (fake) data.

MLOps project setup: Supabase

silverboi — Mon, 01 Jan 2024 21:29:13 +0000

Go get yourself a supabase account.

Once logged in to your account, create a new project and while creating it make sure to remember the db password you provide.

Go into project settings -> database and grab the following out of connection info:

Host
Database name
Port
User
Password

You'll need them later for setting up an airbyte destination.

That's it for supabase.

MLOps project setup: Airbyte

silverboi — Mon, 01 Jan 2024 19:54:58 +0000

I'm just trying to leave my own footprints here, kind of like a school notebook.

Download and install Airbyte on your local, I used the instructions in the official Airbyte documentation: https://docs.airbyte.com/deploying-airbyte/local-deployment.
Obviously follow the installation instructions for your specific operating system.
Ensure that Airbyte is up and running by going into: http://localhost:8000 and entering creds.