Forem: Bhavani Ravi

Python super() vs Base.init Method

Bhavani Ravi — Mon, 08 Jan 2024 08:58:00 +0000

When defining a subclass, there are different ways to call the __init__ method of a parent class. Let’s start with a base class and go through each of these methods.

For this blog, it’s better that you open a sample.py python file and follow along.

class Base(object):
    def __init__ (self):
        print "Base created"

Method 1 :: Using Parent Reference Directly

class ChildA(Base):
    def __init__ (self):
        Base. __init__ (self)
        print ("Child A initlaized")

Method 2:: Using Super with child class

class ChildB(Base):
    def __init__ (self):
        print ("Child B initlaized")
        super(ChildB, self). __init__ ()

Method 3:: Using the super method

class ChildC(Base):
    def __init__ (self):
        super(). __init__ ()
        print ("Child C initlaized")

Questions

What are the pros and cons of each method?
Is there one single right way to do this?

When you run this code as a single Python script, initializing child classes A, B, and C., You will notice absolutely no difference.

cA = ChildA()
cB = ChildB()
cC = ChildC()

How can we demystify this? Let’s start with the documentation.

As of Python3 super() is same as super(ChildB, self).__init__(). That rules out one of the three methods.
To compare Base.__init__(self) and super().__init__() we need multiple Inheritance. Consider the following snippet

class Base1:
    def __init__ (self):     
        print ("Base 1 created")     
        super(). __init__ ()

class Base2:
    def __init__ (self):     
        print ("Base 2created")     
        super(). __init__ ()

Let’s write the subclasses

class A1(Base1, Base2):
    def __init__ (self):     
        print ("Child A1 Initialized")     
        super(). __init__ ()

class A2(Base2, Base1):
    def __init__ (self):     
        print ("Child A Initialized")     
        super(). __init__ ()

Let’s initialize the objects

a1 = A1()
print ("\n\n")
a2 = A2()

On running the above snippet, we get the following Output.

Base 1 created
Base 2 created
Child A1 initialized

Base 2 created
Base 1 created
Child A2 initialized

In the case of class A1(Base1, Base2) Base1 is initialized first, followed by Base2. It’s the inverse for class A2. We can conclude that the methods are called based on the order of specification.

When you use the Base1.__init__() method, you lose out on this feature of Python
When you introduce new hierarchies, renaming the classes will become a nightmare

So how does Python know which function to call first, introducing MRO(Method Resolution Order)

Method Resolution Order

Method Resolution Order(MRO) denotes the way a programming language resolves a method or attribute.

In the case of single inheritance, the attribute is searched only at a single level, with multiple inheritance Python interpreter looks for the attribute in itself then its parents in the order of inheritance. In case of A1 -> Base 1 -> Base 2

One can use the mro function to find the method resolution order of any particular class.

print (A2.mro())

** Output**

[<class ' __main__.A2'>, <class ' __main__.Base2'>, <class ' __main__.Base1'>, <class 'object'>]

The Last Punch

Comment out the super calls in base class and check the ouput of your script.

class Base1:
    def __init__ (self):
        print ("Base 1 created")
        # super(). __init__ () 

class Base2:
    def __init__ (self):
        print ("Base 2 created")
        # super(). __init__ ()

What do you see?

Base 1 created
Child A1 initlaized
Base 2 created
Child A2 initlaized

[<class ' __main__.A2'>, <class ' __main__.Base2'>, <class ' __main__.Base1'>, <class 'object'>]

In spite of having Base1 and Base2 in the MRO list, mro won’t resolve the order unless the super() function is propagated all the way up to the base class, i.e., Python propagates the search for the attribute only until it finds one. Comment on the init method in Base1 and see for yourself.

class Base1:
    pass

    # def __init__ (self):
    # self.prop1 = "Base 1"
    # self.prop11 = "Base 11"
    # print ("Base 1 created")
    # # super(). __init__ ()

Output

Since Python can’t find the __init__ method in Base1 it checks Base2 before sending it all the way to object class

Base 2 created
Child A1 initlaized

Base 2 created
Child A2 initlaized
[<class ' __main__.A2'>, <class ' __main__.Base2'>, <class ' __main__.Base1'>, <class 'object'>]

People ask me why I love Python so much. It’s not because Python is simple and easy. It is all these things Python does to make things easy for us, sometimes a little hard, too :)

_Want to build a project with Python, Join the Python to Project Bootcamp

Caching in Python

Bhavani Ravi — Tue, 10 Jan 2023 19:50:03 +0000

Caching, is a concept that was gifted to software world from the hardware world. A cache is a temporary storage area that stores the used items for easy access. To put it in layman’s terms, it is the chair we all have.

This blog covers the basics of

What are Caches?
Caching Operations
Cache Eviction Policies
Implementation of Cache Eviction Policies
Distributed Caching
Caching In Python

Conventional Caches

In the world of computer science, Caches are the hardware components that store the result of computation for easy and fast access. The major factor that contributes to the speed is its memory size and its location. The memory size of the cache is way less than an RAM. This reduces the number of scans to retrieve data. Caches are located closer to the consumer (CPU) hence the less latency.

Caching Operations

There are two broad types of caches operation. Cache such as Browser caches, server caches, Proxy Caches, Hardware caches works under the principle of read and write caches.

When dealing with caches we always have a huge chunk of memory which is time consuming to read and write to, DB, hard disk, etc., Cache is a piece of software/hardware sitting on top of it making the job faster.

Read Cache

A read cache is a storage that stores the accessed items. Every time the client requests data from storage, the request hits the cache associated with the storage.

If the requested data is available on the cache, then it is a cache hit.

if not it is a Cache miss.

Now when accessing a data from cache some other process changes the data at this point you need to reload the cache with the newly changed data this it is a Cache invalidation

Write Cache

Write caches as the name suggests enables fast writes. Imagine a write-heavy system and we all know that writing to a DB is costly. Caches come handy and handle the DB write load which is later updated to the DB in batches. It is important to notice that, the data between the DB and the cache should always be synchronized. There are 3 ways one can implement a write cache.

Write Through
Write Back
Write Around

Write Through

The write to the DB happens through the cache. Every time a new data is written in the cache it gets updated in the DB.

Advantages - There won’t be a mismatch of data between the cache and the storage

Disadvantage - Both the cache and the storage needs to be updated creating an overhead instead of increasing the performance

Write Back

Write back is when the cache asynchronously updates the values to the DB at set intervals.

This method swaps the advantage and disadvantage of Write through. Though writing to a cache is faster of Data loss and inconsistency

Write Around

Write the data directly to the storage and load the cache only when the data is read.

Advantages

A cache is not overloaded with data that is not read immediately after a write

- Reduces the latency of the write-through method

Disadvantages

Reading recently written data will cause a cache miss and is not suitable for such use-cases.

Cache Eviction Policies

Caches make the reads and write fast. Then it would only make sense to read and write all data to and from caches instead of using DBs. But, remember the speed comes only because caches are small. Larger the cache, longer it will take to search through it.

So it is important that we optimize with space. Once a cache is full, We can make space for new data only by removing the ones are already in the cache. Again, it cannot be a guessing game, we need to maximize the utilization to optimize the output.

The algorithms used to arrive at a decision of which data needs to be discarded from a cache is a cache eviction policy

LRU - Least Recently Used
LFU - Least Frequently Used
MRU - Most Recently Used
FIFO - First In First Out

LRU - Least Recently Used

As the name suggest when a cache runs out of space remove the least recently used element. It is simple and easy to implement and sounds fair but for caching frequency of usage has more weight than when it was last accessed which brings us to the next algorithm.

LFU - Least Frequently Used

LFU takes both the age and frequency of data into account. But the problem here is frequently used data stagnates in the cache for a long time

MRU - Most Recently Used

Why on earth will someone use an MRU algorithm after talking about the frequency of usage? Won’t we always re-read the data we just read? Not necessarily. Imaging the image gallery app, the images of an album gets cached and loaded when you swipe right. What about going back to the previous photo? Yes, the probability of that happening is less.

FIFO - First In First Out

When caches start working like queues, you will have an FIFO cache. This fits well for cases involving pipelines of data sequentially read and processed.

LRU Implementation

Caches are basically a hash table. Every data that goes inside it is hashed and stored making it accessible at O(1).

Now how do we kick out the least recently used item, we by far only have a hash function and it’s data. We need to store the order of access in some fashion.

One thing we can do is have an array where we enter the element as and when it is accessed. But calculating frequency in this approach becomes an overkill. We can’t go for another Hash table it is not an access problem.

A doubly linked list might fit the purpose. Add an item to the linked list every time it is accessed and maintain it’s a reference in a hash table enabling us to access it at O(1).

When the element is already present, remove it from its current position and add it to the end of the linked list.

Where to Place the Caches?

The closer the caches are to its consumer faster it is. Which might implicitly mean to place caches along with the webserver in case of a web application. But there are a couple of problems

When a server goes down, we lose all the data associated with the server’s cache
When there is a need to increase the size of the cache it would invade the memory allocated for the server.

The most viable solution is to maintain a cache outside the server. Though it incorporates additional latency, it is worth for the reliability of caches.

Distributed Caching is the concept of hosting a cache outside the server and scaling it independently.

When to Implement Caches?

Finding the technologies to implement caches is the easiest of all steps. Caches promises high speed APIs and it might feel stupid not to incorporate them, but if you do it for wrong reasons, it just adds additional overhead to the system. So before implementing caches make sure

The hit to your data store is high
You have done everything you can to improve the speed at DB level.
You have learnt and researched on various caching methodologies and systems and found what fits your purpose.

Implementation In Python

To understand caching we need to understand the data we are dealing with. For this example I am using a simple MongoDB Schema of User and Event collections.

We will have APIs to get User and Event by their associated IDs. The following code snippet comprise a helper function to get a respective document from MongoDB

def read_document(db, collection, _id):
    collection = db[collection]
    return collection.find_one({"_id": _id})

Python inbuilt LRU-Caching

from flask import jsonify
from functools import lru_cache

@app.route(“/user/<uid>“)
@lru_cache()
def get_user(uid):

    try:
        return jsonify(read_user(db, uid))
    except KeyError as e:
        return jsonify({”Status": “Error”, “message”: str(e)})

LRU-Caching like you see in the following example is easy to implement since it has out of the box Python support. But there are some disadvantages

It is simple that it can’t be extended for advanced functionalities
Supports only one type of caching algorithm
LRU-Caching is a classic example of server side caching, hence there is a possibility of memory overload in server.
Cache timeout is not implicit, invalidate it manually

Caching In Python Flask

To support other caches like redis or memcache, Flask-Cache provides out of the box support.

config = {’CACHE_TYPE’: ‘redis’} # or memcache
app = Flask( __name__ )
app.config.from_mapping(config)
cache = Cache(app)

@app.route(“/user/<uid>“)
@cache.cached(timeout=30)
def get_user(uid):
    try:
        return jsonify(read_user(db, uid))
    except KeyError as e:
        return jsonify({”Status": “Error”, “message”: str(e)})

With that, We have covered what caches are, when to use one and how to implement it in Python Flask.

Resources

7 Programming Anti-Patterns

Bhavani Ravi — Sat, 23 Jul 2022 17:35:53 +0000

Things to unlearn as you move from CodeNewbie to Developer

Do you know “Spaghetti code”? The code base where everything lies everywhere and no one knows where what starts and where things end. It sounds like a plot for a project to end in a boom. But there are other equally scary anti-patterns you must avoid to run a successful project and to set yourself as an experienced developer.

Anti-patterns are bad programming practices. Every programmer has their style. Certain styles would cause more harm to the codebase than good.

Knowing, Recognizing, and steering away from anti-patterns will set you up for a good start in your Software Development journey.

1. Copy-Paste(Voodo) programming

Copying text word-by-word is a practice writers use to develop their writing style. As a Python trainer, I have strongly vouched for copy-pasting code as a way of learning programming patterns and idioms.

That is great when you are practicing. The problem arises when you bring this practice to work. E.g., when I was a newbie developer, I copied a piece of code to read files from stackoverflow and played around with it, and it seemed to work all fine until I gave it some real input and my system froze. I did not account for processing large files. The program loaded the whole file into the memory and took over my system.

Blindly copy-pasting code without understanding the crux of it will do more harm to you and the system you are building.

So, what is the alternative? Should you never copy-paste from Stackoverflow?

How to Copy for Practice?

Understand the problem, write it down
Find a solution that you find useful
Copy-paste it into your code editor
add comments to each line depending on your understanding
Add print statements to prove your understanding is right
Run the code
Validate your assumptions
If the code doesn’t solve the problem, tweak it
Once the code works, make it better — easier to read, test, and optimize for space and time

How to Copy-Paste at work?

When copy-pasting at work, see how the code interacts with the rest of the system.

Does it solve the problem you’re trying to address?
Does this hold good for all test cases?
Does the code style match with the rest of the codebase

2. Cargo Cult programming

It’s Friday evening. Your customer has reported a high-priority bug. Every alert is going off on slack. The management wants a fix ASAP. You don’t know the root cause of the bug, but you know adding an if statement will suppress it “for now”. That’s cargo-cult programming.

Including code or program structures that serve no real purpose or understanding of the underlying problem by trial and approach is called Cargo cult programming.

It might sound a lot like Voodo programming, but the focus here is on viewing solutions clearly as a larger part of the system rather than whether it works or not.

2.1 Cargo Cult Software Engineering

Cargo cult software engineering is when you adopt a SE practice based on its popularity or mandated as “best practice” without understanding the reasoning behind it.

How to Avoid Cargo Cult Programming?

Know the system before making changes to it
Ensure quick fixes are patched in only after the bug has been identified
Ensure to make educated decisions about incorporating practices
Incorporate code review and pair programming in your development practice

3. A Big Ball of Mud

A big ball of mud is a software system lacking perceivable architecture. Although undesirable from a software engineering point of view, such systems are common in practice due to business pressures, developer turnover, and code entropy. They are a type of design anti-pattern.

To Avoid Ball of Mud

Understand the problem at hand, write it down
Split them into individual solvable pieces
Think of these pieces as individual functions
Merge repeating sections into a single function. Always DRY (Do not repeat yourself)
Solve the individual problems(use stackoverflow only here)
Write tests for the individual pieces and how they work together

A codebase is already a big ball of mud that needs refactored. Make a deal with your product team and get some time on this engineering effect. It will help you fast-track future development.

4. Code/Design smell

Spinning up a set of features without proper design often results in a Code/Design smell. Poor design decisions will leave the system fragile and challenging to maintain.

Adding code to an already flawed design will result in the accumulation of technical debt.

One of the easiest ways to identify code smell is to look for

Functions doing more than one task leave room for design smell.
Classes/Modules that are too long
Code is not segregated into functions and modules
Code not testable

5. Ego Programming

Ego programming is when a programmer infuses their programming style as opposed to the one adopted by the organization because they have done so.

E.g., In Python, one-liners are fancy but tend to make life difficult for other developers.

When the whole Python world follows snake_case for naming functions, it won't hold good for one person to use camelCase only because they come from JS.

There is no harm in developers trying to infuse a change that will improve the lives of other developers. Whenever such change is incorporated, certain checks should pass.

Is the code readable?
Is the code modifiable?
Is the code testable?

6. Golden Hammer

Using one solution to every problem since it was proven successful in the past. Just because one tool, solution, or hack worked for some problem, it won’t work for all problems.

Google might invest a whole SRE team to support the scale of millions, but when it comes to you, all you need is an alerting system ensuring you’re paged when things go down.

7. Boat Anchor

You know how project managers are; they will come back with more features. I have heard more than one tech lead mentioning this.

Adding more code to support a feature that you might need in the future is a boat anchor problem. It increases complexity.

I already hear you saying, “But I know my PM they will come back with features”. In that case, leave in abstractions, not the actual logic.

8. Dead Code

Have you ever come across a function where no team member knows what it does, but it works and is referred to everywhere. That’s dead code because

no one knows how to change/fix it.
One day, one tweak, and it’s going to make the system dead

Fixing Dead Code

From your POV, write down what the code does in a text editor line by line
Add test cases to test each line of the code
Run the test cases one by one as the test passes, and add a comment next to it

Special Thanks to Bowrna and Abhishek for proofreading the blog

Originally published at https://thelearning.dev.

Pytest — How to test that a function is called with specific parameters?

Bhavani Ravi — Mon, 27 Jun 2022 03:12:52 +0000

Unit tests are written to test single possible units of your system. When you interact with an external system, such as reading a file from AWS S3, your tests must run without any dependency on this external system. The standard testing practice for this use case is to mock the function interacting with this external system and ensure everything else works as expected.

To illustrate this example, Let’s start with a simple Python function that reads data from a CSV and writes it to another CSV.

def read_data(input_path=None, output_path=None): 
    input_path = input_path or "input_data/data.csv" 
    output_path = output_path or "output_data/output.csv" 
    with open(input_path, "r") as fin: 
        with open(output_path, "w") as fout: 
            fout.write(fin.read())

The test case we are interested in is to ensure that the open function is called with the passed param and not the default. Since open is a Python built-in, we don't have to test its logic explicitly.

Let’s start by mocking the builtins.open built-in mock_open function using the patch decorator.

import pytest
import script

from unittest.mock import patch, mock_open, call

[@patch](http://twitter.com/patch)("builtins.open", new_callable=mock_open, read_data="data")
def test_file_logic(mock_open_obj):
    read_data(input_path, output_path)

Next, we have to ensure the read_data function calls the open method with the expected input and output path. assert_has_calls is a function of the mock object, which will check the trace of calls and ensure that the call with specific params is present.

import pytest 
import script 
from unittest.mock import patch, mock_open, call 

@patch("builtins.open", new_callable=mock_open, read_data="data") def test_file_logic(mock_open_obj): 
    read_data(input_path, output_path)     
    mock_open_obj.assert_has_calls([call(expected_output_path, "w"),         
                                    call(expected_input_path, "r")], 
                                    any_order=True)

The any_order is important because open go through various stack trace like __enter__ and doesn't appear in the order.

Originally published at https://thelearning.dev.

How to Create a Download Button Using React

Bhavani Ravi — Thu, 02 Jun 2022 03:09:54 +0000

A guide on creating a download button using React.

When I was building 100 Ideas, I wanted to give users an option to export their content in a text file. It was super simple. This post is an attempt to share that knowledge and log it for the future. Assuming you already know the basics of React, let's begin.

Create a Button

<div className="btnDiv">
     <button id="downloadBtn" value="download">Download</button>
</div>

Add Event Handler

const downloadTxtFile = () => {
    console.log("download logic goes here")
}

<div className="btnDiv">
     <button id="downloadBtn" onClick={downloadTxtFile} value="download">Download</button>
</div>

Next, we need to hook an event handler to the button's onClick event. Let's do that and see if everything works.

Add Download Logic

To download a file, we need four things.

The content that's supposed to go into the file
The file object
A link to download the file object
Finally, simulate that the user clicking the link

const downloadTxtFile = () => {
    // text content
    const texts = ["line 1", "line 2", "line 3"]

// file object
    const file = new Blob(texts, {type: 'text/plain'});

// anchor link
    const element = document.createElement("a");
    element.href = URL.createObjectURL(file);
    element.download = "100ideas-" + Date.now() + ".txt";

// simulate link click
    document.body.appendChild(element); // Required for this to work in FireFox
    element.click();
}

Notes

Date.now() is to add the timestamp to file downloads
The file blob doesn't add a newline to the texts automatically. You can use [texts.join('\n')] to add a new line

Originally published at https://thelearning.dev.

3 Steps Improve Code Readability

Bhavani Ravi — Fri, 09 Jul 2021 23:02:53 +0000

Do you want to write great code or step up as a developer? Every programmer wants to write perfect code, but due to ever-demanding business needs, hustle culture, we need to make trade-offs. We often don’t get to write that ideal code we dream of. Despite all that, we can ensure that the code we write is reasonably good. Here are three tips that I use that prevent your code from becoming a noodly mess.

1. Design Before Code

Instead of jumping ahead and starting to code, when you see a problem, take a step back, and I said the problem into a chunk of solvable pieces. You lose the power of visualization when you start dealing with files with classes and functions. The program deconstruction helps us to look at the problems on a deeper level. I often take a piece of paper and lay down the necessary modules to solve the problem and visualize how they interact.

2. Is it Maintainable?

Once you have a solid design, it’s time to poke holes in it and find flaws. Does it bear the load the Product managers going to bring in for the next release? Trust me, “Agile,” “Hustle,” and “startups” are not doing us any favor. You don’t have to over-engineer, but being aware of future changes lets us design change-friendly interfaces.

3. Is it intuitive?

An intuitive Codebase is a step up from readable code. Remember, you are not coding just for yourself. You are coding for your team, your future self, and as the common joke goes, the Psychopath will take over your code 10 years from now. A well-modularised code that reads like English is much better than a single file with 200 lines.

Look at the following example,

questions = read_quiz_questions() 
for question in questions: 
     display_question(question) 
     answer = get_answer() 
     calculate_score(question, answer)

Without telling much about what this program does, You understand what’s happening. It’s more than readable. It’s intuitive.

The above methods ensure that the code is easy to consume for other team members or the future you. While these give you reasonably good code, remember that it is just a starting point. Just because you are doing the above three doesn’t mean ignoring test cases, documentation, design patterns, etc. We will touch base on them in future posts.

Originally published at https://thelearning.dev.

Practice for Perfection

Bhavani Ravi — Mon, 05 Jul 2021 15:45:08 +0000

Let’s say you are watching a youtube video. A youtube video that shows you how to wear a tie. Assuming that you have no prior knowledge of how to tie a tie, do you think you will acquire that skill by watching the video. Nope! I wish I have that kind of magic in me. Unfortunately, we are all humans. There is only one way to do it.

First, you have to buy a tie, if you already don’t own one. Next, you gotta stand in front of the mirror, have the video on by the side. Try to follow through the video step by step. If you make a mistake, you unwrap it and try again. After 10 attempts or so, it gets a little better. Maybe at the 15th attempt, you jump up in joy because you finally got it.

Not just tieing a tie, you can’t learn a skill without practicing it. Think about this, can you build your muscles just by watching your trainer workout? Can you learn pottery by watching a video? Remember all the skills you possess today, right from riding a bike to reading this piece of text. You learned it by practice.

Don’t stop before starting. Pick up that skill you want to acquire, practice every day until it finally clicks.

Originally published at https://thelearning.dev.

Photo by fauxels from Pexels

5 Packages that Python Devs Use Everyday

Bhavani Ravi — Fri, 11 Jun 2021 05:15:41 +0000

As a Python developer, our life lives and revolves around the packages we use. For every problem we have, there is a package in Pypy that would lend its helping hand. Out of them, the following are some of the libraries that developers use every day. These packages are so special that it gets mentioned in almost every interview, there is a rich community around each of them

Pandas

Anytime someone talks about data manipulation or data analytics, Pandas is your way to go. It has a rich set of features that would help you in the Data Science world and the web world. I have found Pandas almost useful in anything from data transformation to data streaming across various data points.

Stars — 29.9K

Contributors- 970

Stackoverflow Questions — 203,458 questions

Flask/Django

The moment you want to build a web application, Flask or Django would be your go-to options. Django comes with many pre-defined modules that get you up and running without a lot of effort. If you have customization and unique functionalities in your web app, flask might be a better option. Both serve it’s purpose well and has rich features.

Flask

Stars — 55.5K

Contributors- 538

Stackoverflow Questions — 45,720 Questions

Django

Stars — 57.7K

Contributors- 723

Stackoverflow Questions — 267,651 Questions

Requests

Another common use case of Python other than building web apps and APIs is consuming existing APIs. Requests library had made it real simple for developers with just 4 key functions. Requests rule the API consumption spectrum of Python despite having urllib in its stdlib

Stars — 45.3K

Contributors — 563

Stackoverflow Questions — 16,909 Questions

Pytest

Pytest is gaining popularity because of its ease of use and makes writing test-cases a pleasurable experience. Just like requests, despite having the unittest built-in in the Python library, Pytest has proven to be an efficient and effective way of writing test cases.

Stars — 7.37K

Contributors- 518

Stackoverflow Questions — 6401 Questions

SQLAlchemy

SQLAlchemy is one of the richest python packages. Every time a Python developer thinks of databases, they will think of SQLAlchemy. People often think SQLAlchemy is all about ORMs for non-Django people, but that’s not the case. SQLAlchemy provides a developer complete control over how they want to play with DBs. Want to run raw queries? You got it. Want to use a complete object-driven approach? You got it. Got a DB already and want to build on top of it, reflect the DB into objects and proceed

Stars — 3.82K

Contributors- 356

StackOverflow Questions — 18,595 Questions

As a Python Developer, I never start a project without one of these libraries. All the libraries come with rich documentation, so what are you waiting for? Fire up your code editor, pip install, and start experimenting with their examples.

Evolve from a Python newbie to a software Engineer. Be the first to get more Resources, tutorials, blog posts, paid and free workshops — Subscribe

Originally published at https://thelearning.dev.

How to get past the beginner stage in Python?

Bhavani Ravi — Sat, 05 Jun 2021 04:28:25 +0000

So you learned python, picked one of the best tutorials online, and aced through it. You loved Python and decided it was the one. But you ended up feeling completely lost when you tried to do your own project. It feels as if you have learned nothing.

You wonder if those hours you have spent on learning the language is utter waste, that you missed something. You are urged to start from the beginning again in hopes of finding something new, only to end up with the same old variables, data structures, and loops.

You switch to other online resources, books that teach Python, but the result are the same. The next resort is to look for advanced python concepts and receive pointers like

Start building your own projects
Learn advanced concepts
Solve Leetcode problems
Keep experimenting

By this time, either it feels too humongous of a steep climb, or you have concluded programming not something that’s meant for you.

I would have suggested the same 3 months back. Infact I would have even nudged you to attend my Bootcamp. As I was teaching the last batch, something came out. It’s not the technology, programming construct, or what I taught that made the difference. What made the difference was the ability to split the problem into smaller solvable chunks. It is this ability that gave my students to tackle any projects that they wanted to build. You would be using this skill years into the industry as a Software Engineer.

As soon as I started to program in chunks, I realized how everything pieced together.

So how to do it? Given a problem, how do you dissect them into smaller solvable chunks? Like how the construction of a building starts with a basement followed by layers of other constructional elements, your program starts with a base structure and layers of logic plugged into it. The skill is not rocket science but comes with a little bit of practice. Your practice can be as simple as solving a single problem in a single Python script.

Program Construction

Let’s start with an example, A calculator program that does the basic mathematical operation. Your first intuition might be to start searching “How would you write a calculator app in Python?” instead, hold your horses. The goal is not to find the answer but to train your brain to solve the problem by yourself.

Now take down a piece of paper or open a notepad and answer these questions? It might not come naturally the first few times and that’s completely fine. But before moving on write down the answers.

What are the core functionalities of the script?
What inputs do I need to achieve it?
How do I get the user input?
What data do I need to achieve it?
Do I need to store anything? If so, how? — Files, DB, cloud storage

Here is a tip to ace it, have a conversation like talking to your friend. As the questions that are popping up and answer them. Spend about 10 minutes on it. Set the timer and start

Done? Continue reading at https://thelearning.dev.

Build a Python App with HarperDB

Bhavani Ravi — Thu, 11 Feb 2021 14:35:59 +0000

With the boom of SaaS, people want to spend more time solving business problems rather than spend their time dealing with technical problems like managing a database.

DBaaS (Database as a service) entered the SaaS ecosystem to solve just that. While you are busy solving business problems, services like HarperDB will take care of managing, scaling, replicating, and optimizing your databases.

HarperDB is a cloud service that exposes your data via an API endpoint, making it language agnostic. It supports all major languages like Python, Java, NodeJS, etc.,

HarperDB — SQL vs NoSQL

If you have worked with MySQL or Mongo, you will get the pros and cons of both. While relational systems provide concrete transactional and ACID properties, NoSQL DB works wonders for reading dominant use cases or streaming data.

HarperDB wants to bring the best of both worlds to the database community by providing a system that.

Scales like a NoSQL system
Is ACID Compliant
Cost and Computing efficient

Python App with HarperDB

Now that we have understood what HarperDB brings to the table let’s build an application using Python while parallelly exploring the features provided by HarperDB.

Note :: The rest of the blog assumes that you have a fair understanding of Python and Flask. For an in-depth tutorial on API, schema, and application design, check outmy flask starter tutorial and come back here.

If you are a “show me the code” kind of person, check out the Github repository, or if you want to follow along, here is the summary of steps we will perform.

Setup HarperDB instance
Create a Schema and Database
Create a Python Project
Accessing HarperDB via python-SDK
Creating and exposing the REST APIs

Setup HarperDB Instance

Create your HarperDB account

The signup process is super smooth. All you have to do is go to the HarperDB signup page and fill in your details.

Create a new DB instance

Once you are inside the account, you can create the DB instance. Harper’s free tier provides 1 GB of free storage and 0.5 GB of RAM.

Click on create a DB instance. You will have two options.

Create an HarperDB cloud instance that is created, managed, and scaled by HarperDB itself.
Install HarperDB in your own server and add it to the UI.

For this blog, I am going to stick with the cloud instance. Give your DB instance a name, select the tier, username, password, and click on create. Make a note of the username and password. It will come in handy when we connect to this DB from our python app.

The instance will take a few minutes to spin up. Once it is ready, we can create tables as per the needs of our application.

The Application

We all want to read books. I truly believe that the habit of reading books is essential for upgrading your thoughts. Now, if that motivated you enough, you would jump the bandwagon and start reading 100 books, but that’s not essential. Reading books is all about quality than quantity.

Let’s build a Book Journal app where we

Track the book we are reading, Add a Book Name, Start Date, End Date, URL, Author and Why are you reading this book
Add our notes, thoughts, and highlights.
Export notes for summary/blogs

Create schema and tables

Let’s start building the application by creating the necessary schema and tables. Click on the created DB instance, create a schema and table.

Besides the table name, HarperDB also asks to enter or assign the field for a hash_attribute. This attribute is similar to a Primary Key in a relational database, used as the unique identifier(inserted_hashes) for each record. A good name for hash attribute would be just id or something more meaningful to the table book_id

The created table will have basic fields like the hash_attribute, created, and updated time. HarperDB will automatically add additional fields when we insert data later.

Creating Python Application

For the python app, we are going to use a flask app with harperdb-python-SDK. Start by cloning the sample repo and checking out the tutorial branch.

git clone [git@github.com](mailto:git@github.com):bhavaniravi/python-harperdb-app-tutorial.git
git checkout tutorial

The packages required for this app are in requirements.txt Let’s install them using the pippython package manager.

pip install -r requirements.txt

Accessing DB using Harper Python SDK

HarperDB provides Python, Java, Node, Rust SDKs. These SDKs make the CRUD apps as easy as calling a function. Once we have this logic in place, it is just a matter of exposing them via our APIs.

Get the DB_URL from the HarperDB cloud username and password we noted earlier and add them to the file. To create a .env file, rename the file from the repo, and update the following fields.

DB_URL=""
DB_USER=""
DB_PASSWORD=""

Let’s Connect to the HarperDB instance using the above credentials. In a typical MVC pattern, all the DB related logic goes into the models file. Open models.py add the following code.

import harperdb
import os

from dotenv import load_dotenv
load_dotenv() # loads the .env file 

schema = f"book_repo"

db = harperdb.HarperDB(
    url=os.environ["DB_URL"],
    username=os.environ["DB_USER"],
    password=os.environ["DB_PASSWORD"])

Run the model file using python app/model.py. We have connected to HarperDB successfully if there are no errors. We will use the above dbobject to create, delete and update DB records.

HarperDB Python SDK — A sneak peek

The HarperDB Python SDK includes Python3 implementations of HarperDB API functions and wrappers for an object-oriented interface. Here is the GitHub repo. We will be using the following in our book application.

Using the SDK methods, we can write our own models.py file, which creates, updates, deletes the books from the table

class BookModel:
    table = "books"
    def create(self, book_list):
        return db.insert(schema, self.table, book_list)

    def update(self, book_list):
        return db.update(schema, self.table, book_list)

    def delete(self, id):
        return db.delete(schema, self.table, [id])

    def get(self, id):
        return db.search_by_hash(schema, self.table, [id], get_attributes=['*'])

    def get_all(self):
        return db.sql(f"select * from {schema}.{self.table}")

I have added a BookService class in service.pya module that ingests business logic. TheBookService acts the controller of our MVC comes in handy to use adding notes to a specific book. It packages the request JSON and uses the update method internally.

from models import BookModel

class BookService:
    def __init__ (self):
        self.model = BookModel()

    def create(self, params):
        return self.model.create(params)

    def update(self, item_id, params):
        if isinstance(params, dict):
            params = [params]

        for param in params:
            param["id"] = item_id

        return self.model.update(params)

    def delete(self, item_id):
        return self.model.delete(item_id)

    def get_all(self):
        response = self.model.get_all()
        return response

    def get(self, book_id):
        response = self.model.get(book_id)
        return response

    def add_notes(self, item_id, notes):
        if isinstance(notes, dict):
            notes = [notes]
        return self.update(item_id, {"notes":notes})

Exposing and consuming APIs

Let’s create a flask app, and that uses BookService and expose our book CRUD API. If you are new to Flask, do switch to the flask beginner tutorial. Here is a little flask primer for you to get going forward.

# hello.py
from flask import Flask # import flask
app = Flask( __name__ ) # create an app instance

@app.route("/") # at the end point /
def hello(): # call method hello
    return "Hello World!" # which returns "hello world"

if __name__ == " __main__": # on running python app.py
    app.run(debug=True, port=5000) # run the flask app

Add the above code to app/hello.py and run python app/hello.py it will start a flask app. Open localhost:5000 in your browser, and you will see your hello world flask app.

The cloned version of our book app’sapp/app.py the file looks like this.

from flask import Flask, request, jsonify
from dotenv import load_dotenv
load_dotenv(verbose=True)

# import BookService Here

app = Flask( __name__ )

_# Add the Flask routes here_ 

if __name__ == " __main__":
    app.run(debug=True, host='0.0.0.0', port=8888)

For us to integrate the API endpoints with the BookService we need to import it.

# import BookService Here
**from service import BookService**

Next, let’s add the independent flask routes to create, update, delete and add notes to the books app.

Create Book

For the create API, we will route /booksrequests via the HTTP POST method. It calls the BookService we created earlier to call our DB via HarperDB SDK, which creates and returns unique IDs. The response from HarperDB is converted into a JSON string using the Flaskjsonify method and returned as an HTTP response.

# app.py

@app.route("/book", methods=["POST"])
def create_book():
    return jsonify(BookService().create(request.get_json()))

Start the application by running python app/app.py

To test our API, I have used the insomnia applicationand configured the APIs. You can also configure using the insomnia-api-consumer.json file from the repo. Download and install insomnia. To import the API consumer, click on the drop-down at the left corner and click on Import/Export

Click on import data -> From file and choose the insomnia-api-consumer.json from the file picker.

You will have all the book APIs configured once the insomnia file is uploaded. Select the create_bookfrom left nav bar and click on the send button. You will see the API response on the right.

Note down one of the book IDs. We will use it when testing the other APIs.

Similarly, we can create the rest of the APIs and test them using the insomnia app.

Update Book

For the update API, we will route /booksrequests via the HTTP PUT method along with a . The acts as a template to receive book IDs generated by create API.

Note: The rest of the APIs calls the respectiveBookService method.

# app.py

@app.route("/book/<book_id>", methods=["PUT"])
def update_book(book_id):
    return jsonify(BookService().update(book_id, request.get_json()))

When testing the update API on insomnia, select update_bookand ensure that you replace the existing ID with the one you noted down from create API.

Get book details

To get a book detail, we will route /books/requests via the HTTP GET method.

# app.py

@app.route("/book/<book_id>", methods=["GET"])
def get_book(book_id):
    return jsonify(BookService().get(book_id))

Add notes

To add notes to the API, we pass the book ID to /note POST API. Internally the service method calls the book update and adds the note to the respective book.

# app.py

@app.route("/book/<book_id>/note", methods=["POST"])
def add_notes(book_id):
    return jsonify(BookService().add_notes(book_id, request.get_json()))

Though notes are stored as a param of books, we give it a logical separation using a different API. As an exercise, try adding notes via the update API we created before.

Once you have used these APIs to add, update and remove books, you can check these records in the HarperDB studio interface.

_Voila!!! W_e have successfully built the backend of our Book Journal app. With a Database as a service like HarperDB, you can achieve a lot by writing as little code as possible. Got an idea? Focus on working on the solution, leave the rest to the experts.

Coding vs Coffee

Bhavani Ravi — Sun, 18 Oct 2020 14:00:00 +0000

I have forever been a lover of coffee. When I say forever, it’s right from when I was 2, to be precise. Yes, I do remember my first smell of coffee. Unfortunately, neither I nor my mom are the best coffee makers in the world. Not that it’s terrible or anything, but it’s not the best I have drunk.

Now let’s come to the act of drinking coffee itself. I am not an addict, I drink 2 cups of coffee a day, and that’s about it. Occasionally, rarely 4. Coffee lovers always have this pride that no one can love coffee as they do. I had that, too, for a brief period. Maybe I didn’t transform into an adult completely.

For you to fight over your love for coffee, you have to be in the presence of good coffee. If not, everyone would just call out “coffee is bad” and mind their own business. In the presence of good coffee, love comes to life. Your eyes close as you sip through that hot gold savoring it every bit. Let me stop right there before I convert this into another poetry about coffee.

Fortunately or unfortunately, good coffee was beyond my reach until I was 23. As I said, the coffee lovers started rising, and we all started sharing how much we love coffee. It was a celebration every time we have coffee together. There is black, dark, light, cold, whatnot—the varieties, the colors, the experiences.

As a coffee lover, It would be unfair to compare my love for coffee with my fellow mates. To imagine that their love is better to mine. Just because an expresso shot doesn’t excite me doesn’t mean I am a bad coffee lover. My inability to take a sip of black coffee can’t question my love for coffee, Right?

Similarly, if you love technology, it doesn’t matter how much other people love it, what tool they are using, how fast they can gulp a concept down. You all have your own taste and your own pace.

No technology is better than the other. It’s all essential, it’s all necessary.

As a technologist, let’s come together and share our love for technology just like we did for coffee: the varieties, the colors, the experiences, the good and the bad.

Let’s share what we learned. Let’s spread knowledge.

Machine Learning Resources with Projects

Bhavani Ravi — Sun, 18 Oct 2020 00:00:00 +0000

With the boom of machine learning and Artificial intelligence, One of the questions I get frequently asked is

How to get started with machine learning?
What books do you suggest?
Do you suggest any courses? etc.,

But the most common question is there are so many resources which one to follow. Worry not! I have curated resources that worked for so many people. The resources come with some projects for you to try out the concepts you learn.

Now you may get carried away by all the links and books here. Machine learning has a learning curve and usually takes six months of constant effort. Do not give up because you do not understand anything in the first few weeks. One can become a master only by continuous practice.

The resources focused on applied ML and not the core of machine learning. This is a starting point. Once you can build a few models, you can dig deeper to understand how they work.

Resources

Machine Learning Mastery

Machine Learning mastery has an extensive set of roadmaps and blog posts that can guide you through the journey of transforming yourself into an ML Engineer.

Hands-on Machine Learning with sklearn, Keras and tensorflow2

I cannot thank this book enough because it takes you through every step of building an ML pipeline. With each chapter, you get a little closer to becoming an ML Engineer. By the time you finish the book, you will know how to tackle an ML problem.

Machine Learning A-Z course

Similar to the above book, this course has helped me simplify my understanding of machine learning. Until then, I thought it was a huge technology that takes years to master. The course suggested few mental models on how to approach each ML problem, which came in handy later when solving intensive problems.

Projects

Pretty much every exercise on Kaggle is a project worth building. While my focus is is mostly on text and NLP, you should focus on different kinds of problems like numerical, categorical, predictive, clustering, etc., The above resources should prepare you for the following projects.

Examples

Twitter hate speech classifier
Hot-dog not hot-dog
Book/Movie recommendation system
More projects

Forem: Bhavani Ravi

Python super() vs Base.__init__ Method

Method 1 :: Using Parent Reference Directly

Method 2:: Using Super with child class

Method 3:: Using the super method

Questions

Method Resolution Order

The Last Punch

Caching in Python

Conventional Caches

Caching Operations

Read Cache

Write Cache

Write Through

Write Back

Write Around

- Reduces the latency of the write-through method

Cache Eviction Policies

LRU - Least Recently Used

LFU - Least Frequently Used

MRU - Most Recently Used

FIFO - First In First Out

LRU Implementation

Where to Place the Caches?

When to Implement Caches?

Implementation In Python

Python inbuilt LRU-Caching

Caching In Python Flask

Resources

7 Programming Anti-Patterns

Things to unlearn as you move from CodeNewbie to Developer

1. Copy-Paste(Voodo) programming

How to Copy for Practice?

How to Copy-Paste at work?

2. Cargo Cult programming

2.1 Cargo Cult Software Engineering

How to Avoid Cargo Cult Programming?

3. A Big Ball of Mud

To Avoid Ball of Mud

4. Code/Design smell

5. Ego Programming

6. Golden Hammer

7. Boat Anchor

8. Dead Code

Fixing Dead Code

Pytest — How to test that a function is called with specific parameters?

How to Create a Download Button Using React

A guide on creating a download button using React.

Create a Button

Add Event Handler

Add Download Logic

3 Steps Improve Code Readability

1. Design Before Code

2. Is it Maintainable?

3. Is it intuitive?

Practice for Perfection

5 Packages that Python Devs Use Everyday

Pandas

Flask/Django

Flask

Django

Requests

Pytest

SQLAlchemy

How to get past the beginner stage in Python?

Program Construction

Build a Python App with HarperDB

HarperDB — SQL vs NoSQL

Python App with HarperDB

Setup HarperDB Instance

Create your HarperDB account

Create a new DB instance

The Application

Create schema and tables

Creating Python Application

Accessing DB using Harper Python SDK

HarperDB Python SDK — A sneak peek

Exposing and consuming APIs

Create Book

Update Book

Python super() vs Base.init Method