Forem: milandhar

What are Heroku Dynos?

milandhar — Thu, 28 Nov 2019 00:08:43 +0000

In recent weeks' posts, I have been making changes to my personal project EffectiveDonate, including making my React components responsive, optimizing the site for mobile, and improving the UX. All of these changes have been getting me closer to considering EffectiveDonate "production ready".

However one barrier that has stood in the way of production had to do with my settings on Heroku. Heroku is the cloud application platform that I have been using to deploy my web development projects, including EffectiveDonate. The particular problem that I faced had to do with a concept called "dynos" in Heroku, which are the building blocks powering any Heroku app. In this post, I will explain what dynos are, the problem that affected my website, and how I fixed it.

Heroku Dynos

To understand dynos, we must first learn about the concept of containerization in app development. Containerization is what abstracts away the need for app developers to manage hardware or virtual machines. Instead, we just deploy the app to a cloud platform like Heroku, which package the app into containers, which are environments that provide compute, memory, an OS, and a file system.

Heroku uses the container model to handle its apps, and the containers are called "dynos". Dynos are isolated Linux containers that are designed to execute code based on user commands. Heroku apps can dynamically scale to any number of dynos based on resource demands.

Dynos allow app developers to focus on building new projects without needing to worry about managing infrastructure. Cool!

Dyno Types

Heroku offers many different types of dynos, each with their own features, functionality and pricing:

Free dynos: ideal for experimenting. These dynos sleep after 30 minutes of inactivity to preserve remaining free dyno hours.
Hobby dynos: good for small scale personal projects. These dynos don't sleep.
Standard dynos: provide enhanced visibility, performance and scalability for professional apps.
Performance dynos: provide superior performance when it's most critical.

Dynos in My Project

When I deployed EffectiveDonate to Heroku, I didn't customize any of the performance settings, so it defaulted to using free dynos. I began to notice that whenever I accessed the site for the first time of the day, it would take a long time to load and then I'd get an error message like this:

But when I refreshed the page, the login screen would appear correctly. Although everything ran smoothly after I refreshed the page, having the user land on an error page was obviously not a good UX and gave a poor first impression of the site.

After doing some digging, I learned that this was actually expected behavior when using free dynos, since they sleep after 30 minutes of inactivity. Now that EffectiveDonate was getting close to production status, I decided it was time to upgrade to hobby dynos.

Luckily, hobby dynos cost only $7/month, and are prorated to the time that the dyno is actually in use. So running a dyno for 15/30 days of a month would cost only $3.50 for that month.

In my Heroku Dashboard, I selected my project, navigated to the "Resources" section, and selected "Change Dyno Type". I selected "Hobby", and the change immediately reflected in my project's dashboard.

Now anytime I enter the EffectiveDonate site, the hobby dynos are awake and I am correctly routed to the login page, without a delay or that nasty error page.

Another cool bonus about upgrading to hobby dynos is that a "Metrics" section gets added to your app's dashboard, giving you stats on events, memory usage, response time, throughput, and dyno load.

Conclusion

Dynos had been a concept that kept popping up during my builds and app management, but I had never really dug into them. Researching them has been a good exercise in learning more about cloud concepts such as containerization. And now EffectiveDonate no longer has the poor UX of landing on an error message, so users now correctly reach the login page.

Thanks for reading!

Drag and Drop Tables - Rails Backend (Part II)

milandhar — Sat, 23 Nov 2019 01:16:00 +0000

In my last post I described how I used the react-beautiful-dnd package to make a drag-and-drop Semantic UI Table. By the end of the process, I was able to implement the DnD action to re-order my table's rows. The only problem was that the newly updated order didn't persist on the backend, so if the user refreshed the page, the new order would be lost. Not a good UX!

So the challenge for this week was to find a way to maintain the new order on the backend so that every time the user refreshes the page, it looks exactly like they'd expect.

Introducing the Problem

I recommend looking over last week's post to understand the use case of the React DnD table in my website, EffectiveDonate. I created a button under the "Starred Projects" table to save the user's new order of starred projects once they are done dragging and dropping. Upon clicking this button, I had a skeleton method called saveOrder() which would POST the new order to my API endpoint. The task for this week was to actually implement that route on the backend and ensure that every time the "Starred Projects" table populates, it fills in with the correctly ordered projects.

Table with the "Save New Order" button (to be implemented)

Database Changes

Before I could begin implementing the new route on my Rails backend, I had to think about the changes I needed to make to my database's user_starred_projects table. This table is a join table that only had two attributes: user_id and project_id. Previously when a user wanted to add a new project to their stars, it would be created with these two attributes and the next sequential id primary key. When the user got to their profile page, their projects would be displayed in reverse-chronological order (ascending id).

In order to allow the starred projects table to be dynamically ordered, I needed to create a new integer attribute in the user_starred_projects table I called order_number. I created a migration: rails generate migration add_order_number_to_user_starred_projects order_number:integer and then a rails db:migrate to migrate the database to the new schema.

Controller Changes

Now that the database was updated with the new order_number column, it was time to build out this attribute in the UserStarredProjects controller. I wanted to ensure that each new starred project was given the next sequential order number, so that the first star had an order_number of 1, and each additional starred project would increment that number. In the create controller action, I added the following code:

number_stars = UserStarredProject.where(user_id: @userStar.user_id).length
@userStar.order_number = number_stars + 1

I had an existing method in the UserStarredProjects controller to remove projects from the user's starred list, but since I had a new order_number attribute, I needed to consider the impact of removing a starred project on the rest of the list's orders. For example, if there were five projects in a user's starred list and the third one was removed, I would shift the following orders: (4 => 3, and 5 => 4). To scale this, I wrote the following code:

def remove_project
    user_id = params[:user_id]
    project_id = params[:project_id]

    @user_project = UserStarredProject.find_by(user_id: user_id, project_id: 
    project_id)
    higher_projects = UserStarredProject.where("order_number > ? AND user_id = 
    ?", @user_project.order_number, user_id)

    if @user_project.delete
      #Find all the projects with an order_number > @user_project.order_number 
      #and decrement them
      higher_projects.map do |project|
        project.order_number -= 1
        project.save
      end
      render json: {message: 'Removed Project' }, status: :accepted
    else
      render json: {error: 'Could Not Remove Project' }, status: :not_acceptable
    end
  end

Now that the removal of a starred project has been handled, the final controller action to implement was the endpoint that is called when the user re-orders their project list and saves the new order on the frontend. In the next section I will go into detail about how I POSTed the order on the frontend, but the order parameter gets sent as an array of the new project IDs (ex: [475, 170, 87]). So in the update_star_orders controller action, I iterate through this array, look up the UserStarredProject object that corresponds to its project_id and then assign it a new incrementing order_number:

def update_star_orders
    user_id = params[:user_id]
    order = 1
    project_array = params[:project_array]
    project_array.each do |id|
      user_star = UserStarredProject.find_by(user_id: user_id, project_id: id)
      user_star.order_number = order
      order += 1
      user_star.save
    end
    render json: {message: 'Updated Order' }, status: :accepted
end

Now the database will reflect the correct order_number for each starred project after the user rearranges their list!

Frontend Changes

Now that I had the update_star_orders endpoint implemented, it was time to write out the fetch action for the user to persist their new order. I first extract the project.ids from each project as an array, and send that array as a parameter to the update_star_orders endpoint I described in the previous section. Here's how I did it:

  saveOrder = () => {
    let { starredProjects } = this.state;
    let projectIds = []

    //go thru list of project objects and unshift their ids to the empty array 

    starredProjects.forEach(project => {
      projectIds.unshift(project.id)
    })

    // Take new state of starred project list and POST to endpoint

    const userId = localStorage.userid
    const url = `${config.get('API_URL')}/api/v1/update_star_orders`
    const headers = {
        method: 'POST',
        headers: {
            'Content-Type': 'application/json'
        },
        body: JSON.stringify({user_id: userId, project_array: projectIds})
    }
    fetch(url, headers)
      .then(res=>res.json())
      .then(json => {
        if(!json.error){
          alert("New order saved!")
        }
      })
  }

The drag-and-drop reorder posted to backend

Conclusion

Overall, this exercise was a good way to get creative while thinking of a way to persist the list's order on the backend. This process reminded me of solving an algorithm, since there were limitless ways of tackling this problem and many different use cases that it had to meet. It would be an interesting exercise to see what other data structures I could use to optimize time/space efficiency here, and I'd love to explore refactoring this in different ways in the future. But for now, this solution worked and neatly maintains the order of each starred project in the database.

Thanks for reading!

Drag and Drop Tables with React-Beautiful-DND (Part I)

milandhar — Sat, 16 Nov 2019 02:40:51 +0000

This week, I wanted to experiment learning a new React component and implementing it into my EffectiveDonate website. I began to think of what aspects of the site could use a cool new feature to improve its UX, and focused in on the Profile page. Previously, the Profile page allowed users to update their default themes (Health, Education, etc), and also to view the nonprofit projects that they had starred. The list of projects was organized in a Semantic UI Table, and enabled users to view key information about the projects, donate to the project, or delete the project from their stars. However, the table was sorted in chronological order, so that the user's most recent starred projects were all the way at the bottom of the table - not the best UX!

While I could have easily just sorted the table in reverse chronological order as a quick fix, I wanted to give the user some more control. So I started to brainstorm some solutions in React to make the table more dynamic. I found this List of Awesome React Components, and read through a list of several drag and drop components. Drag and drop would be a nice, clean way to let the user customize their starred projects! I eventually chose React Beautiful DnD - it had over 17k stars on GitHub, a nice instruction video and many examples.

The original profile page, with starred projects table in chronological order

What is React-Beautiful-DnD?

React-Beautiful-DnD is a React package with a goal of creating drag and drop functionality for lists that anyone can use, even people who can't see. The main design goal is physicality - they want users to feel like they are moving objects around by hand. It also has accessibility features, including drag and drop using just the keyboard.

It also plays nicely with tables, specifically the Semantic UI React Table component, which sealed the deal for me to use it.

Implementing React-Beautiful-DnD on my Website

In order to make my StarredProjectsList component DnD-able, I followed a video course on react-beautiful-dnd, and referenced this example of a Semantic UI table component. Also I made sure to install the package with: npm install react-beautiful-dnd --save.

While I recommend going through the two resources I listed above to thoroughly understand the process for implementing the component in your project, I'll give a few highlights of key components in the API here:

DragDropContext

This component is required to specify which part of your React tree you want to be able to use drag and drop. For me, I wrapped my entire Semantic UI Table component with <DragDropContext />. A required prop for this component is onDragEnd, a function that dictates how the list or table's state should change once the drag operation is complete. The opening tag for my DragDropContext is the following: <DragDropContext onDragEnd={this.onDragEnd}>.

The onDragEnd method finds the index of the starred project I dragged, and splices it into the array of my starredProjects state. Below is my implementation of the method:


 javascript
  onDragEnd = result => {
    const { destination, source, reason } = result;

    // Not a thing to do...
    if (!destination || reason === 'CANCEL') {
      this.setState({
        draggingRowId: null,
      });
      return;
    }

    if (
      destination.droppableId === source.droppableId &&
      destination.index === source.index
    ) {
      return;
    }

    const starredProjects = Object.assign([], this.state.starredProjects);
    const project = this.state.starredProjects[source.index];
    starredProjects.splice(source.index, 1);
    starredProjects.splice(destination.index, 0, project);
    this.setState({
      starredProjects
    });
  }

Droppable

A <Droppable/> is a container for </Draggable/> items. It can be dropped on by </Draggable />s.

The only required prop for <Droppable />s is a string, droppableId. I wrapped my <Table.Body/> in the <Droppable /> component, since that is the container of data on which I will be dragging rows.

Draggable

A <Draggable /> is the React component that will actually be dragged around onto <Droppable />s. It must always be contained by a <Droppable />, but it can also be moved onto other <Droppable />s.

The required props for <Draggable />s are: draggableId and index. Some important notes on these props:

1) the draggableId must be a string. I initially made mine an integer and was stumped when my table rows couldn't be dragged. But once I added the .toString() function to the prop, it was all good.
2) the index prop must be a consecutive integer [1,2,3,etc]. It also must be unique in each <Droppable />.

Below is a snippet of my code where I wrap each <Table.Row> in a <Droppable/> after maping each of the starred projects in state:


 javascript
{this.state.starredProjects.map((project, idx) => {
 return (
     <Draggable
        draggableId={project.id.toString()}
        index={idx}
        key={project.id}
     >
       {(provided, snapshot) => (
       <Ref innerRef={provided.innerRef}>
         <Table.Row
     ...

Children Function

Another quirk about the <Droppable /> and <Draggable /> components is that their React child must be a function that requires a ReactNode. If this child function is not created, the component will error out. The function contains two arguments: provided and snapshot. I recommend reading the documentation for both <Draggable /> and <Droppable /> to fully understand what these two arguments do and what props they take.

Also, the <Draggable /> and <Droppable /> components require an HTMLElement to be provided to them. This element can be created using the ref callback in React or the 'Ref' Semantic UI Component. This react-beautiful-dnd guide does a good job of explaining the purpose of the ref callback and how to avoid any errors.

For an example of how I used the provided and snapshot arguments of the child function, as well as the Ref Semantic UI Component in my table, here is a snippet of the <Droppable /> tag:


 javascript
<Droppable droppableId="table">
    {(provided, snapshot) => (
       <Ref innerRef={provided.innerRef}>
          <Table.Body {...provided.droppableProps}>
          ...

The working DnD table

Conclusion

Overall, it was a fun and informative process to implement my Semantic UI Table with react-beautiful-dnd. I enjoyed learning the component's API and it was interesting to work with concepts that were new to me, like the children functions and ref callbacks.

I definitely recommend viewing the video course on react-beautiful-dnd, and also checking out the example code online. You can also reference my table component file on GitHub to fully see how I implemented the DnD components.

While I am satisfied with the UX that is available on the table component now, the next step is to make it persist on the backend so that when the user refreshes the page, the table re-renders in the new order. This should require a bit of creative manipulation on the backend, which I am excited to tackle next week :)

Thank you for reading and let me know if you have any questions or comments!

Using CSS Media Queries

milandhar — Thu, 07 Nov 2019 20:39:48 +0000

In this post I will be diving into the important topic of Media Queries in CSS, providing examples of elements from my own project that needed mobile optimization and how I approached them. In my previous post I covered how to make React components responsive, which allowed me to create a unique user experience for mobile devices. This week's post zooms in to the individual HTML elements within each component that need mobile optimization. I view this process as analogous to responsiveness in React, except it is more manual whereas Semantic UI React has special props and components that do much of the work for us.

What is a Media Query?

In CSS, a media query uses the @media rule to include certain CSS properties only if a condition is true. Media queries can check several things, like the width or height of a device, the screen's orientation (landscape or portrait), the screen's resolution, and more.

The syntax for a media query is the following:

@media not|only mediatype and (mediafeature and|or|not mediafeature) {
  CSS-Code;
}

You can find all of the different media types and media features at this W3Schools page. I most frequently use the "screen" media type and "max-width" media feature for my responsive web development media queries.

An example of a type of media query I often use is below:

@media screen and (max-width: 500px) {
  .container {
    display: block;
  }
}

This would change the display property to block of the element with class of "container" for screen sizes less than or equal to 500px.

Adding Media Queries to My Project

A note on the media feature I used - for most of the media queries, I used a max-width: 767px feature. I found that setting the breakpoint at 767px max-width allowed me to capture most mobile device sizes (portrait and landscape) for which I needed to adjust CSS properties. I used Chrome DevTools to simulate different mobile devices on my website. I highly recommend using the device mode with DevTools for mobile responsiveness media queries.

In my css file, I grouped most of the media queries I wrote for mobile responsiveness in the following block:

@media only screen and (max-width: 767px) {
  ...
}

However, in the examples below I have repeated the query each time for clarity.

Now let's dive into some real examples of how I used media queries to make elements in EffectiveDonate responsive.

Stacking Elements Using Flexbox Layout

In the "Projects" page of EffectiveDonate, there are dropdowns for countries and themes, allowing the user to filter the types of projects they want to learn about. On desktop screens, it works to have these dropdowns positioned next to each other. However, once the width of the screen shrinks to mobile sizes, the dropdowns become too small to contain all of the content, and they look awkward:

To solve this on mobile, I wrote a media query that changes the display to "flex" and also associated properties justify-content and align-items to "center". This style of container is called the "Flexbox Layout". You can learn all about Flexbox in this excellent CSS-Tricks Guide, which walks you through all of the awesomeness of Flexbox. I enjoy using Flexbox when I can, since it is inherently more responsive than other layouts, and just feels more spatially intuitive to me.

I also increased the width of both of the dropdowns so that the longer countries and themes could be easily readable:

@media only screen and (max-width: 415px) {
  .ui.grid .project-dropdown-row {
    flex-direction: column;
    justify-content: center;
    align-items: center;
  }

  .project-dropdown-row .country-drop {
    min-width: 70% !important;
  }

  .project-dropdown-row .themes-drop {
      min-width: 70% !important;
  }
}

Now the dropdowns are cleanly stacked and easy to read!

Make Table Text Wrap

Another component that needed some responsive media queries was my Profile section. I have a responsive Semantic UI Table for the user's starred projects that collapses on smaller screens. However, the table's text data would not wrap to the next line, so it cut off key project data on mobile:

To fix this, I changed the white-space and word-wrap properties in my media query. white-space was previously set to "nowrap", which forced all text in the table to render on one line. Setting it to "normal" allows text to be broken into multiple lines. Setting word-wrap: break-word is self explanatory: it allows words to be broken up.

I also wanted the red "X" and green "$" buttons to be easier to click on mobile, so I flipped their display to flex and used justify-content: space-evenly to spread the buttons out along the container.

Here are both of the blocks I wrote to solve these issues:

@media only screen and (max-width: 415px) {
 .ui.table[class*="single line"]  {
    white-space: normal !important;
    word-wrap: break-word;
  }

  #remove-donate-div {
    display: flex !important;
    justify-content: space-evenly;
  }
}

Now the entire table content can actually be viewed and the buttons are spread out enough to be clickable on mobile!

Centering Content

The final example of media queries I will discuss here is a common one that is frustrating if you don't know a quick trick. The way to center text within a container is fairly straightforward (text-align:center). But actually horizontally centering an entire container of content is less intuitive. The best way I've learned is to make the margin-left and margin-right equal to "auto". This effectively equalizes the space to the left and right of the container, ensuring that it is always centered. This is also very responsive design friendly, since the content will be centered as the viewport shrinks.

In my Create User Form, I have a checklist of themes the user can choose as their top three favorites. However, each list item was awkwardly positioned left of center:

To horizontally center each theme div, I used the margin trick:

@media only screen and (max-width: 415px) {
  #fields-div .field {
    margin-left: auto;
    margin-right: auto;
    width: 100% !important;
   }
}

I also set the width to 100% to ensure the theme text would not wrap. Now the theme checklist is centered and good to go!

Conclusion

While the examples I included in this post represent many common use cases for media queries, there are limitless possibilities. I recommend skimming the list of Media Features available just to see the different ways media queries can be used.

However, now that I've implemented these responsive queries, I feel ready to share EffectiveDonate. Feel free to check it out and let me know if you have any feedback or see any bugs. Thanks for reading!

Responsive React Components (w/ Semantic UI)

milandhar — Fri, 01 Nov 2019 18:15:19 +0000

In this post I'll cover some strategies to make your React application more responsive. I had a specific component in one of my projects that needed to be bypassed in order for my website to be optimal on mobile, and I will go through the steps I took to tackle that issue. Also, my Navbar component (referred to as "Menu" in Semantic UI) needed some work to look good on mobile, so I'll cover the changes I made to that component as well!

Making My Website Responsive

The main page of my nonprofit donation website, EffectiveDonate, is a world map that uses data visualization with the React D3 Library. While I was able to make the world map responsive on larger screen sizes using some D3 methods and an event listener for window resize, the map just would not look good on mobile screens. It would either be so zoomed in that the user could only see a few countries at a time, or way too zoomed out and small to click on a particular country.

Although I liked visually presenting project data through the world map component, I realized that I needed to explore new options to make my website user friendly on mobile. So I decided I would create a new simple landing page for mobile that would replace the map, which would just appear on desktop and tablet screens. The question then was - how do I do that??

Dynamic and Responsive Routing

I initially came across a feature in React called Dynamic Routing. The philosophy is that frameworks like Rails use static routing, where routes are declared before any rendering takes place. React wanted to change this style, so since v4 routing takes place dynamically as the app is rendering.

React also has a way to make routes responsive to different screen sizes. Similar to CSS media queries, React has a Media Component that listens for CSS media query responses and renders components if the query matches. This React Training gives an excellent example of how to dynamically redirect a user to a different component based on a responsive media query.

My Approach

Although the Responsive Routing I described above is a clean way to compose routes given different screen sizes, I realized the problem I needed to solve was a bit simpler. The training gave some sound advice: "Think about how to solve the problem with React’s declarative composability because nearly every 'React Router question' is probably a 'React question'". This made me think deeply about which React component I really needed to change to have my desired result on mobile.

It turns out that my LoginForm component is where the user gets redirected to the MapBrowser landing page, so that's where I zoomed in and made some changes. Previously, the user would be redirected to the MapBrowser page if they were successfully granted a jwt token upon login: this.props.history.push("/map") This is where I needed to build in some logic to redirect to the mobile landing page on small screens.

The existing map landing page

I found a helpful stackoverflow post that explained how to detect if the screen has changed to mobile in React. I created a new state in my LoginForm component called "mobile", and wrote a function that would set the state to true if the screen width is less than 760px:

resize() {
    let currentMobile = (window.innerWidth <= 760);
    if (currentMobile !== this.state.mobile) {
      this.setState({mobile: currentMobile});
    }
  }

Then, in the componentDidMount() function, I added an event listener for a window resize and then called resize():

window.addEventListener("resize", this.resize.bind(this));
this.resize();

This will ensure that whenever the component renders, the screen size will be accurately saved in state.

Next, in my login() function's fetch method, I added the following conditional logic:

if (this.state.mobile) {
    this.props.history.push("/mobile_landing")
} else {
    this.props.history.push("/map")
}

So the user will be redirected to the new mobile_landing route if they are on a mobile screen at login. Great!

The new mobile landing page

Updating My Navbar Component

Now that the redirect from login was directed to the correct route on mobile, I had to make some changes to my Navbar component. I had attempted to use the Collapsable Menu React component with Semantic UI, since it is supposed to dynamically collapse on mobile screens, but I wasn't able to get it to look exactly like I wanted it to. Plus, my Navbar included a "Map" item, which I needed to change to "Select" on mobile, so I knew I had to find a way to customize the component and make it responsive.

I found out that the Grid Component includes an only prop that makes a Grid Row visible by adding breakpoints for certain screen sizes. I also wanted to make my mobile Navbar vertical. Semantic UI makes this super easy by just passing a vertical prop to the Menu component.

So now I could write a separate Grid Row that would only be visible for mobile screens, and would redirect to the new mobile landing page instead of the map. Here's a preview of what this separate row looks like:

    <Grid.Row columns={1} only='mobile'>
       <Grid.Column>
          <Menu size='massive' vertical inverted>
             <Menu.Item header>EffectiveDonate</Menu.Item>
             <Menu.Menu position='right'>
               <Menu.Item
                 name='select'
                 active={this.state.activeItem === 'mobileLanding'}
                 onClick={this.redirect} />
                 ...

So now the Navbar is responsive with the items that correspond to the correct routes for mobile / desktop! You can check out the two screenshots of the website above to see the different Navbars that render on different screen sizes.

Conclusion

It took some digging to find the best solutions to make my application responsive, but the approaches I took ended up being much more straightforward than expected. Both React and Semantic UI luckily make responsive design relatively easy with built-in components.

Although it would have been nice to have a visual map available on mobile as well, I've learned that the simplest solution from a UI perspective is often the best. It would get really messy to try to zoom/scroll in the map without clicking on a country.

This step felt like a big hurdle to cross in order to make EffectiveDonate fully responsive and ready to distribute more broadly. I still have lots of work to do to make each component mobile friendly, but the trickiest part is now over.

Thanks for reading!

Functional Testing in Rails

milandhar — Thu, 24 Oct 2019 03:41:18 +0000

In last week's post, I described how I learned the basics of unit testing and then wrote unit tests for my EffectiveDonate project. As I mentioned in that post, unit tests in Rails deal with testing a model's validations and methods. This week, I will be diving into the concept of functional testing, which in Rails means testing controllers!

Functional Testing

In general, functional testing is a "type of software testing whereby the system is tested against the functional requirements/specifications." These requirements are tested by feeding them sample input based on the function's specifications, determining the output based on the behavior of the program, and then comparing the actual and expected results. This is a chance for developers to ensure that real-world user stories are fulfilled to satisfy the requirements of the application.

Rails' documentation recommends writing functional tests for the following:

was the web request successful?
was the user redirected to the right page?
was the user successfully authenticated?
was the appropriate message displayed to the user in the view?
was the correct information displayed in the response?

Functional Testing in Rails

To get started learning about the test directory that is built-in to Rails applications, refer to the "Test Directory" section in my previous blog . Similar to creation of unit tests for models, running a command like rails generate controller project will create a file called projects_controller_test.rb in the test/controllers/ directory.

In this newly created file, you will see a class declaration like class ProjectsControllerTest < ActionDispatch::IntegrationTest. Rails has encouraged the use of its "integration-style" controller tests because they perform actual requests, while the "functional-style" tests in Rails just simulate a request. But for the purposes of this post, I still refer to the tests I wrote as "Functional", following the Rails testing naming convention (although some of them do involve multiple components).

Functional Testing My Project

Now that my models for EffectiveDonate have been unit tested, it was time to functionally test my controllers! I used Rails' documentation to learn the syntax for functional testing in the Minitest suite, and would highly recommend reading it to anyone interested. While the assertions for functional testing are similar to those used in unit testing, there are several additional features of functional testing, such as request types (like get, post, put, etc), available instance variables (@controller, @request, @response), and three Hash objects (flash, cookies, and session).

While I wrote functional tests for more controllers than I've included in this post, some tend to follow the same patterns, and I tried to discuss the ones that were most distinct here. Let's dive into the controllers I tested:

Countries Controller

For my first functional test, I wanted to start with a simple request to the index endpoint of the Countries controller. In the conuntries_controller_test.rb file, I wrote the following:


 ruby
    test "should get index" do
      get "/api/v1/countries"
      assert_response :success
    end

This test sends a get request to the "/api/v1/countries" route. The :success parameter of assert_response specifies that the test will pass if the response comes back with a status code between 200-299. You can also specify :redirect (status code between 300-399), :missing (status code of 404), or :error (500-599).

A key function of the Countries controller is its ability to return the correct number of active projects in each country. Below is the test I wrote for this:


 ruby
  test "should get project count" do
      Country.delete_all
      Country.create(name: "Mexico", iso3166CountryCode: "MEX")
      project = Project.new
      project.image_url = "https://files.globalgiving.org/pfil/34796/pict_large.jpg?m=1534281939000"
      project.theme_str_id = "env"
      project.project_link = "https://www.globalgiving.org/projects/nourish-a-young-brain-protect-one-ancient-culture/"
      project.title = "Nourish a Young Brain Protect One Ancient Culture"
      project.save
      mexico = Country.find_by(iso3166CountryCode: "MEX")
      mexico.projects << project
      get "/api/v1/get_project_count"
      res = JSON.parse(@response.body)
      assert_equal 1, res[0][1]
    end

This test creates a single Country object and assigns it a new Project. Then it sends a request to the "get_project_count" endpoint. The assertion I wrote tests whether the JSON response includes the single project that belongs to Mexico.

Users Controller

I next wanted to test my endpoints in the Users controller, another vital set of functions in EffectiveDonate. The User actions I will be testing involve things like creating a new user, returning all of a user's saved projects, verifying whether a project has been starred by a user, and updating a user's information.

Many of the tests in this file will use the following data:


 ruby
  user_params = { user:{
    username: "milan123",
    password: "abc123",
    first_name: "Milan",
    last_name: "Dhar",
    default_country: "USA",
    theme1: 14,
    theme2: 15,
    theme3: 16
    }
  }

  project = Project.new
  project.image_url = "https://files.globalgiving.org/pfil/34796/pict_large.jpg?m=1534281939000"
  project.theme_str_id = "env"
  project.project_link = "https://www.globalgiving.org/projects/nourish-a-young-brain-protect-one-ancient-culture/"
  project.title = "Nourish a Young Brain Protect One Ancient Culture"
  project.save

Creating a New User


 ruby
  test "should create new user" do
      post "/api/v1/users", params: user_params.to_json, headers: { "Content-Type": "application/json" }
      assert_response(:success, message = "failed to create user")
    end

This test will attempt to create a new User object using the user_params listed above. If the response is not "success", this test will fail with a message of "failed to create user".

Updating a User


 ruby
test "should update user" do
      user = User.create(user_params[:user])
      put "/api/v1/users/#{user.id}", params: {id: user.id, first_name: "Dylan"}.to_json, headers: { "Content-Type": "application/json" }
      res = JSON.parse(@response.body)
      assert_equal 'Dylan', res["first_name"]
   end

This test sends a new first name as a parameter to the "/users/:user_id" endpoint. It then checks to make sure the first_name in the response JSON is equal to "Dylan", the new first name.

Return All a User's Saved Projects


 ruby
test "should get all user's saved projects" do
      user = User.create(user_params[:user])
      user.projects << project
      starred_project_params = {user_id: user.id, project_id: project.id}
      post "/api/v1/get_user_projects", params: starred_project_params.to_json, headers: { "Content-Type": "application/json" }
      assert_response(:success, message = "failed to get all projects")
    end

This test assigns the existing project as a favorite of a new user. It then sends a post request to the "get_user_projects" endpoint, sending the user_id and project_id as params. If the response is returned successfully, the test will pass.

Verify a Project is Starred


 ruby
test "should verify that a project is starred" do
       User.delete_all
       user = User.new(user_params[:user])
       user.projects << project
       user.save
       post "/api/v1/check_star", params: {user_id: user.id, project_id: project.id}.to_json, headers: { "Content-Type": "application/json" }
       status_hash = JSON.parse(@response.body)
       assert_equal 'Project is starred.', status_hash["status"]
     end

This test is similar to the previous one, but it pings the "check_star" endpoint and checks the "status" of the response to ensure that the project has been correctly recognized as "starred". The check_star method in the Users Controller includes the following render statement if the project is starred by the user: render json: {status: 'Project is starred.'}, so the test's response is able to parse out the correct status.

Auth Controller

The final controller in EffectiveDonate that I will discuss for testing purposes here is the Auth controller. This controller's only function is to authorize a user has signed in with the correct credentials, and if so, grant them a JWT so they can proceed to the website. Here is the test I wrote (using the same user_params as previous tests):


 ruby
test "should authorize user on login" do
    user = User.create(user_params[:user])
    post "/api/v1/login", params: user_params.to_json, headers: { "Content-Type": "application/json" }
    assert_response :success, "Incorrectly denied user access"
  end

This test sends the user_params to the "/login" endpoint, although it will just check for the username and password of the user. If the controller function works correctly, it will authorize the user with a jwt token and a successful status code, thus passing the test.

Conclusion

Overall, I found functional tests to be a natural next step from writing unit tests, as they were much more involved and as such, more likely to encounter errors. I made some silly mistakes, like forgetting my route paths started with "/api/v1/", and not converting my parameters to json, but these were good learning experiences. Also, writing these functional tests helped me to understand more deeply how my controllers worked, and to feel more confident that the user experience will be as intended. Thanks for reading!

Unit Testing in Rails

milandhar — Thu, 17 Oct 2019 19:37:33 +0000

For my blog this week, I decided to explore the topic of unit testing in Rails. Throughout my time at the Flatiron School, I completed labs that included RSpec tests to determine if the code was correct. Although I indirectly used hundreds of these RSpec tests, I never quite dove into the topic of writing tests myself. I was always curious how to write my own tests and knew it is an extremely skill to learn for Test-Driven Development (TDD), so I decided I needed to teach myself how to write them!

Unit Testing

I wanted to begin my testing journey with the concept of unit testing. Broadly speaking, unit testing is a testing method where individual units of source code, program modules, usage procedures, and operating procedures are tested to determine if they are fit for use. They make sure that a section of an application, or a “unit”, is behaving as intended. In a Rails context, unit tests are what you use to test your models. Although it is possible in Rails to run all tests simultaneously, each unit test case should be tested independently to isolate issues that may arise. The goal of unit testing is to isolate each part of the program and show that the individual parts are correct.

Advantages of Unit Testing

Unit testing has become a standard in many development methods because it encourages well-structured code and good development practices. One advantage of unit testing is that it finds problems early in the development cycle rather than later. It’s always better to find bugs earlier so that features that are added on later in the cycle are using the correct component parts. Additionally, unit testing forces developers to structure functions and objects in better ways, since poorly written code can be impossible to test. Finally, unit testing provides a living documentation of the system as it is developed. The description of the tests can give an outsider a basic understanding of the unit’s API and its features.

The Test Directory

In a normal situation, when creating a new Rails app, you would run the rails new application_name command after installing Rails. Once the skeleton of the new Rails application has been built, you will see a folder called test in the root of the new application, and many subfolders and files within the test directory. For the purposes of this post, I will be focusing on adding tests to the models directory, since unit testing in Rails primarily involves testing models. However, if you are interested in learning about the other directories within the "test" folder (such as helpers, channels, controllers, etc), you can do so here.

Rails automatically adds new test files to the models directory when new models are generated. So if you were to run a rails generate model User, Rails would add a user_test.rb file in the test/models directory. The unit tests will be written in these files for each model.

Unit Testing My Project

I decided a good place to start practicing unit testing would be my project called EffectiveDonate, a nonprofit donation website. I originally developed this project without any tests, and it has a fairly complicated Rails backend with many models, so there was a plethora of features I could test.

It was slightly more work to add in the tests at this point, since I initially ran the rails new command without adding tests. I ended up just creating a new blank project with the test directory, and then dragging it into the root of my EffectiveDonate backend. Then I added files for the models that I wanted to test into the models directory, such as project_test.rb.

To learn how to unit test my models, I followed the Rails Guides for Unit Testing Models. I won't repeat all the information I read there, but I would highly recommend reading it since it is an excellent resource for how to write Rails testing code, testing terminology, how to prepare your app for testing, running tests, and test assertions. An important piece of advice in the documentation is to include a test for everything which could possibly break. And that it is a best practice to have at least one test for each of your validations and at least one test for every method in your model. So with that framework, I began writing tests!

Project Model

The most complex model in my backend for EffectiveDonate is the Project model. This model has several has_many / belongs_to associations with other models, and the primary call to the external API (GlobalGiving) that feeds all the data in my project happens in this model.

But as I began writing tests for the Project model, I realized I didn't have any validations! So theoretically it would be possible for a Project instance to be created without having key attributes like a title or images. While GlobalGiving's API generally has very clean data, I want to make sure all the Project data in my app has the attributes I need. So I went ahead and wrote the following validations:


 ruby
  validates :title, presence: true
  validates :image_url, presence: true
  validates :project_link, presence: true
  validates :theme_str_id, presence: true

These will ensure that Projects without these attributes will not be saved.

Now it was time to test these validations. In my test/models/project_test.rb file, I had the following class set up:


 ruby
require 'test_helper'

class ProjectTest < ActiveSupport::TestCase
  test "the truth" do
    assert true
  end
end

This is the skeleton that Rails adds for each model that is generated. The "the truth" test is an example test that will always pass and allows you to verify that your Test class is up and running. In Rails testing, assertions are the most important part of the test. They actually perform the checks to make sure things are going as planned. The example test above will pass because it just ensures that the argument in the assertion (true) is true. There are many other types of assertions that you get for free with Rails unit tests, each with their own purpose.

In order to test my new Project validations, I wrote similar tests for each validation. Each test created a new Project, then assigned the projects all of the other required attributes, except for the one that I am testing for. Then I wrote an assert_not to ensure that the Project would not save without the missing attribute.

For example, here is my validation for my test to ensure a Project will not save without a title:


 ruby
test "should not save a project without a title" do
    project = Project.new
    project.image_url = "https://files.globalgiving.org/pfil/34796/pict_large.jpg?m=1534281939000"
    project.theme_str_id = "env"
    project.project_link = "https://www.globalgiving.org/projects/nourish-a-young-brain-protect-one-ancient-culture/"
    assert_not project.save, "Saved the project without a title"
  end

If I ran the test and the new project instance did get saved, I would know there was a problem with my validation, and would receive the message in the argument of the assertion: "Saved the project without a title".

To test the other methods in my Project model, I wrote different tests. These methods perform the fetches from the GlobalGiving API, and return JSONs with the project data. I used the assert_nothing_raised assertion to ensure my queryActiveProjects method runs without raising an error (this will ensure the API endpoint and my credentials are correct):


 ruby
  test "should query the active projects API endpoint without error" do
    assert_nothing_raised do
      Project.queryActiveProjects(nextProject:nil)
    end
  end

I also wanted to ensure that the JSON returned by this method has attributes that the application needs, such as nextProjectId, and that the JSON includes a project object with a valid id:


 ruby
    test "should return a json of projects that has a next project ID" do
      assert Project.queryActiveProjects(nextProject:nil)["projects"]["nextProjectId"]
    end

    test "should return a json of projects whose first project has an ID" do
      assert Project.queryActiveProjects(nextProject:nil)["projects"]["project"].first["id"]
    end

These tests use assert to validate that both of these attributes exist.

Country Model

My Country model includes similar API fetch methods to Project above, so I wrote a similar unit test using assert_nothing_raised. I also wrote a test to ensure that my create_all method actually creates Country instances:


 ruby
test "should create countries" do
    Country.delete_all
    Country.create_all
    assert_not_equal Country.all.count, 0, "Didn't create any countries"
  end

Here I used assert_not_equal to ensure that the number of Countrys created was greater than 0.

User Model

My User model already included a validation to ensure that a User cannot create a profile with the same username as someone else: validates :username, uniqueness: { case_sensitive: false }

To test this, I created one User and then used an assert_not to ensure that a second User with the same username could not be saved:



  test "should not allow duplicate user names" do
    user1 = User.create(username: "barryobama", password: "bo")
    user2 = User.new(username: "barryobama", password: "michelle")
    user1.save
    assert_not user2.save, "Saved a duplicate user name"
  end

If user2 was able to be saved, the test would fail and I would see the message "Saved a duplicate user name".

User Starred Project Model

The final model in EffectiveDonate that I needed to test was User_Starred_Project. A user should not be able to star the same project multiple times, so I had already added the following validation to the model: validates :user_id, uniqueness: {scope: :project_id}.

In order to test this validation, I created a new User and a valid Project with all the required attributes. I then successfully created one instance of UserStarredProject using the id of both the User and Project that I created. I then attempted to save a duplicate UserStarredProject, using assert_not to ensure that it would not save:



  test "should prevent a user from starring multiple projects" do
    user1 = User.create(username: "barryobama10", password: "bo")
    project1 = Project.new
    project1.title = "Nourish a young brain, protect one ancient culture"
    project1.theme_str_id = "env"
    project1.image_url = "www.google.com/img"
    project1.project_link = "https://www.globalgiving.org/projects/nourish-a-young-brain-protect-one-ancient-culture/"
    project1.save
    user_star_1 = UserStarredProject.create(project_id: project1.id, user_id: user1.id)
    user_star_2 = UserStarredProject.new(project_id: project1.id, user_id: user1.id)
    assert_not user_star_2.save, "User starred a duplicate project"
  end

If there was a problem in my validation, I would see the message "User starred a duplicate project", and the test would fail.

Conclusion

Ultimately, this process of learning about unit testing and testing my own project's models was very helpful. For some reason, testing seemed like an intimidating skill to learn, but I realized it is actually not too difficult once you get started. I also am starting to understand its importance to maintaining well-organized and clean code. I probably wouldn't have added all of those validations to my Project model unless I started to test it, so my app is already more resilient to bad data than it was before. Also, the tests I wrote to check the API calls will be really helpful to ensure that my API credentials are still valid and that the endpoints I am reaching are correct.

I am looking forward to exploring more facets of testing in the near future, so stay tuned for more testing-related posts. Until next time!

Using Heroku Scheduler

milandhar — Mon, 07 Oct 2019 21:34:35 +0000

It's been a few months since my last post, and am excited to begin posting again!

Following my completion of EffectiveDonate, a nonprofit discovery website I developed for my capstone project, it was time to publicly deploy it. I've had experience deploying other projects I built at the Flatiron School to Heroku (a cloud platform service), so I decided to go with Heroku again for EffectiveDonate. However there were a few additional features of Heroku I needed to use for the first time since I was working with API data that I needed to refresh regularly, requiring me to schedule a job to run daily. Let me tell you how I did it!

What Are Heroku Add-ons?

According to Heroku, add-ons are components that support your application, such as data storage, monitoring, analytics, data processing, and more. Add-ons are managed either by third-parties or by Heroku itself. The purpose of add-ons is to allow developers to focus on the maintenance of their own code and to outsource the services supporting production to the add-ons.

You can install add-ons either through the CLI or the Heroku Dashboard.

Heroku Scheduler

My EffectiveDonate site uses project data from the GlobalGiving API, and I have a method on my Rails backend to fetch new projects. However, these projects are dynamic - they are active only until their funding goal hasn't been reached, and then they stop taking donations. New projects are also started daily. Additionally, project funding amounts are constantly changing. As a result, I need to fetch project data daily regularly from the API to prevent stale data from reaching my application.

I researched several ways of automating query jobs, both with npm packages and Ruby gems, but I finally came across Heroku Scheduler, which turned out to be much more straightforward than those other methods.

Heroku Scheduler is a free add-on that allows you to automatically run jobs at scheduled intervals (daily, hourly, etc).

To install Heroku Scheduler, simply run heroku addons:create scheduler:standard

Creating a scheduled task

In Rails, the convention is to create a rake task file in the lib/tasks/ directory.
Below are the tasks that I have written in my scheduler.rake file:

desc "This task is called by the Heroku scheduler add-on"
task :fetch_projects => :environment do
  puts "Deleting Tables..."
  Project.delete_all
  UserStarredProject.delete_all
  puts "done."
  puts "Fetching Projects..."
  Project.fetch(nextProject: false)
  puts "done."
end

task :fetch_countries => :environment do
  puts "Deleting Countries..."
  Country.delete_all
  puts "Fetching Countries... "
  Country.create_all
  puts "done."
end

As you can see, the fetch_projects task is the one that I will be adding to Heroku Scheduler to run regularly. It completes the actions of deleting the existing Projects and fetches new Projects from the GlobalGiving API.

After making sure the task is working locally and pushing it to Heroku, I can test that it is also functioning correctly on Heroku by running heroku run rake fetch_projects. If it's working right then it will work on Heroku Scheduler!

Scheduling Jobs

To get your rake task up on Heroku Scheduler, either open Scheduler in your app dashboard or the following command: heroku addons:open scheduler.

On the Scheduler dashboard, you can add and edit jobs. Jobs can be scheduled to run daily, hourly, or every 10 minutes. Next, you enter the task that the scheduler should run. In my case it was rake fetch_projects. Then just hit "Save Job", and it will start running when the next scheduled time is reached!

Below is what my scheduled job looks like:

Too much data!!

After I scheduled my fetch_projects job, everything seemed to be going fine and the databases began to populate. However, I checked my email, and saw a message with the shocking subject: "[warning] Database disruption imminent, row limit exceeded for hobby-dev database on Heroku app". That didn't sound good! I did some digging, and sure enough, I had over 45,000 rows in my app's Postgres database, while the limit for Heroku's Hobby-dev plan is 10,000 rows. I could either 1) upgrade to a higher database plan or 2) reduce the number of rows in my database.

Since I still need to build some more features in EffectiveDonate before I can start sharing it broadly, I decided to go with option 2 for now, and eventually I can always upgrade my database plan. This required a slight refactor in my database structure to avoid a join table that was adding multiple rows for each of the thousands of Project objects that I was creating.

Another useful tip I learned to prevent a scheduled job from continuing to run was a special CLI command. You can view all processes that are currently running on your Heroku app with heroku ps. If your scheduled task is running, you will be able to see its associated process id. Then you can stop it using heroku ps:stop <process id from heroku ps>, for example heroku ps:stop run.8359. This is an invaluable tool to use if you see that your database is fast approaching the 10,000 row limit, then you can stop the rake task's runaway train!

Conclusion

As you can see, the Heroku Scheduler add-on is an extremely powerful tool that can automate API fetches, or any other task that needs to be run regularly in your app. I love the Scheduler's easy-to-use interface and how nicely it plays with Rails' rake tasks. Now that I have my row count consistently under 10,000, my scheduled job has been running smoothly and there haven't been any more alarming emails reaching my inbox :)

Additional resources used in this blog:

https://stackoverflow.com/questions/7743039/killing-abandoned-process-on-heroku

China's Social Credit System

milandhar — Wed, 03 Jul 2019 22:42:46 +0000

Now that we are approaching the end of our time at the Flatiron School, we have started to focus more on writing algorithms to practice for technical interviews. I've found that practicing algorithms has been a good challenge - I enjoy the puzzle-solving aspect to them and it works a more mathematical part of the mind than Web Development areas. It can definitely be frustrating at times, especially when I don't even know how to get started, but solving a tough algorithm has been one of the more rewarding aspects of my software development journey thus far.

For this post, I wanted to broach a more controversial subject that involves a government implementing an algorithm at massive scale. I've been both fascinated and concerned by the intersection of technology and society over the past few years, so this topic caught my attention immediately. The issue at hand is China's development of its Social Credit System (社会信用体系).

What is the Social Credit System?

The system is intended to standardize the socioeconomic reputation of both businesses and individuals, or "Social Credit". By 2020, all social credit scores for China's 1.4 billion people will be publically available. Since China's economy and living standards have skyrocketed in the past decades, it hasn't been able to develop the type of credit scores we are used to in the West. This allowed for people to default on loans or businesses to use shady practices without much backlash. The social credit system is an attempt to answer the question of who to trust. The guidance for the system called for a nationwide system that would allow the trustworthy to roam everywhere under heaven, while making it hard for the discredited to take a single step.”

While the system is planned to launch in 2020, several Chinese companies have been licensed to start trials of social credit systems and develop the software and algorithms to calculate credit. The most prominent of these include Sesame Credit (owned by Alibaba) and Didi Chuxing (China's largest ride-sharing service). Sesame's pilot system determines a social credit rating from 350 - 950 based on "a thousand variables across five data sets".

What are the Scoring Factors?

Similar to credit scores, one's social credit rating can slide up and down based on their personal behavior. However, that is where the similarities end.

China's government agencies and companies are collecting massive amounts of data on its citizens, like social media activity, health records, and online purchases. Combined with its 200 million hidden cameras and facial recognition software, China's social tracking infrastructure makes it an unsettlingly perfect place for a social credit system.

The precise factors that the algorithm will use are secret, but some examples of infractions are bad driving, smoking in non-smoking zones, playing too many video games, and posting fake news. Examples of actions that would boost your score include purchasing diapers for a child, donating to charity, turning in a lost wallet, or picking up garbage.

How will a Social Credit Score be Used?

At this point, you may be wondering how the social credit system will end up affecting Chinese citizens' daily lives. How could playing too much PS4 impact their rights? The answer is: significantly.

If individuals have higher social credit scores, they enjoy lower energy bills, improved visibility on dating websites, and may rent things without a deposit. Chinese hospitals are also experimenting with social credit scores - enabling people with higher scores to skip a line and receive treatment immediately.

If someone has a low score, they may find it harder to purchase luxury goods, rent/buy a house, find a job, or purchase a train/plane ticket. Others may find it difficult to enroll their children in their desired school, may be publicly shamed at movie theaters, could have their internet speeds throttled, or have their dog taken away

Reflections

If all this sounds exactly like an episode of Black Mirror to you, that's because it is! The series 3 episode "Nosedive" explored this precise scenario. In this world, people rate each other from 1-5 stars for every interaction they have. One's current score can be seen by the public, causing an intense obsession by the episode's protagonist regarding her own score. I found this episode poignantly captures the phenomenon of social media obsession while also painting a haunting picture of future autocratic regimes' uses of big data.

I remember this episode was especially ominous to me, even though it wasn't as terrifying as many of the show's other episodes. It really brought out the worst of social media and how it plays into the human mind's intense desire for status and acceptance. It also seemed so plausible in real life given how ubiquitous social media and data tracking have gotten. When I heard that China was developing a similar system, it was clear to me that the future is now. I think that democratic governments like the US must get ahead of this and enact legislation both to restrict itself and private corporations from immorally collecting data on its citizens. Otherwise the dystopia presented by Black Mirror and the reality of China's social credit system could reach without us knowing it.

Another aspect of China's system that shocked me was how supportive people are of it - a survey shows that 80% of respondents either somewhat or strongly approve of social credit systems. This statistic made me realize that coming from a Western context, the idea of a social credit system seems Orwellian and completely backwards. However, Chinese citizens are closely aware that the government has been tracking them for years, and in a way this system merely digitizes the practice. Also the system will likely promote good behavior and dissuade people and businesses from stealing and committing fraud, so it is understandable that people could support it.

Finally, researching China's developing system further reinforced to me how powerful big data and algorithms can be. To think that a country of 1.4 billion people could have their rights controlled by an algorithm is mind blowing to me. It really makes the importance of ethical technologists and policy leaders even more pressing in the years and decades to come.

Thanks for reading!

Chrome Local Storage in Extensions

milandhar — Mon, 24 Jun 2019 22:20:21 +0000

For my Javascript project at Flatiron School's Web Development course, my partner and I decided to build a Chrome extension. When we got the project requirement, we wanted to create something that we would actually use, and also learn about a new area that we hadn't yet explored. Since we often find ourselves needing to remind ourselves to take a break, we decided to make a break timer extension!

While we knew creating an extension would be a different challenge than other Javascript projects we'd worked on, we underestimated how difficult it would be to make the app's data persist for a sufficient amount of time. The beauty of Chrome extensions is that they allow users to access a minimal app on their toolbar, opening and closing it whenever they want. However even a straightforward challenge like a break timer was tough, since clicking away from the extension effectively kills the Javascript session, wiping away all regular variable data.

Luckily, Chrome provides its own chrome.storage API that helps solve this problem, making developing extensions feasible.

What is Local Storage?

So far at the Flatiron School, we've been exposed to some browser-storage methods such as cookies and sessions in a Rails context. The Web Storage API contains two mechanisms: localStorage and sessionStorage. LocalStorage is similar to sessionStorage in that they are both read-only, but data stored in localStorage has no expiration time, while sessionStorage gets cleared every time the page session ends (the tab is closed).

Since localStorage is persistent among multiple page sessions, it is the ideal option for saving Chrome extension data, allowing users to click off and on the extension to their heart's content.

Using Chrome's Local Storage in Your Extension

Google has developed its own chrome.storage API that provides the same storage capacity as the Web Storage API. One thing to keep in mind when using chrome.storage is that the storage space is not encrypted, so confidential customer data should not be stored.

There are a few steps you must take and things to keep in mind when implementing Chrome's storage in your extension:

Manifest

Every Chrome extension has a manifest file called manifest.json which describes the app and provides important metadata such as its name, scripts, permissions and version. If you want to use chrome storage, you must declare the "storage" permission to your manifest.json file. Here is a snippet from our extension:

{
  "name": "break alarm",
  "version": "1.0",
  "description": "Be more productive.",
  "permissions": ["alarms", "declarativeContent", "storage", "<all_urls>"],
  "background": {
    "scripts": ["helpers.js", "background.js"],
    "persistent": false
  },
...
}

Storage.sync

One key difference between Chrome's storage API and the Web Storage API is that Chrome offers a storage.sync mechanism that allows user data to be automatically synced with Chrome sync. This would allow a user to access their data on a different device, assuming they have Chrome Sync enabled on their account.
So when developing your extension, you will need to decide whether you want to use storage.local or storage.sync. We decided to use storage.local for our break timer extension. The main difference between the two is the storage limit: storage.local offers about 5.2MB of data while storage.sync's limit is about 102KB, with maximum number of items stored of 512. A nice feature of storage.sync is that even if the user has disabled Chrome Sync, it will still work and just behave like storage.local.

Asynchronous Calling

Another cool feature of chrome.storage is that all of its calls are asynchronous, while the localStorage in the Web Storage API is called synchronously. While in our project I found the process of getting every locally stored variable asynchronously to be annoying at times, this mechanism allows extensions to run much faster.

Setting a new object in chrome.storage is relatively straightforward. In our break timer extension, when the user clicked the "resume" button, we would set the following in local storage:

chrome.storage.local.set({ isPaused: false })
chrome.storage.local.set({'user_name': json.user_name})
chrome.storage.local.set({'phone_number': json.phone_number})
chrome.storage.local.set({'redirect_url': json.default_url})
chrome.storage.local.set({'break_time': json.default_break_time});

When calling saved objects from chrome.storage, we can use a bulk async get function that will access multiple objects at once. Here's an example from our break timer extension:

  chrome.storage.local.get(['user_name', 'phone_number', 'redirect_url',
 'break_time', 'isPaused'], function(data) {
    h1.textContent = data.user_name
    timerInput.value = data.break_time
    urlInput.value = data.redirect_url
    phoneInput.value = data.phone_number

if (!data.isPaused) {
      updateCountdown();
      countdownInterval = setInterval(updateCountdown, 100);
      isNotPausedDisplay();
    } else {
      chrome.storage.local.get('pausedCount', function(data) {
        counterElement.innerHTML = secToMin(data.pausedCount);
      });
      isPausedDisplay();
    }

  });

Removing Items from Chrome Storage

In order to make full use of chrome.storage, it is important to understand how and when to remove stored data. In our timer extension, we had user data stored such as their preferred break time, their preferred "work" url, and their phone number for timer alerts. We wanted to persist the user's data while they were logged in so that they could set several timers and didn't have to re-enter all their settings if they returned to the extension. However when they log out, we want to remove the user's stored data from chrome.storage so that another user could log in on the same machine.

Luckily, chrome.storage makes it easy to do this! Chrome offers both a chrome.storage.local.remove function that lets you specify which key(s) you would like to remove, and a chrome.storage.local.clear function that will clear all data stored in the extension's local storage. Both functions have optional callback function parameters that will return on success or failure (and then runtime.lastError will be set).

Since in our extension, we wanted to remove all the user's data once they logged out of the app, we wrote a method that used the clear function with an error-catching callback:

function clearLocalStorage(){
  chrome.storage.local.clear(function() {
    var error = chrome.runtime.lastError;
      if (error) {
        console.error(error);
      }
   })
 }

Conclusion

Overall, chrome.storage was an indispensable tool to ensure that our extension ran correctly and persisted the user data that we wanted. While building an extension seemed daunting at first, with so many new files and specifications, learning how to use chrome.storage made things so much easier. I would highly recommend heading over to Chrome's extensions documentation when developing your first extension, it is very helpful and concise.

Thanks for reading!

The Benefits of Pair Programming

milandhar — Wed, 12 Jun 2019 21:40:11 +0000

"It is the long history of humankind (and animal kind, too) that those who learned to collaborate and improvise most effectively have prevailed." – Charles Darwin

One aspect of my time at the Flatiron School that I have greatly enjoyed has been getting to program with other members of my cohort. The curriculum includes both smaller "pairing labs" that last a few hours and larger pair projects that span four to five days. Both styles have given me a taste of what it will be like to work in a team development environment, and also have been immensely beneficial to my learning and fluency with many concepts. It has been interesting to work with people of different personality types and technical strengths. Pair programming also has challenged the stereotypical image I used to have of engineers locking themselves in a dark room alone with a can of Mountain Dew at 2am.

While these pairing labs and projects have been great so far, I've found myself interested in how to optimize my own efficiency and performance when working in pairs. I have also wondered whether pair programming has been proven to be more effective than individual work. These questions have led me to conduct additional research on the topic of pair programming, which resulted in this blog!

What is Pair Programming?

At a basic level, pair programming is the agile technique of two people sitting side by side while developing software on one computer. In most pair programming scenarios, one programmer (the driver) writes code while the other (the navigator) reviews each line as it's typed. While at face value it seems that this practice is a waste of resources since it is requiring two people to complete a task that one could do, studies have shown that pair programming has a significant impact on engineering and organizational effectiveness.

Economics of Pair Programming

In a study at the University of Utah, students with no exposure to pair programming were assigned three tasks to complete in pairs, with 1/3 of students working on their own as a control group. As you can see from the graph below, after the adjustment period shown in first program, the paired students only spent about 15% more time than the individual students.

Offsetting the increased man-hours spent on pair programming, the code produced by the pairs had about 15% fewer defects than the individually developed code (see graph below). Since the average bug takes between four and 16 hours to resolve, the benefit of pair programming to reduced bugs would outweigh the cost of additional man-hours in most projects.

Satisfaction

Having never heard of pair programming before starting at Flatiron, I was initially skeptical when I discovered we would be doing so much of our work in pairs. I thought it would hinder my learning to watch someone else do the work 50% of the time, and was worried that communication would be difficult. However, this uncertainty has gradually faded as we have progressed through the curriculum and gotten more practice working in pairs.

The study at University of Utah reflected a similar sentiment among new pair programmers. The graph below shows the results of surveys of both the student pair programmers and professional pair programmers. It is clear that most of the programmers preferred to work collaboratively.

Design Quality

Another effect of pair programming is an improved design quality of the resulting program. In the University of Utah study, the paired programmers finished their projects with a higher quality than the individuals and also implemented the same functionality as the individuals with fewer lines of code.

Styles of Pair Programming

There are several styles of pair programming that fit different scenarios and levels of experience between the teammates.

Driver-Navigator

Driver-navigator is likely the most widely-used style of pair programming. The analogy is relatively clear: one member of the pair acts as the "driver", doing the typing, ensuring the mechanics of the code are in place and any obstacles are avoided; the other member plays the part of the "navigator", checking for mistakes, and thinking big picture about what goals must be accomplished and the overall architecture of the project.

The driver-navigator style works well with either two experts or an expert-novice pair. In this situation it is best for the expert to be the navigator and the novice to be the driver, to prevent the novice from merely observing while the expert assumes both roles.

Backseat Navigator

This style is similar to driver-navigator, but the navigator takes over more of the tactical roles from the driver. In backseat navigation, the driver still handles the keyboard and typing, but the navigator dictates syntactical instructions, such as what name to call a variable or what specific method to call.

The backseat navigator style works best with a novice as the driver and expert as the navigator, allowing the novice to learn by doing.

Ping Pong Pairing

Another style of pair programming that is often use in development is ping pong pairing. In this pattern, the first person writes a test that is currently failing, and the second person gets the test to pass. Then the second person writes a failing test and the first person gets it to pass.

The benefits of ping pong pairing are that it enables roles to switch frequently and forces engineers to pay attention to both the coding and testing aspects of development, gaining familiarity with TDD in the process. This style of pairing works best with two experts, especially if they are also experts in TDD. But it can also work well with novice-expert pairing, and novice-novice pairing, to give beginners practice with writing tests.

Distributed Pairing

This is still more of an unofficial style of pairing, but it is likely the future of pair programming. As more development workspaces shift towards remote and global environments, and screen sharing technology becomes ubiquitous, distributed pairing continues to grow in prevalence.

Distributed pair programming involves both individuals working via a real-time collaborative editor, shared desktop, or remote IDE plugin. Distributed pair programming benefits from the flexibility of being in separate locations, but introduces new difficulties that come with the loss of in-person communication. This can make remembering which programmer has the role of driver ambiguous at times.

Conclusion

Ultimately, it has been inspiring to learn and experience how much pair programming has been emphasized both at Flatiron and the larger developer community. I look forward to continuing to refine my pair programming skills and learn from my colleagues!

Sources I referenced in this post:

https://en.wikipedia.org/wiki/Pair_programming
https://collaboration.csc.ncsu.edu/laurie/Papers/XPSardinia.PDF
https://stackify.com/pair-programming-styles/
https://medium.com/@weblab_tech/pair-programming-guide-a76ca43ff389

How To Select a Relational Database Management System

milandhar — Mon, 03 Jun 2019 20:15:17 +0000

At end of my 6th week at Flatiron School's Web Development course, my partner and I had finally completed our Caribbean Stud Poker Rails Project and were ready to deploy it to Heroku. However, we had started our project using a SQLite database instead of using PostgreSQL, which ended up causing a frustrating (but not insurmountable) problem. Heroku, it turns out, is not compatible with SQLite. While we were eventually able to migrate our sqlite3 database over to postgres and deploy to Heroku, it was not immediately clear to me why the database migration was necessary, or even what the differences were between the two Database Management Systems (DBMs)'s. I think this topic will prove even more relevant as we continue developing more complex websites, which is why I wanted to learn more about it!

Database Terminology

While this seems like a basic topic - I've noticed the terms "database" and "database management system" used interchangeably, and it's important to get these right.

A database is an organized collection of data, generally stored and accessed electronically from a computer system. For example, the file 'development.sqlite3' in the above image is a database that stores valuable information for our poker app in tables like Users, Cards, Hands, etc.

A database management system (DBMS) is the software that interacts with end users, applications, and the database itself to capture and analyze the data (run queries). In our poker example, we initially used the SQLite DBMS and then migrated to PostgreSQL.

SQL is a domain-specific language designed for managing data held in a relational database management system. This is the language that we use to actually create our tables and store, manipulate and read our poker game data.

Types of DBMS's

Another topic that initially confused me was why there was a need for multiple DBMS's. Wouldn't it be better if there just be one that worked for every use case? Maybe, but it turns out that different types of projects are best suited to different DBMS's based on their storage, speed, and functionality.

SQLite

Embedded Database

According to its website, SQLite is described as "serverless", which is different from most DMBS's. Usually, a program that wants to access a database can communicate with the server to send requests and receive data back. SQLite skips the intermediate server process and requires programs to read and write directly from the saved database file.

Advantages of SQLite

A serverless DBMS avoids the administrative need to install, setup, configure, initialize, manage, and troubleshoot a server process. This makes it easier to integrate with an application.
SQLite allows multiple applications to access the same database simultaneously (rare for serverless DBMS's).
Unsurprisingly, SQLite's library is very lightweight. And the entire database is stored in one file, making it extremely portable.

Disadvantages of SQLite

Being serverless makes it harder to provide protection from bugs and corrupted memory in the client application.
Even though it can host multiple applications at the same time, only one process can make changes to a SQLite database at one time.
SQLite does not have a user management system that give users defined access privileges to the database.

When to use SQLite

Embedded applications like mobile applications, game consoles, televisions, airplanes, drones, watches, and robots (the internet of things).
Testing. Includes an in-memory mode that can run tests without the overhead of actual database operations.
Working with small amounts of data. The SQLite website recommends that any database approaching 1TB be housed on a centralized client-server database.
Only require 1 writer at a time.

PostgreSQL

Client-Server Model

PostgreSQL labels itself as "The World's Most Advanced Open Source Relational Database", and uses a client-server model requiring a DB server to set up and run over the network, as opposed to SQLite's serverless model. In the client-server architecture, each sessions consists of:

A server process, which manages the database files, accepts connections to the database from client applications, and performs database actions for the client. The database server is called postgres.
The user' clients application that performs database operations. These client applications could be anything from a web server to a graphical application to a special database maintenance tool.

The Postgres server can handle multiple simultaneous connections from different clients, which is a huge strength. For each new connection, the server "forks" a new server process, and then the client and new server process can communicate without intervention by the original postgres process.

Advantages of PostgreSQL

Postgres is very compliant to SQL standards.
It is an open source project and community driven.

Disadvantages of PostgreSQL

Since it forks a new process for every new client connection, it has poor memory performance (each new process is allotted about 10MB of memory).
It is less popular than other client-server DBMS's like MySQL, meaning there aren't as many third-party tools to help manager a Postgres DB.

When to use PostgreSQL

Need to integrate the DB with other tools. Postgres is compatible with numerous other programming languages and platforms. So if you ever need to migrate your database to fit with a different tool, Postgres is a great option.
Using complex operations such as data warehousing and online transaction processing.
Your project doesn't require the fastest read speed.
You are interested in optimizing data integrity.

Our un-deployable sqlite3 file

Migrating from SQLite to Postgres

Based on the factors detailed in the section above, it is understandable that a cloud-based platform like Heroku would require a client-server DBMS like Postgres to maintain data consistency and not the disk-backed storage provided by SQLite.

Since Rails has SQLite installed by default and we used an sqlite3 database for our Poker project, we had to migrate our DB to Postgres in order to deploy to Heroku. Here is what we did:

Remove the following line in the Gemfile:
gem 'sqlite3'
and replace it with
gem 'pg'. Then run bundle install
Next, convert the config/database.yml file. Replace the adaptor lines that read adapter: sqlite3 to adapter: postgresql. Also, change the database: to a custom line.
After changing the Gemfile and the config/database.yml, you will have to create and migrate your new Postgres database:
```
$ rake db:create
$ rake db:migrate
```
Finally, just run a git add . and git commit -m "postgres", and you will all set up with your Postgres DB. At this point you can deploy your project to Heroku with git, if that is your end goal.

Conclusion

Hopefully this post was able to detail some of the main differences between two widely-used DBMS's (PostgreSQL and SQLite), and will help you select an optimal DBMS for your next project. Feel free to add any questions or comments below!

Apart from the inline links, I used the following references in this post:

https://en.wikipedia.org/wiki/Database
https://en.wikipedia.org/wiki/SQL
https://www.digitalocean.com/community/tutorials/sqlite-vs-mysql-vs-postgresql-a-comparison-of-relational-database-management-systems
https://devcenter.heroku.com/articles/sqlite3