Forem: Divine Hycenth

Front-end Engineer in 2021

Divine Hycenth — Sun, 07 Mar 2021 16:47:01 +0000

Technical requirements for a front-end engineer in 2021.

Disclaimer: This article is based on my observation, opinion, and personal
preference.

Table of content

Programming Language
Frontend Framework
CSS Frameworks
Git and Git workflow
UI Libraries
- React
- Vue
Design Tools
Static Site Generators
SEO and Accessibility
JAMstack
Progressive Web Apps (PWA)
Testing
Graphql
Animations
Other things you should know to level up your game

Back in the days, All you need to know as a front-end engineer is just HTML,
CSS, jQuery/Javascript, and probably, A tool like Adobe Illustrator to create
designs. However, Things have changed in the past few years and it's constantly
changing.

There're a lot of front-end frameworks React, Vue, Svelte, Redwood.js, just to
mention but a few, and also uncountable libraries that support us in building
standard applications and websites.

Programming Language.

Speaking of programming languages, the first thing that comes to your mind is
Javascript, and that's correct because it's the language of the web, However,
You need to know Typescript in 2021 and only a few companies do not have
Typescript as a requirement. You might as well consider learning Web Assembly as
it's is becoming more and more popular.
Read more about web assembly.

Frontend Framework

The three most common Front-end frameworks are React,
Angular, and Vue, and they're going
to be around for a long time. The fight right now is between Facebook's React
and Vue. Vue currently has over
180k stars on Github while
React is at 165k stars, However, React has
over 10.4m weekly downloads while Vue is at 2.3m downloads per week. In
conclusion, React is more popular than Vue with about 80%.

Everyday, We are nearing Death, and a new Javascript framework. "Anonymous"

Survey from
stateifjs.com

CSS Frameworks

You're going to need a CSS framework most of the time because of speed,
flexibility, and a shorter time to build an application.

There's a lot of CSS frameworks out there that suits your specific need, and the
first, oldest, and most popular on my list is Bootstrap.

Git and Git workflow.

I strongly recommend that you learn how git and git-flow commands as it is
widely used for version control and most importantly, It's going to save you
a*s most of the time. Here are some basic git commands you need to know:

git init
git clone
git status
git add
git commit
git push
git pull
git branch

Check out this cheat sheet for a comprehensive guide on how to use
git-flow.

UI Libraries (React and Vue).

This section is targeted towards React/Vue developers as I have remarkable
experience with these frameworks.

As a front-end engineer, You might want to rapidly build an application/website,
and writing CSS alone is time-consuming.

React.

There're countless React UI libraries out there, Nevertheless, I would like to
mention a few that I've used and can comfortably use again and again.

Nature UI: This library is focused on tailwindcss lovers because most of its customizations are done with tailwindcss and it also offers you the ability to style components with emotions CSS prop. I am the creator of this library BDW so check it out :).
Material UI: This is by far the most popular UI Library out there with a lot of components to handle nearly everything you need to build your application. You must also note that customization can be tedious.
Chakra UI: Chakra UI is another goto library for building react applications with speed and flexibility if you prefer to style your components with props.
Ant Design: This is more of a slick and minimal design-focused UI library with a lot of components to handle nearly everything you need to build your application.
Semantic UI: I have used Semantic UI in the past and must say that it is quite flexible and easy to get started with.

Vue.js

Vuetify: Vuetify is another Google's material design standard library, same as Material UI but for Vuejs.
Chakra UI Vue: This is just a Chakra UI library for building Vuejs applications.
Vue Tailwind: As a tailwindcss lover, I am not going to skip this library, although it doesn't have a lot of components like the rest, the developers are constantly adding more components.

Design Tools

As a front-end engineer, Most people assume that you must be pretty good at
designing, Therefore, I would like to mention a few popular design tools I use
daily:

Figma: This is the tool I use for almost all my designs. A few benefits of Figma is that it has a cross-platform desktop application for most operating systems and it is open source. The web version works the same as the desktop version.
Adobe XD: Although I had the opportunity to try out Adobe XD, I couldn't continue to use it because of its compatibility with Linux Distros and I'm a Linux user. It is quite easy to use and I can say that It's no different from Figma.

Static Site Generators

Static sites have been the standard output of websites in the past because most
websites comprised of HTML, CSS, and Javascript, which made them SEO-ready.
However, Things are no longer the case since the emerging of Front-end
frameworks like React, and Vue, therefore, the outputs are most likely to be
dynamic.

As a good front-end engineer, You should know how to use at least one of the
following Static Site generators.

Nextjs (React based)
Gatsbyjs (React based)
Nuxtjs (Vue based)
Gridsome (Vue based)

Learn more about
Static Site Generator

SEO and Accessibility.

Most companies usually have a designated team for handling Search engine-related
things, However, I suggest that you learn how to write accessible and search
engine optimized code as it's going to be a huge plus to you and your company in
the long run.

JAMstack

The term JAMstack stands for JavaScript (running on the client - for example,
React, Vue, or VanillaJS), API (server-side processes are abstracted and
accessed over HTTPS via JavaScript), and markup (templated markup that's
prebuilt at deploy time). Find out more about JAMstack on
jamstack.wtf.

Client-side validation.

Most APIs are bundled with built-in validations, However, I recommend that you
add an extra validation on the client-side. This can reduce the number of
requests sent to the server, add an extra layer of security, and saves a
substantial amount of bandwidth when consuming an API with rate limiting.

Here are a few validation libraries I have used and I strongly recommend:

React-hook-form (React-based): This is a Performant, flexible, and extensible form validation library.
Formik (React-based): Formik is one of the most popular forms validation libraries.
Yup: This is a JavaScript schema builder for value parsing and validation.

You might as well try JSON schema for rapid development of forms with
validations.

PWA

Progressive Web Apps are cross-platform,
performant, hybrid applications with access to native modules. Here are a few
benefits of PWAs:

Build once, run everywhere: This implies that your single code can run on the Web, Android, iOS, and Desktop, as native applications.
Boosts performance.
Tiny app size.
Push notifications.

Testing

Building an application without tests is like solving a mathematical problem
without proof. You need to test for behaviors and states of components in your
application to determine that it is working as expected.

You should at least know how to use some of these testing libraries:

Graphql 🔥

Graphql has been dominating both in the server-side world
and the front-end world. It was first released in 2015 by Facebook to tackle
issues developers are facing.

Check out this comprehensive comparison between Graphql and Rest
https://blog.api.rakuten.net/graphql-vs-rest/.

Will Graphql replace REST?. I'd like to know your take on this.

Edit: If you're already using Graphql to build APIs, then I recommend you checkout Butter CMS. Here's a more recent comparison between graphql and Rest, with detailed examples https://buttercms.com/blog/graphql-vs-rest-api

Animations

Animation is one of the most effective tools that you have to use on your
website. The skillful use of animation on your website or blog can help to
increase your targeted traffic in ways that ordinary text-based content cannot
do for you.

Here are a few most popular animation libraries:

Other things you should know to level up your game.

Storybook js: This is an open-source tool for developing UI components in isolation for React, Vue, and Angular. It makes building stunning UIs organized and efficient. An alternative to Storybookjs is Bit.
Add search functionality to your website or web app as it helps users to easily search and navigate through the contents of your app. Here are a few open-source search libraries I strongly recommend:
- Algolia Docsearch
- Match Sorter

What do you think is missing in this article? Let me know in the comments.

Happy Codding 💻 🤓

Convert a Base64 data into an Image in Node.js

Divine Hycenth — Sun, 15 Nov 2020 16:48:02 +0000

Learn how to convert an image into a base64 string and back to an image.

Bonus:

You will also learn how to compress images with Jimp.

Recently, I got involved in a project where images are returned from the browser in base64 format and we need to write the image to disk. I did some research to see if i could get any detailed explanation on how things work with base64, Buffer, binary and real images but couldn't find any solution related to nodejs.

Trying to get this to work directly by reading an image with the fs.readFile... method and piping it back to an image would work perfectly because by default, fs.readFile... returns a Buffer and that is what we need. Here's an example.

const fs = require("fs");
// Reads file in form buffer => <Buffer ff d8 ff db 00 43 00 ...
const buffer = fs.readFileSync("path-to-image.jpg");
// Pipes an image with "new-path.jpg" as the name.
fs.writeFileSync("new-path.jpg", buffer);

Things tend to get a bit more complicated when you want to convert a regular (base64) string into an actual image. You need to first convert your string into Buffer before saving it as a real image otherwise, you're going to run into issues.

// Base64 => Buffer => Image

Luckily, Node.js provides a native module called Buffer that can be used to perform Base64 encoding and decoding. Buffer is available as a global object which means that you don't need to explicitly require this module in your application.

Internally, Buffer represents binary data in the form of a sequence of bytes. The Buffer object provides several methods to perform different encoding and decoding conversions. This includes to and from UTF-8, UCS2, Base64, ASCII, UTF-16, and even the HEX encoding scheme.

Base64 Encoding

Let us first of all convert our image into base64 and then to Buffer

const fs = require("fs");
// Create a base64 string from an image => ztso+Mfuej2mPmLQxgD ...
const base64 = fs.readFileSync("path-to-image.jpg", "base64");
// Convert base64 to buffer => <Buffer ff d8 ff db 00 43 00 ...
const buffer = Buffer.from(base64, "base64");

Base64 To Image

Now you can convert the Buffer into an actual image with just a single line of code:

// Pipes an image with "new-path.jpg" as the name.
fs.writeFileSync("new-path.jpg", buffer);

Bonus

Image optimization is very important these days because most images come in huge sizes and this will definitely slow down our websites. I'll be using Jimp to show you how to reduce image quality on the fly by reducing the size of the Buffer data we receive and convert it into an actual image.

Install Jimp:

yarn add jimp
   # or
npm install jimp

You can simply reduce the quality of your image by 50% with just a few lines of code. I am going to use a base64 string as my data source.

const Jimp = require("jimp");
const fs = require("fs");
// Base64 string
const data =
  "/9j/4AAQSkZJRgABAQAAAQABAAD/2wCEAP////////////////////////////////////////////////////////////////////////////////////8B///////////////////////////////////////////////////////////////////////////////////////AABEIAMgBLAMBEQACEQEDEQH/xAGiAAABBQEBAQEBAQAAAAAAAAAAAQIDBAUGBwgJCgsQAAIBAwMCBAMFBQQEAAABfQECAwAEEQUSITFBBhNRYQcicRQygZGhCCNCscEVUtHwJDNicoIJChYXGBkaJSYnKCkqNDU2Nzg5OkNERUZHSElKU1RVVldYWVpjZGVmZ2hpanN0dXZ3eHl6g4SFhoeIiYqSk5SVlpeYmZqio6Slpqeoqaqys7S1tre4ubrCw8TFxsfIycrS09TV1tfY2drh4uPk5ebn6Onq8fLz9PX29/j5+gEAAwEBAQEBAQEBAQAAAAAAAAECAwQFBgcICQoLEQACAQIEBAMEBwUEBAABAncAAQIDEQQFITEGEkFRB2FxEyIygQgUQpGhscEJIzNS8BVictEKFiQ04SXxFxgZGiYnKCkqNTY3ODk6Q0RFRkdISUpTVFVWV1hZWmNkZWZnaGlqc3R1dnd4eXqCg4SFhoeIiYqSk5SVlpeYmZqio6Slpqeoqaqys7S1tre4ubrCw8TFxsfIycrS09TV1tfY2dri4+Tl5ufo6ery8/T19vf4+fr/2gAMAwEAAhEDEQA/ACqJCgAoAKACgAoAWgAoAKBBQAUAFABQAUAFABQAmaAEoGFABQAUgFoAWgQUAFABQAUAJQAUAFABQMKACgAoASgAoAKACgBaYBQAUAFABQAUALQAUAFAgoAKACgAoAKAEoGJQAUAJQAtAC0AFIBaBBQAUAFABQAUAJQAlAwoAKAFoAKAEoAKACgAoAKAFpgFABQAUAFAC0AFAgoAKACgAoAKACgBCaAE/wA/WgYlAwoAKBC0gFoAWgQUAFABQAlAwoAKAEoAKACgAoAKACgAoAKACgAoAKAFpgFABQAUALQAUAFAgoAKACgAoAKAEoASgYlAxKAFoAKBDqQC0CCgAoASgAoAKBhQAUAJQAUAJQAUDCgAoEFABQAUALQAUALTAKACgAoAWgAoAKBBQAUAFABQA2gYUDEoAKAEoAWkIWgBaBC0AJQAUDCmFhM0BYKACgAoAKACgBKACgYUAFIAoAKACgBaAEoAdTEFABQAUALQAUAFAgoAKACgBDQMbQMKACgBKAFoAWkIWgAoEFACUxiUAFABQAUAFAC0AFABQAlABQMKACkAUAFABQAUAFADqYgoAKAFoAKBBQAUAFABQAhoGNoGFABQAUAFAC0CFpALQISmAlAwoAKAEoGFABQAUAFABmgAoAKACgAoAKACgAoAKACkAtAC0xBQAUALQAUCCgAoAKACgBDQMbQMKAEoAWgAoAWgQtIAoEJTGFACUDCgAoAMUALigVwxQFxMUDCgBKACgAoAKAFoAKAEoAKAFpAFADqYgoAKAFoAKBBQAUAFABQAhoGNoGFABSAWgAoEFABTGFAhKBhQAUCHYoC4tIQUAFACUAFAxKAEpgJQMKACgAoAKACgYUhC0CFoAWmAUAFAC0AFAgoAKACgAoASgYlABQAUgFoAKACgBOtMBKACgYUAAoELmgLBmgAoASgAoAKBhQAUAJQAUAFABQAUAFAxaQhaBBQAtMAoAKAFoAKBBQAUAFABQAUAJSGJQAUALQAUCCgBD60xiUDCgAoAKACgAoAKACgAoAKBBQAUDCgBKACgAoGFABQIWkAtAgoAWmAUAFAC0AFAgoAKACgAoAKACkAlABQMKACgQUDDrTATFACUDCgBaBBQAUAGKACgAoAKACgAoAKAEoGFACUALQAUALSEFABQAtMAoAKAFoAKBBQAUAFABQAUAFIBKBhQAUAJTAKACgAoAKACgAoAKAFxQIOlADaBjhQDCkIKACmAUDEoAKAEoGFAC0hC0AFACUALTAKAFoAKACgQUAFABQAUAFABSASgYUAFABTASgAoAKACgAoAKACgB1AhtAxKBjhQSxaQgoGJTAKBiUAFABQAUAFIBaACgAoAKYBQAUAFAC0CCgAoAKACgAoAKQBQAlACUDFoASgBKYBQAtABQAUAAoAdSJExQUJimFxwoEFIQUAFMYlAxKACgANAAKACkAtABQIKACmMKACgBaACgQUAFABQAUAFABSAKAEoAKBhQAUAJTAKACgAoAKACgB1IkKACgBaAEoAKACmMSgYlABQAGgAoAWkAUAFAgoAKYwoAKAFoAKBBQAUAFABQAUAFIAoASgdwxQFxKACgApgFABQAtACUAAoBjqQgoELQAUAJTASgYlAwoAKACgBKQxaBC0CCgAoAKACmMKACgBaACgQUAFABQAUAFABSAKACgBKACgYUAJTAKAFoAKAEoBCUAPpCCgQtACUAFMYlAxKACgAoAKACkAtAgoAWgAoASgApjCgAoAKAFoEFABQAUAFABQAUgCgBKACgBaAEoGFACUALTEFACUDCgBaBC5oAKAEoAKACgBKBhQAUgCgBaBBQAUALQAUAFACUxhQAUALQAUCCgAoAKACgAoAKQBQAUAFABQAlAwoAKACmAUCDFABQAUAFABQAUAJQMKACgAoAKQC0AFAgoAWgAoAKACgBKYwoAKACgBaACgAoEFABQAUAFABQAtIBKAEoAWgAoAKAEpgGKAFpAFABTEJQMKBiUAFIBaAEoAKACgBaAFoEFABQAUAFABQAlMYUAFABQAtABQAUCCgAoAKACgAoAKQBQAlAC0AFABQAUwCgApAGKAEphYM0DsHNACUALSAKACgAoAKBC0AFABQAUAFABQAUAJTGFABQAUALQAUAFAgoAKACgAoAKACkAUAFABQAUAFABQAUAFABQAUAJQAUAFABQAtACUDCgQtABQAUAFABQAUAFMAoASgYUAFABQAUALQAUCCgAoAKACgAoAKQBQAUAFABQAUAFABQAUAFABQAUAFABQAUAFACUDFoEFABQAUAFABQAUAFMAoASgYUAFABQAUALQAUAFABQIKACgAoAKQBQAUAFABQAUAFABQAUAFAC0CCgAoAKACgBKBhQAUAFABQAUAFABTAKACgAoA/wD/2Q==";
// Convert base64 to buffer => <Buffer ff d8 ff db 00 43 00 ...
const buffer = Buffer.from(data, "base64");
Jimp.read(buffer, (err, res) => {
  if (err) throw new Error(err);
  res.quality(5).write("resized.jpg");
});

I would love to give some extra credits to the creators of Jimp because it solves nearly everything you would want to do with an actual image in nodejs and has zero dependency. I've had zero issues using it in an electron application.

Conclusion

We were able to:

Understand how images work in nodejs on different stages.
Convert from Buffer to base64.
Convert from Buffer into an actual image.
Use Jimp to reduce image size.

If you made it to this point, then I'm going to say congratulations 🍻 because you're awesome :).

I hope you find this helpful.

Happy codding 💻 🙂

Build a Desktop app with Gridsome and Electron.

Divine Hycenth — Fri, 07 Aug 2020 20:39:44 +0000

Learn how to combine your knowledge of Gridsome for SPA and Electron for desktop apps to build a desktop app.

Electron and Gridsome was the basic requirement for an application i and my team were working on so i started to make some research on how to use gridsome in electron both in development and production with a smooth developer experience but i couldn't find any article or video on building desktop applications with electron and gridsome that is why i'm writing this article because I'm sure someone might run into some issues i faced setting it up :).

Prerequisites

Basic knowledge of Javascript
Must have Node installed on your machine.
Basic knowledge of Electron

Project structure

What is Gridsome?

Gridsome is a Vue-powered static site generator for building blazing fast static websites. It is data-driven meaning it uses a GraphQL layer to get data from different sources in order to dynamically generate pages from it. Gridsome is the Vue alternative to Gatsby for React.js.

As this tutorial is not a beginners guide for gridsome, I'll not go into full details of gridsome.

Setting up Gridsome.

Installation.

Run the following command to install Gridsome globally in your computer:

yarn global add @gridsome/cli
 # or
npm install -g @gridsome/cli

Next step is to create a new project using the CLI we just installed:

gridsome create gridsome-electron-boilerplate

This command will bootstrap a new gridsome site for you and install all the necessary dependencies.

Once that is done installing we can run the Gridsome site with this command:

cd gridsome-electron-boilerplate && gridsome develop

Electron

Electron, managed by Github, is used in a myriad of projects + companies. Making your web apps available for desktop makes them that much more versatile. So it makes sense to have a fundamental understanding of Electron, and how to integrate it into current projects.

What is Electron?

Electron allows you to build cross-platform desktop applications, using a Node.js backend and a Chromium front-end. It might seem like a challenge to turn your Gridsome web-app into a full fledged, downloadable desktop app — but Electron’s objective is to make this process seamless. I was intimidated at first, but the learning curve wasn’t nearly what I thought it would be.

Basic structure of an electron app:

Install electron in your project by running:

yarn add -D electron@latest

You might have noticed that our base electron files are situated in a folder called electron and I'm emphasizing on this because electron is not going to work without making few changes to our package.json file.
You need to point the main in your package.json file to point to your electron main.js fille.

After that, your package.json should look like this:

{
    "name": "hardocs-electron-gridsome",
    "private": true,
    "main": "electron/main.js", // pointing to the main.js in electron folder
    "dependencies": {
        "@gridsome/source-filesystem": "^0.6.2",
        "gridsome": "^0.7.0",
    },
    "devDependencies": {
        "electron": "^9.1.0",
        ...
    },
    ...
}

You can simply run the following commands to startup your electron app:

npx electron .

Now we need to load gridsome contents into our electron app.

Add the following lines of code to load gridsome url in your electron app:

const { app, BrowserWindow } = require("electron");

function createWindow() {
  const mainWindow = new BrowserWindow({
    width: 800,
    height: 600,
  });

  mainWindow.loadURL("http://localhost:8080");
}

app.whenReady().then(() => {
  createWindow();

  app.on("activate", function () {
    if (BrowserWindow.getAllWindows().length === 0) createWindow();
  });
});

app.on("window-all-closed", function () {
  if (process.platform !== "darwin") app.quit();
});

Before you start up your electron app you need to make sure your gridsome website is running on http://localhost:8080 then you can proceed to run this command in your terminal:

npx electron .

Bravo 😁. Gridsome is working perfectly in electron.

It's clear that electron is loading gridsome pretty fine in development mode. However, It's totally different in production mode because in production, gridsome generates just html, css and javascript files.

Now lets get everything working in both development and production environments.

To do this, we need the help of this packages:

yarn add live-server

Live-server: Live server helps us to serve html, css and javascript files including folders with LIVE RELOAD.

Basically, what we're going to do is to conditionally load the build index.html file from gridsome or load gridsome development server url.

const { app, BrowserWindow } = require("electron");
const path = require("path");
const liveServer = require("live-server");

const distDir = path.join(__dirname, "..", "dist"); // require build directory
const host = "localhost"; // host: localhost or 0.0.0.0
const port = 4000;
const url = "http://" + host + ":" + port; // => http://localhost:8081
const params = {
  port: port,
  host: host,
  root: distDir,
  open: false, // prevents auto-open in browser
  file: "index.html",
};

function createWindow() {
  const mainWindow = new BrowserWindow({
    width: 800,
    height: 600,
  });
  if (process.env.NODE_ENV === "prod" || process.env.NODE_ENV === "production") {
    // Here we check if we're in production env
    liveServer.start(params); // startup live-server
    mainWindow.loadURL(url); // load the url in gridsome
  } else {
    mainWindow.loadURL("http://localhost:8000");
  }
}

app.whenReady().then(() => {
  createWindow();

  app.on("activate", function () {
    if (BrowserWindow.getAllWindows().length === 0) createWindow();
  });
});

app.on("window-all-closed", function () {
  if (process.platform !== "darwin") app.quit();
});

From the code above, you can see that we're either loading the url we get from live-server or the gridsome dev server url.

However, we need to build our gridsome application into static html, css and js files and to do that we have to tweak our package.json a little bit.

your final package.json file should look exactly like this:

{
  "name": "hardocs-electron-gridsome",
  "private": true,
  "main": "electron/main.js",
  "scripts": {
    "build": "gridsome build",
    "develop": "NODE_ENV=dev gridsome develop",
    "electron": "electron ."
  },
  "dependencies": {
    "@gridsome/source-filesystem": "^0.6.2",
    "gridsome": "^0.7.0",
    "live-server": "^1.2.1"
  },
  "devDependencies": {
    "electron": "^9.1.0"
  }
}

Now, run the build command to generate a build directory:

yarn build

We can now run yarn start our electron app by running:

yarn electron

However, you might run into issues when your live-server fails to start before the electron app so we need some extra help to handle that part and also setup a proper development experience.

yarn add wait-on

wait-on: wait-on - wait for files, ports, sockets, http(s) resources.

Finally, we need to do some shell scripting.

Create a startServer.sh file in your root directory and add the following lines of code:

#!/bin/bash
read -p "Enter your environment: " NODE_ENV

if [ "$NODE_ENV" = "production" ] || [ "$NODE_ENV" = "prod" ]
then
    echo $NODE_ENV;
    exec yarn prod;
else
    echo $NODE_ENV;
    exec yarn develop & yarn wait
fi

What the above code does is that it would prompt you to enter your environment and if it matches prod or production then it would execute the first if block else it would execute the else block.

Now add these scripts to your package.json and you're all set :

{
    ...
    "scripts": {
      "build": "gridsome build",
      "develop": "gridsome develop -p 8000",
      "explore": "gridsome explore",
      "prod": "NODE_ENV=production electron .",
      "wait": "wait-on http://localhost:8000 && electron .",
      "grant": "chmod +x ./startServer.sh",
      "start": "./startServer.sh"
  },
  ...
}

I'll focus on the develop, prod, wait, grant and start commands.

develop: I specified a port 8000 for the development mode of the gridsome server.
prod: assigns the NODE_ENV to production
wait: uses the wait-on package we installed earlier to wait for port 8000 to be ready before executing the electron . command
grant: makes startServer.sh executable.
start command executes startServer.sh file.

make startServer.sh executable by running:

yarn grant

then startup your project smoothly with:

yarn start

You can find the entire code for this article on my github repo https://github.com/DNature/gridsome-electron-boilerplate.

It's open source so feel free to contribute and also don't forget to star it 😉.

Conclusion:

To conclude, we were able to:

Setup gridsome
Setup electron
Work with electron and gridsome in development mode
Overcome some challenges that usually occur when gridsome is in production mode with the help of wait-on package and some shell scripts.

If you are reading this then I'm sure you know that you are awesome and if you have any questions let me know in the comments or contact me: contact@divinehycenth.com

I hope you find this helpful.

Happy codding 💻 🙂

Graphql CRUD operations on a CouchDB database with nodejs.

Divine Hycenth — Mon, 13 Jul 2020 17:48:11 +0000

Learn how to perform basic CRUD (Create, Read, Update, and Delete) operations on a Couchdb database through a Graphql API.

Prerequisites

Basic knowledge of typescript & javascriptt
Basic knowledge of graphql
Good understanding of Nodejs
Knowledge of couchdb is an advantage
You must have Nodejs installed on your computer
You must have couchdb installed on your computer https://couchdb.apache.org/

CouchDB

CouchDB falls under the document-oriented database in the NoSQL landscape and it is known for its ease of use and scalable architecture. It is highly available and partition tolerant but is also eventually consistent. Thus, it is an AP based system according to CAP (Consistency, Availability, and Partition Tolerance) theorem for distribute database systems.

Note: This write-up doesn’t discuss the suitability of the CouchDB to various use-cases, rather emphasizes on connecting to the database and interacting with it by performing CRUD operations through a Graphql API.

Architecture

CouchDB organizes data into multiple databases. A database is a collection of documents, and each document is a JSON object. As CouchDB stores data in the form of JSON documents, it is schema-less and highly flexible.

Each document in a database contains a bare minimum of two fields: _id which represents unique identity of the document, and _rev which represents the revision number of the document. If the document posted while document creation doesn’t have _id attribute, CouchDB generates one and saves the document. On the other hand, _rev is used to resolve document update conflict. If two clients tries to update the same document, the first update wins and the second one has to get the update from first client before it’s update.

Creating a database

CouchDB installation come up with a web administration console and can be accessible from http://localhost:5984/_utils. This page lists all the databases available in the running CouchDB instance.

Click of the Create Database to manually create a database.

Nano is a greate tool for communicating with our CouchDB database, However, It has one of the most terrible documentaions expecially when it commes to Typescript.

Without further ado, Lets get into the coding part 🚀.

The first step is to install all the necessary dependencies. I'll be using Yarn throughout this tutorial.

yarn add graphql-yoga nano dotenv;

graphql-yoga: Fully-featured GraphQL Server with focus on easy setup, performance & great developer experience
dotenv: Dotenv is a zero-dependency module that loads environment variables from a .env file into process.env.
nano: Official Apache CouchDB library for Node.js.

Dev Dependencies

yarn add -D typescript ts-node @types/node nodemon

Our project structure

└── src
    └── dbConnection
        └── couch.ts
    ├── index.js
    ├── resolvers.js
    └── typeDefs.js
├── package.json
└── .env

Code walkthrough

This section describes the application code in a bottom-up fashion.

1: Setting up Graphql Server

Your src/index.ts should look like:

require("dotenv").config(); // For environment variables
import { GraphQLServer } from "graphql-yoga";
import { Server } from "http";
import { Server as HTTPSServer } from "https";

import typeDefs from "./typeDefs"; // we are going to create this in a minute
import resolvers from "./resolvers"; // we are going to create this in a minute

export default (async (): Promise<Server | HTTPSServer> => {
  const server = new GraphQLServer({
    typeDefs,
    resolvers,
  });

  const port = process.env.PORT || 4000;
  return await server.start(
    {
      port,
    },
    () => console.log(`server is running on http://localhost:${port}`)
  );
})();

2: Creating an instance of Nano

Add the following snippet to your ./dbConnection/couch.ts file:

import * as Nano from "nano";

export default (async () => {
  const dbName = "hello";
  const nano = Nano(process.env.DB_HOST_AUTH); // I'll be storing the database connection uri
  // in an environment variable since it contains vital credentials.

  const dbList = await nano.db.list(); // Returns a list of database

  try {
    if (!dbList.includes(dbName)) {
      // create a new DB if database doesn't exist.
      await nano.db.create(dbName);
      const db = nano.use(dbName);
      console.log("database created successfully");
      return db;
    } else {
      const db = nano.use(dbName);
      console.log("connected to database successfully");
      return db;
    }
  } catch (err) {
    throw new Error(err);
  }
})();

The above code snippet first retrieves all the database names in our couchDB then checks if it includes a the database we want to use and then uses it with the nano.use(dbName) function. If it doesn't include the our database name we want to use then it will automatically create a new database with the given name.

Nano(process.env.DB_HOST_AUTH) receives a connection string which varies depending on wether we require authentication or not.

http://username:password@localhost:5984 includes credentials thus stored in the .env file as DB_HOST_AUTH=http://username:password@localhost:5984
http://localhost:5984 does not include any credentials and can be used directly.

3: Graphql Type Definitions

Add the following code to your src/typeDefs.ts file:

export default `
  type Doc {
    name: String!
    email: String!
    age: Int!
    nice: Boolean!
    updated: Boolean
  }

   type Mutation {
    createRecord(name: String!, email: String!, age: Int!, nice: Boolean!): Boolean!
    delete(id: String, rev: String): Boolean!
    update(id: String, rev: String, updated: Boolean): Boolean!
  }

  type Query {
    findAll: [Doc!]
    findSingle(id: String!): Doc!
  }
`;

I won't go in detail on how graphql works like in Mutations and Queries as this is not a Beginners Graphql guide.

4: Resolvers.

Resolvers are per field functions that are given a parent object, arguments, and the execution context, and are responsible for returning a result for that field. Resolvers cannot be included in the GraphQL schema language, so they must be added separately. The collection of resolvers is called the "resolver map". It mostly consist of Queries and Mutations.

Mutations -

4a: Creating a record - nano.insert().

The first operation in CRUD is Create. nano.insert() is used to both insert and update the document. This function takes either an object or a string as an argument and inserts/updates the document given.

import { MaybeDocument } from "nano";
import couch from "./dbConnection/couch";

// Lets define the interfaces for each resolver.
interface User extends MaybeDocument {
  name: string;
  email: string;
  age: number;
  nice: boolean;
}

interface Update extends MaybeDocument {
  updated: boolean;
  id: string;
  rev: string;
}

export default {
  Mutation: {
    createRecord: async (_parent: any, args: User) => {
      try {
        const record = await (await couch).insert(args);

        console.log(record);
        return true;
      } catch (err) {
        console.log(err);
        return false;
      }
    },
  },
};

4b: Update a record - nano.insert(id, rev).

As stated earlier, nano.insert() is used to both insert and update the document. When this function has given a document with both _id and _rev, this function performs an update. If the _rev given in the document is obsolete, update fails and the client is expected to get the latest revision of the document before performing any further updates

Below code demonstrates retrieving a blog by it’s id.

...

export default {
  Mutation: {
    update: async (_: any, { id, rev, ...args }: Update) => {
        const findFile = await (await couch).get(id);
        if (findFile) {
            const file = await (await couch).insert({
                _id: id,
                _rev: rev,
                ...findFile,
                ...args,
            });
            console.log(file);
            return true;
        }
        return false;
    },
    ...
  },
};

4c: Delete record - nano.destroy(id, rev).

nano.destroy(id, rev, [callback]) is used delete a document from database. Underneath method deletes a blog entry given it’s _id and _rev

The Nano delete function requires a document _id and a _rev

Below code demonstrates deletion of a record by its id and rev.

...

export default {
  Mutation: {
    delete: async (_: any, { id, rev }: { id: string; rev: string }) => {
        const record = await (await couch).destroy(id, rev);
        console.log(record);
        return true;
    },
    ...
  },
};

4d 1: Retrieve a record by id - nano.get(id).

nano.get(id, [params], [callback]) is used to get the document by its id. Underneath method in BlogService class gets the blog given its id.

Below code demonstrates retrieving a document by it’s id.

...

export default {
  Query: {
    findSingle: async (_: any, { id }: { id: string }) => {
        const file = await (await couch).get(id);
        console.log(file);
        return file;
    },
    ...
  },
};

4d 2: Retrieve multiple files - nano.find(selector).

nano.find(selector, [callback]) performs a "Mango" query by supplying a JavaScript object containing a selector:
the fields option can be used to retrieve specific fields.

Below code demonstrates how to retrieve documents from couchdb.

...

export default {
  Query: {
    findAll: async () => {
        const files = await (await couch).find({
            selector: {}, // parameters can be added to query specific documents.
            fields: ['name', 'email', 'age', 'nice', 'updated'],
        });
        console.log(files.docs);
        return files.docs;
    },
    ...
  },
};

Your final resolvers.ts file should not be different from the below code:

import { MaybeDocument } from "nano";

import couch from "./dbConnection/couch";

interface User extends MaybeDocument {
  name: string;
  email: string;
  age: number;
  nice: boolean;
}

interface Update extends MaybeDocument {
  updated: boolean;
  id: string;
  rev: string;
}

export default {
  Mutation: {
    createRecord: async (_parent: any, args: User) => {
      try {
        const record = await (await couch).insert(args);

        console.log(record);
        return true;
      } catch (err) {
        console.log(err);
        return false;
      }
    },

    delete: async (_: any, { id, rev }: { id: string; rev: string }) => {
      const record = await (await couch).destroy(id, rev);
      console.log(record);
      return true;
    },

    update: async (_: any, { id, rev, ...args }: Update) => {
      const findFile = await (await couch).get(id);
      if (findFile) {
        const file = await (await couch).insert({
          _id: id,
          _rev: rev,
          ...findFile,
          ...args,
        });
        console.log(file);
        return true;
      }
      return false;
    },
  },
  Query: {
    findAll: async () => {
      const files = await (await couch).find({
        selector: {},
        fields: ["name", "email", "age", "nice", "updated"],
      });
      console.log(files.docs);
      return files.docs;
    },

    findSingle: async (_: any, { id }: { id: string }) => {
      const file = await (await couch).get(id);
      console.log(file);
      return file;
    },
  },
};

You can find the entire code for this article on my github repo https://github.com/DNature/couchdb-graphql

Conclusion:

To conclude, the blog has discussed CouchDB basics, and explained how to perfrom CRUD operations on a CouchDB database using Node, Graphql and Nano.

I hope you find this helpful.

Happy codding 💻 🙂

Web Scraping — Scrape data from your instagram page with Nodejs, Playwright and Firebase.

Divine Hycenth — Wed, 24 Jun 2020 16:13:42 +0000

An introduction to web scraping with playwright, nodejs and firebase.

Prerequisites

If you want to follow along this tutorial, you'll need the following:

Basic Knowledge of Firebase and a Firebase account https://firebase.google.com/
Basic knowledge of javascript
A coding Editor Vscode Preferred
API Development/Debugging tool.

What is web scrapping?

Web scrapping refers to the extraction of data from a website. This information
is collected and exported into a format (i.e csv) that is more useful to the
user.

What is a Headless Browser?

You might have heard of the term Headless Browser but still don't know what it
means. You don't have to worry because the Internet's got our back 🙂

Headless browsers provide automated control of a web page in an environment
similar to popular web browsers, but are executed via a command-line interface
or using network communication.
wikipedia.

Here are few most popular Headless Browsers 👇

Puppeteer: Puppeteer is a
Node library which provides a high-level API to control headless Chrome or
Chromium over the DevTools Protocol. It can also be configured to use full
(non-headless) Chrome or Chromium.

Playwright: Playwright is a Node library developed by
microsoft to automate Chromium, Firefox and WebKit with a single API. Playwright
is built to enable cross-browser web automation that is ever-green, capable,
reliable and fast.

Initial Setup 🚀

Lets start off by initializing firebase cloud functions for javascript:

firebase init functions

cd functions

npm install playwright

This installs Playwright and browser binaries for Chromium, Firefox and WebKit.
Once installed, you can require Playwright in a Node.js script and automate web
browser interactions.

Now lets create our Instagram Scraper

Instagram on the web uses React, which means we won’t see any dynamic content
util the page is fully loaded. Playwright is available in the Clould Functions
runtime, allowing you to spin up a Chrome/Firefox/Webkit browser on your server.
It will render JavaScript and handle events just like the browser you’re using
right now.

First, the function logs into a real instagram account. The page.type method
will find the cooresponding DOM element and type characters into it. Once logged
in, we navigate to a specific username and wait for the img tags to render on
the screen, then scrape the src attribute from them.

const functions = require('firebase-functions');
const playwright = require('playwright');

exports.scrapeImages = functions.https.onRequest(async (req, res) => {
  // Randomly select a browser
  // You can also specify a single browser that you prefer
  for (const browserType of ['firefox', 'chromium', 'webkit']) {
    console.log(browserType); // To know the chosen one 😁
    const browser = await playwright[browserType].launch();
    const context = await browser.newContext();
    const page = await context.newPage();
    await page.goto('https://www.instagram.com/accounts/login/');

    await page.waitForSelector('[type=submit]', {
      state: 'visible',
    });
    // You can also take screenshots of pages
    await page.screenshot({
      path: `ig-sign-in.png`,
    });
    await page.type('[name=username]', '<your-username>'); // ->
    await page.type('[type="password"]', '<your-password>'); // ->

    await page.click('[type=submit]');
    await page.waitForSelector('[placeholder=Search]', { state: 'visible' });
    await page.goto(`https://www.instagram.com/<your-username>`); // ->
    await page.waitForSelector('img', {
      state: 'visible',
    });
    await page.screenshot({ path: `profile.png` });
    // Execute code in the DOM
    const data = await page.evaluate(() => {
      const images = document.querySelectorAll('img');
      const urls = Array.from(images).map((v) => v.src);
      return urls;
    });
    await browser.close();
    console.log(data);
    // Return the data in form of json
    return res.status(200).json(data);
  }
});

Replace highlighted fields with valid credentials.

Now we need to test out our API and to do that we're going to need an API
debugging tool and I'd recommend Insomnia because it's
the best API tool i've ever used and it has tons of features. You can also use
tools like Postman.

start the dev server by running:

npm run serve

Open Insomnia and send a post request to the generated url from your firebase
dev server.

Example http://localhost:5001/playwright-faecb/us-central1/scrapeImages

In the above image, you can see the response JSON data colored with yellow on
the right side. It is an array of urls that points to individual images on your
Instagram page.

Let's reflect back step by step to better understand what we're doing.

Playwright provides us with three different browser types so why not try three
of them at the same time.

We loop through the three browser types ['chromium', 'firefox', 'webkit'].
launch()
We launch a new browser with the launch method.
newContext()
Creates a new browser context. A
Browser context
provides us with most operations like creating a new tab in the browser.
goto
method navigates to a specified URL path.
waitForSelector()
waits for an element to either be or not be present in the browser.
screeshot()
takes a screenshot of the current page.
text()
provides us with the ability to fill out form fields.
click()
allows us to click on an element or page in the dom.
evaluate().
You can do a lot in the callback of this function; However, we're just
returning the src value of each image in the callback.
browser.close()
destroys our current browser as you already know.

You can do whatever you want with the data (Download the images) but in our case
we are just logging it out on the console and returning it as a JSON string with
a 200 response to see what it looks like.

Conclusion

Most times Bad dudes applies this technique to Illegally extract data from a
website and I'm pretty sure that the person reading this is not one of them.

Always remember to Use your code for good 🙂

Complete code: https://github.com/dnature/playwright-example

I hope you find this helpful.

Happy codding 💻 🙂.

You might as well check my blog for other tech related articles. https://divinehycenth.com/blog

File uploads with React and apollo (Part 2).

Divine Hycenth — Sun, 17 May 2020 17:59:45 +0000

A complete guide on how to upload files to graphql server with react and apollo-upload-client.

Bonus: You will also learn how to serve files from your apollo server with express.

Prerequisites:

Knowledge of React
Basic knowledge of Apollo
Graphql API with file upload capabilities (Here is a complete guide on how to create Upload files on apollo-server)

Here is a demo of what we are going to build.

Let's get started 🚀

First we are going to use the create-react-app cli to bootstrap a new react project by running:



npx create-react-app react-apollo-upload
    # or
yarn create react-app react-apollo-upload
# Change directory into react-apollo-upload by running
cd react-apollo-upload

Open the project in your favourite editor/IDE. I'll be using vs-code which is my favourite editor.

We are going to install all the required packages for now then i'll explain the function of each package.



npm install graphql graphql-tag apollo-upload-client @apollo/react-hooks apollo-cache-inmemory react-dropzone

Next thing is to setup our react application to be able use apollo-upload-client so we are going to make few changes to our src/index.js to look like:



import React from "react";
import ReactDOM from "react-dom";
import "./index.css";
import App from "./App.jsx";
import ApolloClient from "apollo-client";
import { createUploadLink } from "apollo-upload-client";
import { ApolloProvider } from "@apollo/react-hooks";
import { InMemoryCache } from "apollo-cache-inmemory";
const httpLink = createUploadLink({
  uri: "http://localhost:4000",
});
const client = new ApolloClient({
  link: httpLink,
  cache: new InMemoryCache(),
});
ReactDOM.render(
  <ApolloProvider client={client}>
    <App />
  </ApolloProvider>,
  document.getElementById("root")
);

The traditional apollo-client react application uses apollo-link-http.

Apollo-link-http is a terminating link that fetches GraphQL results from a GraphQL endpoint over an http connection. The http link supports both POST and GET requests with the ability to change the http options on a per query basis. This can be used for authentication, persisted queries, dynamic uris, and other granular updates.

However, apollo-link-http doesn't support file uploads thats we we are going to use apollo-upload-client.

We created our upload link and stored it in a variable called httpLink then we used the link as an option in the ApolloClient option. we also added apollo-cache-inmemory for caching. then we wrap our <App /> component with ApolloProvider and pass in the client prop and now our entire application has access to the apollo client we created.

For the purpose of code readability we are going to split our code into different components and they're going to live in the src/components directory.

Create an upload.jsx file in your src/components and add the following code which i will explain to you in a sec.



import React, { useCallback } from "react";
import { useDropzone } from "react-dropzone";
const FileUpload = () => {
  const onDrop = useCallback((acceptedFiles) => {
    // do something here
    console.log(acceptedFiles);
  }, []);
  const { getRootProps, getInputProps, isDragActive } = useDropzone({
    onDrop,
  });
  return (
    <>
      <div {...getRootProps()} className={`dropzone ${isDragActive && "isActive"}`}>
        <input {...getInputProps()} />
        {isDragActive ? <p>Drop the files here ...</p> : <p>Drag 'n' drop some files here, or click to select files</p>}
      </div>
    </>
  );
};
export default FileUpload;

In the above code, we imported useCallback hook from react and useDropzone hook form react-dropzone. Next we destructured getRootProps, getInputProps, and isDragActive from useDropzone and we passed an onDrop callback as an option.

The useDropzone hook contains a lot of props which you can learn more about in there official github repo https://github.com/react-dropzone/react-dropzone/

Next we spread ...getRootProps() in our wrapper div and ...getInputProps() in the default html input element and react-dropzone will handle the rest for us.

We can perform a lot of operations in the onDrop callback. However, I'm just going to console.log the file for now to see what it looks like.

To test this out we need to import our component into the App.js component so your src/App.js should look like:



import React from "react";
import logo from "./logo.svg";
import "./App.css";
import FileUpload from "./components/upload";
function App() {
  return (
    <div className="App">
      <header className="App-header">
        <img src={logo} className="App-logo" alt="logo" />
        <h1>Upload files effortlessly</h1>
      </header>
      <div className="container">
        <FileUpload />
      </div>
    </div>
  );
}
export default App;

As we can see from the image above we get an array of files from react-dropzone. However, we only care about a single file because our server is currently configured to only accept a single file so we are going to use the first file by accessing its index which is at 0.

We are going to create our mutation and the graphql-tag package we installed enables us to do that.



...
import gql from 'graphql-tag';
const UploadMutation = gql`
  mutation uploadFile($file: Upload!) {
    uploadFile(file: $file) {
      path
      id
      filename
      mimetype
    }
  }
`;
...

First we imported gql from graphql-tag then we create our Upload mutation which has a parameter file (in graphql, variables are written with a dollar sign prefix followed by the name \$file) and its value is a graphql scaler type Upload.



...
// import usemutation hook from @pollo/react-hooks
import { useMutation } from '@apollo/react-hooks';
...
// pass in the UploadMutation mutation we created earlier.
const [uploadFile] = useMutation(UploadMutation);
  const onDrop = useCallback(
    (acceptedFiles) => {
      // select the first file from the Array of files
      const file = acceptedFiles[0];
      // use the uploadFile variable created earlier
      uploadFile({
        // use the variables option so that you can pass in the file we got above
        variables: { file },
        onCompleted: () => {},
      });
    },
    // pass in uploadFile as a dependency
    [uploadFile]
  );
...

Finally, your src/components/upload.js should look like



import React, { useCallback } from "react";
import { useDropzone } from "react-dropzone";
import { useMutation } from "@apollo/react-hooks";
import gql from "graphql-tag";
const UploadMutation = gql`
  mutation uploadFile($file: Upload!) {
    uploadFile(file: $file) {
      path
      id
      filename
      mimetype
    }
  }
`;
// pass in the UploadMutation mutation we created earlier.
const FileUpload = () => {
  const [uploadFile] = useMutation(UploadMutation);
  const onDrop = useCallback(
    (acceptedFiles) => {
      // select the first file from the Array of files
      const file = acceptedFiles[0];
      // use the uploadFile variable created earlier
      uploadFile({
        // use the variables option so that you can pass in the file we got above
        variables: { file },
        onCompleted: () => {},
      });
    },
    // pass in uploadFile as a dependency
    [uploadFile]
  );
  const { getRootProps, getInputProps, isDragActive } = useDropzone({
    onDrop,
  });
  return (
    <>
      <div {...getRootProps()} className={`dropzone ${isDragActive && "isActive"}`}>
        <input {...getInputProps()} />
        {isDragActive ? <p>Drop the files here ...</p> : <p>Drag 'n' drop some files here, or click to select files</p>}
      </div>
    </>
  );
};
export default FileUpload;

And thats all you need to upload files with apollo-upload-client and react. However, You are going to run into issues when trying to display files like images on the client side of your application but don't worry because that's what we are going to work on next.

...

BONUS 🙂

Henceforth, i'm just going to give you a brief walk through on how these code works and you can find the complete source code for both the server and client on github.

...

Server

Now we are going to configure our server to be able to serve static files so we are going to switch from the regular apollo-server to apollo-server-express.

Install express, cors and apollo-server-express by running



npm install cors express apollo-server-express

CORS: Cross-origin resource sharing is a mechanism that allows restricted resources on a web page to be requested from another domain outside the domain from which the first resource was served. A web page may freely embed cross-origin images, stylesheets, scripts, iframes, and videos. Wikipedia

It's funny but i think this image best explains cors:

Add the following piece of code to make your server look like this



import { ApolloServer } from "apollo-server-express"; // switched from apollo-server to apollo-server-express
import typeDefs from "./typeDefs";
import resolvers from "./resolvers";
import express from "express";
import cors from "cors"; // import cors
import path from "path";
const app = express();
// Import your database configuration
import connect from "./db";
export default (async function () {
  try {
    await connect.then(() => {
      console.log("Connected 🚀 To MongoDB Successfully");
    });
    const server = new ApolloServer({
      typeDefs,
      resolvers,
    });
    const dir = path.join(process.cwd(), "images");
    app.use("/images", express.static(dir)); // serve all files in the /images directory
    app.use(cors("*")); // All Cross-origin resource sharing from any network
    server.applyMiddleware({ app }); // apply express as a graphql middleware
    // server.listen(4000, () => {
    app.listen(4000, () => {
      console.log(`🚀 server running @ http://localhost:4000`);
    });
  } catch (err) {
    console.error(err);
  }
})();

Client

We are going to do two things on the client.

Display files from the server,
Create a new upload drop-zone that displays file preview.

Add a proxy that points to your server's domain in your package.json file.



{
  ...
  "proxy": "http://localhost:4000/"
}

Our server no longer uses apollo-server but uses apollo-server-express and the default endpoint of apollo-server-express is /graphql so we need to add that to our createUploadLink uri.

Now your src/index.js should look like this



import React from "react";
import ReactDOM from "react-dom";
import "./index.css";
import App from "./App.jsx";
import ApolloClient from "apollo-client";
import { createUploadLink } from "apollo-upload-client";
import { ApolloProvider } from "@apollo/react-hooks";
import { InMemoryCache } from "apollo-cache-inmemory";
const httpLink = createUploadLink({
  uri: "http://localhost:4000/graphql", // changed
});
const client = new ApolloClient({
  link: httpLink,
  cache: new InMemoryCache(),
});
ReactDOM.render(
  <ApolloProvider client={client}>
    <App />
  </ApolloProvider>,
  document.getElementById("root")
);

Crate a file and name it Uploads.js in your src/components directory then add the following code:



import React from "react";
import { useQuery } from "@apollo/react-hooks"; // import useQuery hook
import gql from "graphql-tag";
// FilesQuery
export const FileQuery = gql`
  {
    files {
      id
      filename
      mimetype
      path
    }
  }
`;
export default function Uploads() {
  const { loading, data } = useQuery(FileQuery); /* useQuery returns and object with **loading, 
   data, and error** but we only care about the loading state and the data object.
   */
  if (loading) {
    // display loading when files are being loaded
    return <h1>Loading...</h1>;
  } else if (!data) {
    return <h1>No images to show</h1>;
  } else {
    return (
      <>
        <h1 className="text-center">Recent uploads</h1>
        {data.files.map((file) => {
          console.log(file);
          return (
            file.mimetype.split("/")[0].includes("image") && (
              <div
                style={{
                  padding: 16,
                  border: "1px solid gray",
                  borderRadius: 5,
                  margin: "16px 0",
                }}
                key={file.filename}
              >
                <img src={"/" + file.path} /* Note the '/'. we added a slash prefix because our file path 
                  comes in this format: images/<filename>.jpg.
                  */ alt={file.filename} style={{ width: "100%" }} />
                <p>{file.filename}</p>
              </div>
            )
          );
        })}
      </>
    );
  }
}

If you have files in your database then you should be able to see them in your browser.

Create a file and name it uploadWithPreview.js in your src/components directory then add the following piece of code



import React, { useEffect, useState } from "react";
import { useDropzone } from "react-dropzone";
import { useMutation } from "@apollo/react-hooks";
import { UploadMutation } from "./upload";
import { FileQuery } from "./Uploads"; // import FileQuery we created in the Uploads.js file
export default function WithPreviews(props) {
  const [file, setFile] = useState({}); // empty state that will be populated with a file object
  const [uploadFile] = useMutation(UploadMutation);
  // submit function
  const handleUpload = async () => {
    if (file) {
      uploadFile({
        variables: { file },
        refetchQueries: [{ query: FileQuery, variables: file }], // update the store after a successful upload.
      });
      setFile({}); // reset state after a successful upload
      console.log("Uploaded successfully: ", file);
    } else {
      console.log("No files to upload");
    }
  };
  const { getRootProps, getInputProps } = useDropzone({
    accept: "image/*",
    onDrop: (acceptedFile) => {
      setFile(
        // convert preview string into a URL
        Object.assign(acceptedFile[0], {
          preview: URL.createObjectURL(acceptedFile[0]),
        })
      );
    },
  });
  const thumbs = (
    <div className="thumb" key={file.name}>
      <div className="thumb-inner">
        <img src={file.preview} className="img" alt={file.length && "img"} />
      </div>
    </div>
  );
  useEffect(
    () => () => {
      URL.revokeObjectURL(file.preview);
    },
    [file]
  );
  return (
    <section className="container">
      <div {...getRootProps({ className: "dropzone" })}>
        <input {...getInputProps()} />
        <p>Drag 'n' drop some file here, or click to select file</p>
      </div>
      <aside className="thumb-container">
        {thumbs}
        <button type="submit" className={`button`} style={{ display: file && !Object.keys(file).length && "none" }} onClick={handleUpload}>
          Upload
        </button>
      </aside>
    </section>
  );
}

Congratulations if you made it to this pont 👏

Handling file upload on both Rest and Graph APIs are a little bit tricky. However, with modern tools we can now upload files with lesser effort.

We learnt how to setup a react application for uploads based on a graphql api.
We also learnt how to configure our backend so that it can serve files to the client.

I hope you find this helpful.

I'm Divine Hycenth and I love writing about things I've learn't. Visit https://divinehycenth.com/blog to see some articles I've written.

Happy codding 💻 🙂

Handling File Uploads with Apollo Server 2.0.

Divine Hycenth — Wed, 22 Apr 2020 14:41:42 +0000

A comprehensive guide on how to upload files with Apollo-server 2.0 and Mongodb.

...

Prerequisites

Altair (Recommended alternative to the default graphql playground)
You must have nodejs installed on your machine.

File Uploads have an interesting history in the Apollo ecosystem.

With Apollo Server 2.0, you can perform file uploads right out of the box. Apollo Server ships with the ability to handle multipart requests that contain file data. This means you can send a mutation to Apollo Server containing a file, pipe it to the filesystem, or pipe it to a cloud storage provider instead.

Depending on your problem domain and your use case, the way you set up file uploads may differ. Handling Multipart upload request in graphql can be a pain in the ass especially when your are coming from a Rest background like me. However, I'm going to show you how to upload files with apollo-server 2.0

One of the simplest ways of achieving file uploads in a single request is to base64-encode a file and send as a string variable in a mutation.

How it works

The upload functionality follows the GraphQL multipart form requests specification. Two parts are needed to make the upload work correctly. The server and the client:

The Client: On the client, file objects are mapped into a mutation and sent to the server in a multipart request.
The Server: The multipart request is received. The server processes it and provides an upload argument to a resolver. In the resolver function, the upload promise resolves an object.

Our project structure

├── images
│   └── 9A1ufNLv-bg-works.jpg
├── package.json
└── src
    ├── db.js
    ├── fileModel.js
    ├── index.js
    ├── resolvers.js
    └── typeDefs.js

Lets Begin 🚀

We will start off by initializing our project with npm, install the necessary packages and configure our server.

npm init -y
yarn add esm apollo-server graphql mongoose shortid
yarn add -D nodemon

I'm going to explain what each package will handle shortly.

The next step is to setup our server with apollo and mongoose. Create a db.js file in your /src directory and add the following configuration code to connect to your mongodb database:

import mongoose from "mongoose";
const MONGO_CONNECTION = "mongodb://localhost:27017/fileUploads";
export default (async function connect() {
  try {
    await mongoose.connect(MONGO_CONNECTION, {
      useNewUrlParser: true,
      useUnifiedTopology: true,
    });
  } catch (err) {
    console.error(err);
  }
})();

Now create an index.js file in your /src directory and paste the following code start off your server on http://localhost:4000

import { ApolloServer } from "apollo-server";
import typeDefs from "./typeDefs";
import resolvers from "./resolvers";
// Import your database configuration
import connect from "./db";
export default (async function () {
  try {
    await connect.then(() => {
      console.log("Connected 🚀 To MongoDB Successfully");
    });
    const server = new ApolloServer({
      typeDefs,
      resolvers,
    });
    server.listen(4000, () => {
      console.log(`🚀 server running @ http://localhost:4000`);
    });
  } catch (err) {
    console.error(err);
  }
})();

Next we'll create our resolvers and typeDefs and put it in a separate file:

// src/typeDefs.js
import { gql } from "apollo-server";
export default gql`
  type Query {
    hello: String
  }
`;

// src/resolvers.js
export default {
  Query: {
    hello: () => "Hello world",
  },
};

Lol 😅 thats just a simple Hello world Query.

Now add a dev script to your package.json file to enable us start up our server.

You may be wondering why we've been using ES6 syntax without configuring babel and thats because of the esm module we installed earlier.

esm is the world’s most advanced ECMAScript module loader. This fast, production ready, zero dependency loader is all you need to support ECMAScript modules in Node 6+.

// package.json
{
  "name": "apollo-upload",
  "main": "src/index.js",
  "scripts": {
    "dev": "nodemon -r esm src/index.js" /* we are requiring the esm module 
    with [-r] flag to transpile our es6 code */
  },
  "dependencies": {
    "apollo-server": "^2.11.0",
    "graphql": "^14.6.0",
    "mongoose": "^5.9.4",
    "esm": "^3.2.25",
    "shortid": "^2.2.15"
  },
  "devDependencies": {
    "nodemon": "^2.0.2"
  }
}

yarn dev
   #or
npm run dev

We can see that out server is running on http://localhost:4000. Let's test our Hello world query in out graphql playground.

For server integrations that support file uploads (e.g. Express, hapi, Koa), Apollo Server enables file uploads by default. To enable file uploads reference the Upload type in the schema passed to the Apollo Server construction.

Now your typeDefs file should look exactly like this:

// src/typeDefs.js
import { gql } from "apollo-server";
export default gql`
  type File {
    id: ID!
    filename: String!
    mimetype: String!
    path: String!
  }
  type Query {
    hello: String
    files: [File!]
  }
  type Mutation {
    uploadFile(file: Upload!): File
  }
`;

Note: When using typeDefs, Apollo Server adds scalar Upload to your schema, so any existing declaration of scalar Upload in the type definitions should be removed. If you create your schema with makeExecutableSchema and pass it to ApolloServer constructor using the schema param, make sure to include scalar Upload.

The server is going to return a rpomise that resolves an object. The object contains the following:

createReadStream: The upload stram manages straming the file(s) to a filesystemor any storage location of your choice.
filename: The original name of the uploaded file(s)
mimetype: The MIME type of the file(s) such as image/jpg, application/json, etc.
encoding: The file encoding i.e UTF-8

Now we are going to create a function that will process our file and pipe it into a directory.

// src/resolvers.js
import shortid from "shortid";
import { createWriteStream, mkdir } from "fs";
import File from "./fileModel";
const storeUpload = async ({ stream, filename, mimetype }) => {
  const id = shortid.generate();
  const path = `images/${id}-${filename}`;
  // (createWriteStream) writes our file to the images directory
  return new Promise((resolve, reject) =>
    stream
      .pipe(createWriteStream(path))
      .on("finish", () => resolve({ id, path, filename, mimetype }))
      .on("error", reject)
  );
};
const processUpload = async (upload) => {
  const { createReadStream, filename, mimetype } = await upload;
  const stream = createReadStream();
  const file = await storeUpload({ stream, filename, mimetype });
  return file;
};
export default {
  Query: {
    hello: () => "Hello world",
  },
  Mutation: {
    uploadFile: async (_, { file }) => {
      // Creates an images folder in the root directory
      mkdir("images", { recursive: true }, (err) => {
        if (err) throw err;
      });
      // Process upload
      const upload = await processUpload(file);
      return upload;
    },
  },
};

For the demo below i'm going to use Altair which is a graphql playground and it's very efficient for file uploads.

Click here to view Demo

Saving to database(mongodb)

We used file system to handle our file uploads because of the following reasons:

Performance can be better than when you do it in a database. To justify this, if you store large files in DB, then it may slow down the performance because a simple query to retrieve the list of files or filename will also load the file data if you used Select * in your query. In a file system, accessing a file is quite simple and light weight.
Saving the files and downloading them in the file system is much simpler than it is in a database since a simple "Save As" function will help you out. Downloading can be done by addressing a URL with the location of the saved file.
Migrating the data is an easy process. You can just copy and paste the folder to your desired destination while ensuring that write permissions are provided to your destination. ...Read more

In the future i'm going to show you how to query the files from our images directory through the file path specified in the database.

We are going to create our database schema and save it in a src/fileModel.js file.

Your code should look like:

// src/fileModel.js
import { Schema, model } from "mongoose";
const fileSchema = new Schema({
  filename: String,
  mimetype: String,
  path: String,
});
export default model("File", fileSchema);

Next step is to make use our file schema.

Your src/resolvers.js code should look like this:

// src/resolvers.js

import shortid from "shortid";
import { createWriteStream, mkdir } from "fs";
// import our model
import File from "./fileModel";
const storeUpload = async ({ stream, filename, mimetype }) => {
  const id = shortid.generate();
  const path = `images/${id}-${filename}`;
  return new Promise((resolve, reject) =>
    stream
      .pipe(createWriteStream(path))
      .on("finish", () => resolve({ id, path, filename, mimetype }))
      .on("error", reject)
  );
};
const processUpload = async (upload) => {
  const { createReadStream, filename, mimetype } = await upload;
  const stream = createReadStream();
  const file = await storeUpload({ stream, filename, mimetype });
  return file;
};
export default {
  Query: {
    hello: () => "Hello world",
  },
  Mutation: {
    uploadFile: async (_, { file }) => {
      mkdir("images", { recursive: true }, (err) => {
        if (err) throw err;
      });
      const upload = await processUpload(file);
      // save our file to the mongodb
      await File.create(upload);
      return upload;
    },
  },
};

Complete code https://github.com/DNature/apollo-upload

Now you now understand how file uploads work in Apollo server 2.0. I hope to see you next time 😀.

You can also check out the part 2 where you will learn how to upload files to this server we've built

Checkout some blog posts i've written on my website

Forem: Divine Hycenth

Front-end Engineer in 2021

Technical requirements for a front-end engineer in 2021.

Table of content

Programming Language.

Frontend Framework

CSS Frameworks

Git and Git workflow.

UI Libraries (React and Vue).

React.

Vue.js

Design Tools

Static Site Generators

SEO and Accessibility.

JAMstack

Client-side validation.

PWA

Testing

Graphql 🔥

Animations

Other things you should know to level up your game.

What do you think is missing in this article? Let me know in the comments.

Convert a Base64 data into an Image in Node.js

Learn how to convert an image into a base64 string and back to an image.

Bonus:

Base64 Encoding

Base64 To Image

Bonus

Conclusion

Build a Desktop app with Gridsome and Electron.

Learn how to combine your knowledge of Gridsome for SPA and Electron for desktop apps to build a desktop app.

Prerequisites

Project structure

What is Gridsome?

Setting up Gridsome.

Installation.

Electron

What is Electron?

Conclusion:

Happy codding 💻 🙂

Graphql CRUD operations on a CouchDB database with nodejs.

Learn how to perform basic CRUD (Create, Read, Update, and Delete) operations on a Couchdb database through a Graphql API.

Prerequisites

CouchDB

Architecture

Creating a database

Our project structure

Code walkthrough

1: Setting up Graphql Server

2: Creating an instance of Nano

3: Graphql Type Definitions

4: Resolvers.

Mutations -

Conclusion:

Happy codding 💻 🙂

Web Scraping — Scrape data from your instagram page with Nodejs, Playwright and Firebase.

An introduction to web scraping with playwright, nodejs and firebase.

Prerequisites

What is web scrapping?

What is a Headless Browser?

Initial Setup 🚀

Now lets create our Instagram Scraper

Let's reflect back step by step to better understand what we're doing.

Conclusion

File uploads with React and apollo (Part 2).

A complete guide on how to upload files to graphql server with react and apollo-upload-client.

Bonus: You will also learn how to serve files from your apollo server with express.

Prerequisites:

Let's get started 🚀

BONUS 🙂

Server

Client

Happy codding 💻 🙂

Handling File Uploads with Apollo Server 2.0.

A comprehensive guide on how to upload files with Apollo-server 2.0 and Mongodb.

How it works

Our project structure

Lets Begin 🚀

Saving to database(mongodb)

Happy Codding 🤓