Forem: Camille Gabrieli

Set up an Express development environment

Camille Gabrieli — Thu, 24 Sep 2020 13:43:39 +0000

I'm learning NodeJS following Odin's Express course and write here my notes. Today we're setting up the environment to use Express on macOS.

Overview

We'll need to install:

Node and npm globally. I recommend you to use nvm, it will help you manage multiple versions of node on your machine and updating them is needed. For the node version, uses the LTS which is the more stable, or the current with newer features. Npm is installed alongside with node.
Express locally for each app, using npm.
Any other libraries like template engines, database drivers, authentication middleware, middleware to serve static files... using npm. I won't talk about them here since they really depends of your need.

The Express Application Generator is a handy tool to create a modular skeleton Express app following the MVC pattern. We'll use it here.

Express Application Generator

After installing npm (with node, using nvm), we can use it to install the Express Application Generator.

We install it globaly (-g flag) because it won't be a dependency of our application. Instead, it's run once to create "skeletons" of apps. Run this command:

npm install express-generator -g

Then cd where you want to create your app and run:

express helloNode

This should create a folder named "helloNode" with the base express app. express -h outputs available options, like installing a template library.

Then enter the folder and install the dependencies:

cd helloNode
npm install

Now ready to go ! Let's run the app with the DEBUG environment variable set to the name of your app:

DEBUG=helloworld:* npm start

Our app is running at http://127.0.0.1:3000/ !

Now our bare app is running, we'll probably need to install others libraries to work with databases, authentication and others middlewares. Then we'll want to prepare the environment on the server to run the app in production.

Express 101

Camille Gabrieli — Wed, 23 Sep 2020 15:35:17 +0000

I'm learning NodeJS following Odin's Express course and write here my notes. Each courses on Odin have a Learning Outcomes section for self evaluation, today I'll use it here as a template:

Describe Express and Node’s main benefits.

Node is a great choice for web development because:

It's optimized for network speed and scalability because it's event-driven.
The ecosystem (with npm) is big and the community active.
As it's plain JS, they're easy to write and maintain with the front end (in JS too). Also, the popularity of JS brought many others languages to compile in it: TypeScript, ClojureScript, coffeeScript, Scala, LiveScript...
Node is also portable in Windows, macOS, Linux, Solaris, FreeBSD, OpenBSD, WebOS and NonStopOS.

Express provide tools and helpers for common tasks on web servers:

Handlers for HTTP routes and verbs
Integrate with view rendering engines
Set common webapp settings: ports, location for templates.
custom Middlewares
Lots of middleware packages: cookies, sessions, login, url params...

Describe the relationship between Node and Express.

Express is a node framework: it provides functions to do common tasks. However, it's unopinionated: more flexible but harder to set up.

Explain what a module is and how Express fits in.

A module is a bit of code (usually a file or folder) that we can import in our code to add functionalities. Express is a module, so are express middlewares.

Import and create modules.

Import a module with require: var importedModule = require('./path/to/module'). Create a module with modules.exports = {exportedKey: 'value} or exports.exportedKey = 'value'.

Describe asynchronous APIs.

Asynchronous APIs handle actions that takes time without blocking the thread: the action run in the background and trigger a "callback" when it's resolved. There's various way to handle them: callback, promises, generators, async/await... The more important is to keep code simple, not nested and split up in small modules. Be carefully of callback hell!

A convention for node and Express is to use Error First Callbacks.

Describe and create route handlers.

We use handlers to trigger actions when a route is reach by a user. Usually, the action is to respond with a page! There's many handler methods to respond to different cases: checkout(), copy(), delete(), get(), head(), lock(), merge(), mkactivity(), mkcol(), move(), m-search(), notify(), options(), patch(), post(), purge(), put(), report(), search(), subscribe(), trace(), unlock(), unsubscribe(). They mostly use te same arguments: a pattern to match the route (ex: /about), and a callback function that takes as arguments the request and the response. Uses one of the response methods to return:

// code/02-route-handler.js

app.get('/about', function (req, res) {
  res.send('about page');
});

app.all is usually used for middlewares.

We can group handlers to organize the code. Do so with app.use() and router.

Describe and use middleware.

A Middleware is a method executing before or after handlers. They are frequent in Express and helps with many tasks. The order the middlewares executes is up to the developer. It must call next() (third argument of the function) to pass to the next middleware or handler, or end the request. next('router') to pass controll out of the router instance.

Add a middleware the same way as route handlers: app.use('/', a_middleware, another_middleware).

We can find express middlewares on the official doc or on NPM. We also may write our own middleware. The express doc has pages about using and writing them.

Describe error handling in Express.

Express comes with a default build-in error middleware: any error occurring in synchronous code will be catch. For async functions, we can pass the error object to next(). Then the error stack will be sent to the client (if we're not in production env).

Customs error middleware must take four arguments to preserve signature and get the error object as first parameter:

// code/02-handle-errors.js

app.use(function (err, req, res, next) {
  console.error(err.stack);
  res.status(500).send('Something wrong!');
});

Custom handlers must be placed at the end of the middleware list, after all other app.use() or route handlers to be sure to get all potential errors.

Express error handling page

Describe what the main parts of an Express app might look like.

It really depends on the purpose of our app. Modules may be:

settings: network options, middlewares...
routes handlers
database write/access
[views engines](https://expressjs.com/en/resources/template-engines.html() (templates)

Conclusion

Express seems very solid and well documented. It's flexibility makes it a good choice for most off the apps. Many fonctions have to be added by third party modules (databases, views engines...), and it can be a time-consuming to make the good choice. Fortunately there's plentiful of articles and docs about them because of express popularity.

How we reached react devs among our users

Camille Gabrieli — Fri, 18 Sep 2020 13:29:01 +0000

tldr: devtools globals variables let you guess if your user has a React debugger installed

Last year my team was striving to find React developers to join us and happily translate fancy product team ideas into JavaScript. With a few million happy users per week, some of our users may be a good fit for our team! Did they know we hire? Probably not! Our career page was accessible by a tiny link between press and affiliation at the bottom of the unlogged home page. And indeed our product was so captivating that they were probably too busy using the app to think about career aspirations! Still we didn't want show a career message to everyone as it would be irrelevant for most of them.

🧑‍💻 Method

While performing delightful debugging session, Nicolas and myself came with this idea: as React devtools plugin exposes a global variable, we can use it to know if a user has installed it and toggle a message !

When the page loads, the devtools sets a global named __REACT_DEVTOOLS_GLOBAL_HOOK__, then React communicates with that hook during initialization.

Easy ! Let's try it:

// code/hello-react-dev.js

if (!!window.__REACT_DEVTOOLS_GLOBAL_HOOK__) {
  console.log('Hello React Dev!');
}

We can also detect any other dev plugin that exposes some variables. We came with __REDUX_DEVTOOLS_EXTENSION__ and __VUE_DEVTOOLS_GLOBAL_HOOK__, let me know if you know more!

We also need a way to close the message and persist the information to avoid displaying it again. Let's wrap this in a component:

// code/dev-message.jsx

import React from 'react';
import { useCookies } from 'react-cookie';

export const DevMessage = () => {
  // Store if user has already closed the message
  const [cookies, setCookie] = useCookies();
  const cookieName = 'hasClosedDevMessage';
  const hasClosedDevMessage = cookies[cookieName] === 'true';
  const handleClose = () => setCookie(cookieName, true);

  // Check what plugins are installed
  const userPluginNames = [
    ['react', !!window.__REACT_DEVTOOLS_GLOBAL_HOOK__],
    ['redux', !!window.__REDUX_DEVTOOLS_EXTENSION__],
    ['vue', !!window.__VUE_DEVTOOLS_GLOBAL_HOOK__],
  ]
    .filter(([name, value]) => value)
    .map(([name]) => name)
    .join(', ');

  // Only display the message if user has a plugins installed
  // and if s.he didn't close the message yet
  if (hasClosedDevMessage) return 'reset your cookies to see the message';
  if (!userPluginNames) return 'install a plugin to see the message';
  return [
    <h1>Hello {userPluginNames} Dev !</h1>,
    <a href="https://youtu.be/dQw4w9WgXcQ" target="_blank">
      interested ?
    </a>,
    <button onClick={handleClose}>no thanks</button>,
  ];
};

And that's it ! You can try it live here

🕵️ Others techniques

With React, Redux and Vue plugins we target a precise but narrow field of developers. Others methods may be inspired by fingerprinting techniques :

Detect available fonts used by developer: if you have FiraCode or Jetbrains Mono installed on your machine, there's a good chance it's for your IDE!
Uber "Code on the Road" seems to only use you location to propose you a test during you ride.
Google uses it's own data -your search history- to propose you to pass the foo.bar challenge.
You may look for for Github, Stack Overflow, docs references... in the navigation history. Not sure if that's legal or not, I wouldn't recommend you to do that.
Last but not least: console.log messages. I like the fact that it's logged for everyone but only the curious will see it.

🕶️ Privacy concern

As long as the cookie is anonymous and stays on the user machine it seems fine: looking for someone's global variables is quite similar to approaching him/her in the street to compliment his/her cool webpack t-shirt.

🥳 Did it work ?

Kind of: with around 1.5 click/day the exposure seems sufficient to keep the experiment going. And we did receive a candidacy from it!

The approach I shared here with you is fairly simple. However I think it's always nice to think about new original ways to take advantage of the tools we use every day. Hope it gave you some inspiration!

Create a simple Node server with http, url and fs

Camille Gabrieli — Tue, 15 Sep 2020 16:32:44 +0000

Minimal server

Continuing my journey through the Odin Project, I go along w3school NodeJS introduction. Here's a minimal server that return "Hello Node!" to localhost:8080:

// code/01-minimal-server.js

var http = require('http');

http
  .createServer(function (req, res) {
    res.writeHead(200, { 'Content-Type': 'text/html' });
    res.end('Hello Node!');
  })
  .listen(8080);

Modules

We can also create our own modules using the exports keyword:

// code/01-exports-module.js

exports.myDateTime = function () {
  return Date();
};

The FileStystem module fs includes methods to work with files:

Read a file on the system: fs.readFile('file.html', (err, data) => {...});
Create a file with appendFile, open (with w flag) or writeFile
Update a file with appendFile or writeFile
Delete a file with unlink.
Rename a file with rename.

The url module helps to retrieve url's parts:

// code/01-url-module.js

var url = require('url');
var adr = 'http://localhost:8080/default.html?year=2017&month=february';
var q = url.parse(adr, true);

console.log(q.host); //returns 'localhost:8080'
console.log(q.pathname); //returns '/default.html'
console.log(q.search); //returns '?year=2017&month=february'

var qdata = q.query; //returns an object: { year: 2017, month: 'february' }
console.log(qdata.month); //returns 'february'

We may use the modules http, url and fs to serve specific files depending on path the user navigates to.

We can use npm the same way as in front end: npm i module-name and then use is with var module = require('module-name'). Here's two useful modules:

[formidale][https://www.npmjs.com/package/formidable] to parse file upload
[nodemailer][https://www.npmjs.com/package/nodemailer] to send emails

Events

Node is perfect for event-driven application ! Every action is an event, like a when a connection is made, or a file opened. Some objects can fire events, like readStream that listen for opening or closing a file: fs.createReadStream('./file.txt').on('open', () => {...}).

Then module events let us create, fire and listen for our own events:

// code/01-events-module.js

var events = require('events');
var customEvent = new events.EventEmitter();

// handler
var handleNodeLearned = () => console.log('Congrats, you learned Node!');

// listener
customEvent.on('nodeLearned', handleNodeLearned);

// fire events with emit()
customEvent.emit('nodeLearned');

Simple Site

The assignment is to create a simple site with and index, two pages, and a 404-page handler. Here's the server and index page, and you'll find the full code here.

// code/01-simple-site/index.js

var http = require('http');
var url = require('url');
var fs = require('fs');

var writeAndEnd = function (res, data) {
  res.write(data);
  res.end();
};

http
  .createServer((req, res) => {
    var pathname = url.parse(req.url, true).pathname;
    if (pathname === '/') pathname = '/index.html'; // handle root path

    fs.readFile(`.${pathname}`, (err, data) => {
      if (err) {
        res.writeHead(404, { 'Content-Type': 'text/html' });
        fs.readFile('404.html', (err, data) => {
          if (err) {
            writeAndEnd(res, '404 not handled :('); //
          } else {
            writeAndEnd(res, data);
          }
        });
      } else {
        res.writeHead(200, { 'Content-Type': 'text/html' });
        writeAndEnd(res, data);
      }
    });
  })
  .listen(8080);

<!--code/01-simple-site/index.html-->

<!DOCTYPE html>
<html>
  <head>
    <title>Simple Site</title>
  </head>
  <body>
    <h1>Welcome to the Simple Site</h1>
    <p>
      You may navigate to the
      <a href="/about.html">about</a>
      or
      <a href="/contact.html">contact</a>
      pages. You also may try others routes:
    </p>
    <form onsubmit="window.location.href = event.target[0].value; return false">
      <input type="text" />
      <button type="submit">Go !</button>
    </form>
  </body>
</html>

Introduction and background

Camille Gabrieli — Tue, 15 Sep 2020 16:07:43 +0000

Today I started Odin's Node-js course. My company is very enthusiastic about self training and propose a curated list of courses to learn and practice different technologies. I choose NodeJS because as I'm already proficient with JS, I will be able to give more focus on good practice than syntax details.

Here I will log my learnings through it. The incentives to make it public are:

Defeat timidity! I'm not used to publicly posting on the web and doing so is already a big step for me!
Learn by teaching: trying to re-explain what's just digested may benefit the memorization.

Let's dive in 🤿 !

Introduction

NodeJS exist since 2009.
It's a JavaScript runtime: it runs on a local machine, contrary to JavaScript that runs in a browser. in the background I guess Node is running in a virtual machine as JS does in the browser.
It's asynchronous event driven: the lines of code won't necessarily execute in order. Instead, we write functions that gets called by events such as network request or callbacks

Event Driven

Let's say I want an app that tell me if I forgot my umbrella. The steps are:

fetch the weather forecast
if it's sunny, stop
check in my database "backpack" if I already got the umbrella
If I got it, stop
Remind me to take the umbrella

There's two asynchronous process here: fetching the forecast and checking the DB. If I run the steps one by one the program will work. However, we can make it quicker by executing several actions at the same time:

fetch the forecast THEN if it's sunny, stop
check the database THEN if umbrella is there, stop
THEN is it's not sunny, and if I don't have the umbrella, remind me to take it

That way, we stop the program (or pass to the next action) as soon as we got the first response from the forecast or DB. While both actions are running (fetch & query), node waits on an event, here the completion of fetch/query. This process is close to the usage of addEventListener in frontend, but instead of waiting for mouse click, our events will mostly be network requests and DB query.

To handle events, we'll use callbacks.

A callback is a function pass to another function as argument.

Many fonctions uses callback, even map and forEach : [1, 2, 3].forEach(n => console.log(n). Event listeners uses callback the same way: element.addEventListener('click', console.log).

Here's an exemple of callback usage in NodeJS:

http
  .createServer((req, res) => {
    res.writeHead(200, { 'Content-Type': 'text/html' });
    res.end('Hello Node!');
  })
  .listen(8080);

Assignment: Read & Watch

NodeJS website

nodejs.org

Blocking vs not-blocking

During my navigation of the site, I encountered blocking vs non blocking page that triggered my curiosity because it mentions methods whose names end with Sync. That might be in relationship with the "Event Driven" section above, so I kept on reading:

Blocking methods execute synchronously and non-blocking methods execute asynchronously.

const fs = require('fs');
const data = fs.readFileSync('/file.md'); // 1. blocks the process until the file is read
console.log(data); // 2. called after data read
nextMethod(); // 3. called after console.log

fs.readFile('/file.md', (err, data) => {
  // 1. async method, do not block the process !
  if (err) throw err;
  console.log(data); // 3. console when 1. resolved
});
nextMethod(); // 2. called without waiting completion of 1.

Blocking methods execute synchronously and will block the execution until resolved. If there is an error, the error need to be caught, or the process will crash.
Non-Blocking methods execute asynchronously and will not block the execution. Usually a callback can be passed to execute others operation after resolution.

NodeJS is single threaded, therefore concurrent process refers to the event loop. Let's consider a server receiving multiple requests, and the request handler performing timely I/O operation on a database. Using blocking methods would handle the request one by one, delaying resolution of later requests. Using non-blocking methods however will let the server handle others requests while the I/O operation are in progress. This may lead to huge performance difference.

Beware of mixing sync and async methods ! The next exemple will cause errors:

fs.readFile('/file.md', (err, data) => {
  // 1. start reading file.md
  if (err) throw err; // 3. crash ! can't read data as the file has been deleted whild reading it.
  console.log(data);
});
fs.unlinkSync('/file.md'); // 2. delete file.md

Placing non-blocking call inside callback guarantees a safer order of operations:

fs.readFile('/file.md', (err, data) => {
  // 1. start reading file.md
  if (err) throw err;
  console.log(data); // 2. log data
  fs.unlink('/file.md', err => {
    // 3. start removing file
    if (err) throw err; // handle error if necessary
  });
});

MDN server-side introduction

MDN

HTTP

Http is the protocol used to transfer hypertext (linked web documents) between two computers (client & server). It is textual (all commands are plain text ad human readable) and stateless: Neither the client not the server keeps states of the previous exchanges. Only clients can make an HTTP request, and the server can respond to it. Servers must answer every http request, at least with an error message!

Static vs Dynamic

A static site always deliver the same content whenever a ressource is requested. Gatsby's sites are static: they're build and stored in the server, then sent as it is.

Dynamic sites are generated on-demand, usually inserting data from a DB into a template. The server can adapt the content based on user preferences sent in the request or already stored in a DB.

Benefits

Efficient storage and delivery: storing large amount of data like amazon's products or facebook posts will be way more efficient in a database than in individual webpages. A server can access this database and return constructed web pages (html) or data (json). ex: Amazon result's and product's pages.

Customize user experience: the server can store user's information like credit cards detail, search history or habits. ex: google auto-complete uses previous search.

Control access: control who can view, edit, delete... information. This can be the user but also other servers. ex: bank website needs you to be logged to view your account information or edit your password. The bank itself can edit your accounts balance.

Store session/state information: the server can remember the current user of a site and adapt it's response. ex: a newspaper asking you to subscribe after n pages views.

Notification & communication: servers can send emails, sms, push notifications... but also live messaging and video conversations.

Data analysis: we may collect lot of data about our user and use them. ex: facebook news feed or advertising.

Web Frameworks

Framework handles common operations we often need to repeat for apps:

Easily Mapping urls for ressources/pages to handler functions:

from django.conf.urls import url
from . import views

urlpatterns = [
   url(r'^$', views.index),         # example: /best/
   url(r'^junior/$', views.junior), # example: /best/junior/
]

Filling html templates with filtered data from model:

from django.shortcuts import render
from .models import Team

def junior(request):
    list_teams = Team.objects.filter(team_type__exact="junior")
    context = {'list': list_teams}
    return render(request, 'best/index.html', context)

Express is an extremely popular node framework. It is fast, unopinionated, flexible, providing utilities for web apps and http requests. It is minimalist: some functionalities (DB access, user restrictions) might be loaded from libraries or others frameworks based on express.

Deno is a simple, modern and secure JS/TS runtime and framework build with V8 and Rust. It tries to fix some problems with node: security, formating, modules versions.

Freecodecamp introduction to Node

Freecodecamp article and the linked video explaining clearly (!) the event loop:

The speaker uses a tool he creates to display the current state of call stack, web apis, callback queue and render queue: Loupe

This is a more in depth article about event loop and asynchronous execution.

Exemple of application

7 things I can build with NodeJS. Those are all interesting and real projects:

Gif chat room :D
Remote control a car
We play pokemon (as twitch does !)
Remote control computer
Build a hangout-like website
Collaborative drawing tool
Desktop RSS reader

Conclusion

Odin course start with the elementary notions. As I'm self trained and did not have CS academic, my learning sometimes jumped back and forth in complexity. Now I'm very happy to review the ordered basics!

Forem: Camille Gabrieli

Set up an Express development environment

Overview

Express Application Generator

Next

Express 101

Describe Express and Node’s main benefits.

Describe the relationship between Node and Express.

Explain what a module is and how Express fits in.

Import and create modules.

Describe asynchronous APIs.

Describe and create route handlers.

Describe and use middleware.

Describe error handling in Express.

Describe what the main parts of an Express app might look like.

Conclusion

How we reached react devs among our users

🧑‍💻 Method

🕵️ Others techniques

🕶️ Privacy concern

🥳 Did it work ?

Create a simple Node server with http, url and fs

Minimal server

Modules

Events

Simple Site

Introduction and background

Introduction

Event Driven

Assignment: Read & Watch

NodeJS website

Blocking vs not-blocking

MDN server-side introduction

HTTP

Static vs Dynamic

Benefits

Web Frameworks

Freecodecamp introduction to Node

Exemple of application

Conclusion