Forem: Aleksei Zhilyuk

Next.JS CMS — Top choices in 2024

Aleksei Zhilyuk — Thu, 31 Oct 2024 09:38:54 +0000

WHY A HEADLESS CMS FOR YOUR NEXT.JS PROJECT?

A headless CMS serves as a backend-only content repository that provides content via an API, making it displayable on any device. This approach separates content management from its presentation, which is particularly advantageous for NextJS projects. By using a headless CMS, developers can fully leverage Next.js features like server-side rendering and static site generation, optimizing both SEO and performance.

ADVANTAGES OF INTEGRATING A HEADLESS CMS WITH NEXTJS

NextJS is a powerful framework for building efficient and scalable web applications.

Here are specific advantages of integrating a headless CMS with Next.js:

Enhanced Performance: Next.js excels in fast page loads through features like incremental static regeneration and server-side rendering. A headless CMS complements this by efficiently delivering content through APIs, reducing load times and server strain.
Improved SEO: Next.js facilitates SEO-friendly site structures, which can be enhanced with a headless CMS that supports SEO optimization through metadata management and structured content delivery.
Scalability: As your project grows, the need to handle more traffic and content efficiently becomes crucial. A headless CMS can scale independently from the front-end, making it easier to manage large volumes of data without impacting the front-end performance.
Flexibility in Development: Developers can use their preferred front-end technologies and frameworks without constraints imposed by traditional CMS platforms. This means you can use the latest Next.js features and updates as soon as they are released.
Better Security and Stability: By decoupling the CMS from the presentation layer, security is enhanced as potential vulnerabilities in the CMS do not directly expose the front-end to security risks.
Streamlined Workflows: A headless CMS can support multiple front-ends simultaneously, which is advantageous for projects aiming to deliver content across different platforms and devices.

These benefits make a headless CMS an excellent match for Next.js, ensuring developers can build modern, secure, and high-performing applications.

Contentful

Contentful is a leader in the headless CMS market, offering a comprehensive suite of features that cater to developers and content creators alike.

Ease of Integration: With its React SDK, integrating Contentful with Next.js projects is a breeze, providing a seamless workflow for developers.
Flexibility: Contentful's content modeling capabilities are unmatched, allowing for the creation of complex content structures tailored to any project's needs. Its environment feature supports staging and production setups, enabling a robust development workflow.
Performance: Its Delivery API is built for speed, ensuring that content loads quickly, which is crucial for maintaining high performance in Next.js applications.
Community and Support: A vast community and extensive documentation mean that developers can easily find solutions or get help when needed. Contentful also offers professional support services for enterprise users, ensuring that any issues can be swiftly addressed.

Contentful pricing

Free
Basic: $300/month
Premium: Custom pricing

Storyblok

Storyblok stands out for its visual editor and block-based approach to content management, making it particularly friendly for both developers and content creators.

Ease of Integration: Storyblok's component-based approach integrates seamlessly with Next.js, allowing developers to map components in Storyblok to React components.
Flexibility: The platform offers incredible flexibility through its block-based content system, enabling creators to build dynamic, rich content experiences without needing developer intervention for layout changes.
Performance: Storyblok's content delivery is optimized for speed, utilizing a CDN to ensure content is served quickly to Next.js applications globally.
Community and Support: Storyblok has a rapidly growing community and provides extensive documentation, tutorials, and customer support, making it easy for developers to get started and find help when needed.

Storyblok pricing

Community Plan: Free
Entry Plan: €99/month
Enterprise: €3299/month
Enterprise Plus: Custom pricing

Sanity.io

Sanity.io differentiates itself with its real-time content editing experience and a highly customizable content studio.

Ease of Integration: The platform offers excellent support for React and Next.js, including real-time preview capabilities that are invaluable during development.
Flexibility: The customizable content studio and the innovative portable text feature provide developers and content editors with the power to embed rich content structures directly within textual content, making it highly versatile.
Performance: Sanity.io leverages GROQ and GraphQL for data querying, offering efficient and flexible options for fetching data, which enhances app performance.
Community and Support: With a vibrant community and plenty of guides, finding help or inspiration is easy. Sanity.io also hosts regular developer streams and community events, fostering a strong sense of community.

Sanity pricing

Free
Growth: $15/month per user
Enterprise: Custom pricing

Directus

Directus is an open-source option that stands out for its API-driven approach and extensive customization capabilities.

Ease of Integration: Offering both RESTful API and GraphQL endpoint, Directus provides versatile integration options with Next.js, catering to different developer preferences.
Flexibility: Being open-source, Directus allows for deep customization, from the admin UI to the underlying database schema. This flexibility ensures that developers can tailor the CMS to fit the project's exact requirements.
Performance: While performance is generally excellent, it can vary depending on how the Directus backend is configured. However, the ability to directly query the database ensures minimal overhead.
Community and Support: The growing open-source community around Directus is a rich resource for troubleshooting and inspiration. Directus also offers dedicated support plans for businesses requiring more personalized assistance.

Directus pricing

Self-hosted: License required
Professional Cloud: $99/month
Enterprise Cloud: Custom

Payload CMS

Payload CMS is a newer entrant in the headless CMS space but has quickly gained attention for its developer-focused features and flexibility.

Ease of Integration: Designed with developers in mind, Payload CMS offers full TypeScript support and a highly customizable admin UI, making it a perfect match for Next.js applications that utilize TypeScript.
Flexibility: It stands out for its deep customization capabilities, allowing full control over the admin UI and the APIs. This level of customization makes it possible to tailor every aspect of the CMS to the project's needs.
Performance: Payload CMS is optimized for performance, featuring efficient data handling and delivery mechanisms. This focus on performance is evident in the speed and responsiveness of Next.js applications powered by Payload.
Community and Support: Although the community is currently smaller, it's rapidly growing. The developers are actively involved in expanding the documentation and offering support, ensuring users have the resources they need.

Payload CMS pricing

Free
Standard: $35/month
Pro: $199/month
Enterprise: Custom

Prismic

Prismic offers a straightforward and developer-friendly approach to content management, making it a popular choice for many Next.js projects.

Ease of Integration: Its REST API and support for webhooks make integrating Prismic with Next.js straightforward, facilitating both static site generation and server-side rendering.
Flexibility: The slices feature allows for building dynamic page layouts with ease, offering developers and content creators a flexible and powerful tool for designing content-rich pages.
Performance: Prismic's use of a CDN for content delivery ensures fast load times, a critical factor for maintaining high performance in web applications.
Community and Support: While Prismic has good documentation and support, its community may not be as large as some others, but the quality of support and resources is high.

Prismic pricing

Free:
Starter: $10/month
Small: $25/month
Medium: $150/month
Platinum: $675/month
Enterprise: Custom

Ghost CMS

Ghost CMS is an open-source platform focused on blogging and online publishing. Built on Node.js, it is renowned for its simplicity and speed, providing a seamless content creation process. Ghost is ideal for bloggers and organizations seeking a lightweight CMS that enhances SEO and content delivery.

SEO Optimization: Provides SEO-friendly features like automatic sitemap generation and rich meta tagging, complementing Next.js's SEO capabilities.
Modern Technology Stack: Built on a Node.js stack, Ghost aligns well with the server-side rendering features of Next.js.
Lightweight and Fast: Its lightweight nature enhances the performance benefits of Next.js, making it ideal for projects requiring high-speed content delivery.
Membership and Subscriptions: Native support for memberships and subscriptions, enabling monetization in Next.js applications.

Ghost CMS pricing

Starter: from $10/month
Creator: $25/month
Team: $50/month
Business: $199/month

Kentico Kontent

Kontent.ai, formerly known as Kentico Kontent, is a cloud-based CMS designed for content collaboration and delivery across multiple channels. It supports robust content modeling and real-time collaboration, making it suitable for large enterprises and digital agencies managing diverse content ecosystems.

Workflow Management: Offers robust tools for collaborative content creation and workflow management, supporting the development cycles typical in Next.js projects.
Content Modeling: Features powerful content modeling tools to structure content efficiently, which is key when building scalable Next.js applications.
API Capabilities: Comprehensive APIs facilitate efficient data exchanges, crucial for integrating with Next.js's data fetching methods like getStaticProps and getServerSideProps.
Multi-channel Delivery: Ensures consistent content delivery across all platforms, which pairs well with Next.js's universal rendering capabilities.

Hygraph (formerly GraphCMS)

Hygraph is a headless CMS that uses GraphQL to enhance content management and delivery. It allows for the quick development of content APIs with minimal coding, ideal for projects requiring complex content structures and relationships.

GraphQL Native: Being a GraphQL-native CMS, Hygraph provides powerful developer tools that perfectly match Next.js's data fetching strategies, enhancing the development of complex applications.
Schema Management: Easy management of content schemas with an intuitive interface complements Next.js's dynamic routing and static generation features.
Real-time Collaboration: Supports real-time collaboration and version control, which are beneficial for teams working on Next.js applications.
Rich Media Management: Advanced media handling capabilities ensure that multimedia content is optimized for performance in Next.js applications.

Hygraph pricing

Free forever:
Professional: $199/month
Scale: $799/month
Custom: Contact to get a quote

ButterCMS

ButterCMS is a straightforward headless CMS that integrates easily with existing projects thanks to its comprehensive API and SDKs. Known for its ease of use, Butter CMS is favored by marketers and developers needing to deploy content-rich sites quickly.

Marketing Integration: Offers robust tools for SEO and content marketing, enhancing the marketing aspects of Next.js websites.
Simple API: Its simple RESTful API is easy to integrate with Next.js, supporting quick implementation in existing projects.
SDKs and Libraries: Provides SDKs for various programming languages, making it easy to integrate with Next.js and other parts of your technology stack.
Personalization Features: Content personalization capabilities align well with the dynamic and static rendering options of Next.js.

Butter CMS pricing

Micror: $99/month
Startup: $199/month
Small Business: $375/month
Enterprise + Agency: Custom

Strapi

Strapi is an open-source, customizable headless CMS that supports various databases and frontend frameworks. Its API-centric approach and community-driven development make it a flexible choice for building scalable web applications. Strapi appeals to developers looking for control over their CMS environment.

Open Source: One of the few fully open-source options available, giving developers complete control over their CMS.
Self-hosting Capability: Can be self-hosted, providing full control over the hosting environment and security aspects.
Customizable UI: Offers a customizable admin panel built with React, making it a natural choice for React developers.
Community and Plugins: A vibrant community and a wide range of plugins available for extended functionalities.

Strapi pricing

Community: Free
Developer: $29/month
Pro: $99/month
Team: $499/month
Enterprise: Custom

Conclusion and Recommendations

After a detailed analysis of each platform, our top recommendations for Next.js developers based on specific project needs and priorities are:

Sanity.io:

Perfect for projects that require real-time content updates and a high degree of customization in the content editing experience.

Storyblok:

Ideal for those who value intuitive content management with a visual editor and a block-based approach, offering ease of use for developers and content creators alike.

If you are considering a migration to Headless CMS or you are building it from scratch - contact us, we are headless cms agency and we can set up a free consultation around the project you have in mind

Next.js Conf 2024 Highlights and React Framework Future

Aleksei Zhilyuk — Thu, 31 Oct 2024 09:26:32 +0000

While the overall ambitious tone of the previous conference editions have cooled off, we enthusiastically welcomed the direction the presenting team have taken, focusing on making Next.js API more streamlined, and self-hosting friendly. We saw updates and celebrated stable release of Turbopack for Dev, and insightful deep-dives from people behind the framework.

Followed up by case studies of major Next.js adopters, and strategic partners, including, our most technological Headless CMS of choice - Sanity, which went all in with their Next.js 15 support and innovative “live” CMS content approach. We got so excited about the last one that we even wrote a separate blog post breaking down the vision behind the new architecture of choice when it comes to Next.js+Sanity combo.

Let’s break down some of the talks we followed the most:

Opening Keynote

Guillermo Rauch (CEO, Vercel & Creator of Next.js) welcomes everyone to the fifth anniversary of Next.js and begins his speech by expressing gratitude to the main contributors to Next.js.

A lot happened during the year:

4000 PRs
5 Next.js releases were made
570 new contributors joined the project
Next.js reaches 7 million downloads on NPM
Many large companies have already migrated to APP Router, such as PayPal, wayfair, xAI

Guillermo Rauch believes that Next.js and its ecosystem is something we can rely on in the future. Next.js is used all over the world to create everything from the simplest websites to complex, sophisticated applications.

Guillermo Rauch was guided by these principles when creating Next.js

Make it work
Make it right
Make it fast
Make it blazing fast

APP Router was created to make data fetching and rendering more predictable, since according to Guillermo Rauch, developers should not worry about caching and focus on creating the application.

In order for the idea to turn into a product as quickly as possible and the time for each iteration to be as short as possible Guillermo Rauch is pleased to announce that Turbopack is finally stable.

The new compiler is 50% faster than the previous one for initial compilation and 90% faster for Fast Refresh.

Coming soon for Turbopack: Turbopack uses persistent cache for its fast performance.

Simplifying Next.js Cache API

Next, Delba de Oliveira (DX Engineer at Vercel) shows how to make the demo application faster using caching and introduces a new experimental NextJS “use cache” directive that allows you to explicitly specify that a component or the result of a function execution should be cached.

This directive works both in components, similar to “use client”, and in functions, similar to “use server”, as well as a new API “cacheTag” for cache keys and “cacheLife” for specifying how long a given cache should live.

Developers can try these experimental changes in the canary version of Next.js. We will also soon write up our own deep-dive into the new “use cache” API.

Better self-hosting of Next.js

Next, Lee Robinson (VP of Product at Vercel) appears on stage to uncover improvements for self-hosted Next.js they have shipped, re-iterating Vercel’s focus on keeping Next.js open-source first.

In v15, the Vercel team has made it easier to configure how caching works by default, also simplifying the required minimal DevOps setup for self hosting Next.js. Documentation covering various deployment scenarios and templates have also been greatly improved, including the launch of Next.js community GitHub org with up-to-date redeployment recipes for community’s most popular hosting targets.

Previously, Next.js used a web assembly-based image optimization library, but they were dissatisfied with the fact that this library consumed a lot of memory, so now it uses “Sharp”, which is installed automatically.

Default cache control headers have been updated

AMA: Next.js Team

Variety of topics have been covered during the Ask Me Anything session with the Next.js team, we suggest you dive into it yourself on the stream recording.

React Server Components: Elevating speed, interactivity, and user experience

Aurora starts her talk by saying that the RSC have changed the way we build apps.

Some of the new possibilities, that come with the RSC:

Fetch data async inside the component itself
Access backend resources directly from component
No JS is ship to the client
Streaming

RSC and latest next.js features help increase speed of not only websites, but also development velocity. New versions of next.js and react help you split compute load between client and a server, which will help you build an application based on your needs. Another important aspects that have beed improved, are interactivity and ability to easily build responsive apps.

With the new features comes new limitations, for example you can’t access client hooks such as useState, useEffect, or browser information in such components.

New development patterns to leverage the latest features and create a stunning UI:

Move data fetching closer to UI

Instead of fetching data on top level, and make requests blocking render, move data fetching inside the component, and wrap component in Suspence boundary.

Before and after

Add client component and preserve request non-blocking fashion

If you need to make a component inside Suspence boundary client-side, you will be forced to remove server function call from it. In this case, create a promise on the top level, pass it to client component as a prop, and resolve using react new use() function. It will help you avoid render blocking and keep client component in suspense boundary, to show a static shell on initial render.

useTransition, useOptimistic

These new react hooks allows you to create a smooth transitions on any user’s action. Moreover, using these hooks you are able to show instant feedback to the user and mnot depend on network connection.

React cache

Use new cache function to prevent firing the same request multiple times. This means we can reuse existing pattern of calling data directly inside component and maintain composition.

All these new features and patters allow us build fully interactive apps without use of useState and useEffect.

Best development practices:

Resolve promises deep in the tree
Display pending indicators
Put state in the URL

New tools:

Leverage react 19 features:

cache() - perform per-render caching for expensive func calls
useOptimistic() - respond to user interactions instantly even when the request is slow
use() - suspend client components as they resolve a promise passed down

Next.js features:

staleTimes - set state times for dynamic page segments to reuse them across subsequent requests
PPR - statically renders parts of the page or layout to improve performance

The long and winding road: CSR to static export to SSG

Building user interfaces in the age of AI

Oleg starts of by defining what is great interface? Great interface is measured by the ability to help users accomplish their task as quickly and as effortlessly as possible.

The most important things here are speed and reliability.

What is Generative UI?

It is an umbrella for any project leveraging LLMs, in order to enhance user interface. It is a spectrum. Each side of the spectrum has it’s own tradeoffs and requirements. But they are united by the crucial role of LLM.

Both edges of the spectrum has it’s own requirements, tradeoffs and performance characteristics.

The slide represents the current state of LLM’s capabilities for generating UI.

Generating HTML from text is a hard problem with following challenges:

Slow. takes up to 5-7s to generate a HTML page on perplexity
No reliable way to deploy it
Hard to sync with design system
Token inefficiency

The Perplexity team is focused on text to object generations instead.

One of the patterns guys in Perplexity use is define schemas for each presentational component and define props based on schema.

Example:

Then use component’s schema as input for the model. This way you can create simple components with streaming functionality using LLMs.

Key Takeaways:

Use LLMs as APIs
Faster and cheaper
Fits well into existing flow
Easy to integrate
Structured output
React friendly JSON data
Guaranteed to be valid
Schemas are useful without LLMs
Streaming UX
Address re-rendering issues
Use OSS libs
Implementation depends on your code

Optimizing LCP: Partial Prerendering deep dive

Wyatt Johnson

Wyatt Johnson, a software engineer at Vercel, provides a comprehensive overview of “Partial Prerendering”, an experimental feature aimed at optimizing the Largest Contentful Paint (LCP) by combining the best of static and dynamic rendering techniques. This method has been developed to tackle the limitations of traditional rendering methods, improving site performance and user experience.

Wyatt begins by explaining the significance of Core Web Vitals, particularly focusing on LCP, which measures the render time of the largest image or text block visible within the viewport. He highlights the challenges faced with traditional rendering approaches where developers must choose between the speed of static rendering and the flexibility of dynamic rendering. Static rendering, while fast, cannot incorporate request data, leading to delays in rendering dynamic content. Conversely, dynamic rendering incorporates request data but often at the expense of speed due to server response times.

The session delves into the mechanics of Partial Prerendering (PPR), which allows for a static shell of a page to be generated at build time and served from the edge. Simultaneously, it sends a request back to the origin to complete the dynamic rendering. This approach minimizes the time to first byte and ensures that the page loads quickly while still supporting dynamic capabilities.

Wyatt demonstrates practical applications of PPR in an e-commerce setting, showing how PPR can streamline the rendering process, reduce latency, and improve the user experience by delivering a fast initial load with dynamic capabilities intact. He further explains the technical implementation of PPR, discussing how it leverages React’s capabilities to suspend and resume rendering as needed, based on the dynamic content requirements.

To conclude, Wyatt emphasizes the future potential of PPR in Next.js and the ongoing efforts to integrate this feature across different hosting environments. He expresses enthusiasm for the capabilities of Next.js in bridging the gap between static speed and dynamic flexibility, ultimately providing developers with the tools to build faster and more responsive web applications.

That's a Wrap

Thank you for following our highlights from the Next.js Conference 2024, we enjoy sharing these with the community, as we eagerly follow all the developments in the Next.js space.

Follow our blog for more Next.js content and practical use cases that we have solved building a variety of Next.js apps; as well as the practices we developed through our Performance and SEO audits for our clients.

We are excited about the direction Next.js team took, prioritizing API simplicity,increased transparency and support towards non-Vercel clients, which is dearly important for us. We support the renewed open source spirit and simplicity it brings for deployment of Next.js to own cloud environments.

Top React Conferences to Attend in 2024

Aleksei Zhilyuk — Tue, 27 Aug 2024 10:31:42 +0000

The React community continues to grow and evolve, making 2024 an exciting year for developers who want to connect, learn, and share their knowledge. Whether you're an experienced pro or just getting started, attending a React conference is an excellent way to stay up-to-date with the latest trends, tools, and best practices. Below, we've outlined some of the top React conferences coming up in 2024 that you won't want to miss.

For a full list of events, check out the React events on GitNation.

1. React Summit US

Date: November 19 - 22, 2024
Location: New York, USA

React Summit US is one of the largest React gatherings worldwide, bringing together industry leaders to discuss the latest in React technology, state management, performance optimization, and more. Whether you're interested in deep technical dives or broader industry trends, you'll find sessions and workshops that fit your needs.

2. React Day Berlin

Date: December 13 - 16, 2024
Location: Berlin, Germany

Berlin, a vibrant tech hub, hosts React Day Berlin, a must-attend event for React developers in Europe. This conference offers a perfect blend of technical sessions, hands-on workshops, and networking opportunities. Expect to hear from top-notch speakers, including React core team members and contributors from the open-source community.

3. React Advanced London

Date: October 25 - 28, 2024
Location: London, UK

React Advanced London is one of the most anticipated React events. Known for its diverse lineup of speakers and in-depth sessions, this conference covers everything from fundamental React concepts to advanced design patterns. It's a great opportunity to meet fellow developers and immerse yourself in the tech scene.

Meet the FocusReactive Team

FocusReactive is a co-organizer of these conferences, and our team will be actively participating in all of them. We’d love to connect with you—whether to chat about the latest trends in React, explore potential collaborations, or just say hello. If you're attending any of these events, don't hesitate to reach out to us!

Conclusion

With so many incredible events on the horizon, 2024 is set to be a fantastic year for the React community. Each of these conferences offers unique opportunities to learn, network, and grow as a developer. Be sure to grab your tickets and join the community at these exciting gatherings.

For more details on these and other React conferences, check out the full React conferences list by GitNation.

How to rank Higher on Google / How NextJS can improve SEO

Aleksei Zhilyuk — Wed, 08 Nov 2023 09:04:25 +0000

Creating a website is just the first step towards its success. One of the most important factors in the success of any website is its search engine optimization. SEO is the process of optimizing a website to increase its search visibility and attract more traffic through search engines. There are many factors that impact SEO, one of which is the choice of a suitable website development technology.

In this article, we will discuss how using NextJS can help improve the SEO of your website and what steps you can take to achieve a higher rank in search engines like Google.

What is NextJS?

NextJS is a popular React framework used for developing fast, scalable, and powerful web applications. It also allows for easy creation of universal applications with server-side rendering SSR, SSG, and client-side applications using the same code.

How can NextJS help improve SEO?

Server-side rendering

One of the most significant advantages of NextJS is the ability to use server-side rendering. This means that web pages can be rendered on the server and then sent to the client's browser as fully rendered pages, improving site speed and perception. SSR also helps search engine robots better index site pages, which can improve SEO.

Static site generation

NextJS also allows for the creation of static sites, which means that website pages can be pre-generated during application build time. This reduces page load time and improves SEO. Static sites are also easier to cache and can reduce the number of requests to the server.

Image optimization

NextJS automatically optimizes images, which can help reduce page load time. This can also help improve SEO, as search engines consider page load time when ranking websites. By optimizing images, you can help ensure that your website loads quickly and is more likely to rank higher in search engine results pages.

Code splitting

NextJS supports code splitting, which allows for the separation of code into smaller chunks that can be loaded on demand. This reduces the initial load time of a page and improves user experience. It also helps search engines crawl and index pages faster, which can improve SEO.

Meta tags and SEO optimization

NextJS makes it easy to add meta tags to web pages, which are crucial for SEO. Meta tags provide information about a web page to search engines and social media platforms, which can improve its visibility and ranking. NextJS also supports SEO optimization features like canonical tags and structured data, which can help further improve your website's SEO.

Page speed

In addition to its SEO benefits, NextJS can also help improve your website's page speed. By using features like server-side rendering and static site generation, NextJS can significantly reduce the initial load time of your website's pages. This means that your website can load faster, which can lead to a better user experience and higher search engine rankings. In fact, websites built with NextJS often achieve high scores on Google's PageSpeed Insights tool, which measures page speed performance. By using NextJS, you can ensure that your website not only ranks well in search engines but also provides a fast and seamless experience for your users.

In conclusion, using NextJS can help improve the SEO of your website by leveraging its features like server-side rendering, static site generation, image optimization, code splitting, and SEO optimization features. By implementing these features, you can help ensure that your website loads quickly, is more likely to rank higher in search engine results pages, and provides a better user experience.

Please find the links to read more about how NextJS can improve your SEO:

How we used Neural network, Replicate.com, and WebGL to generate photorealistic 3D avatars for the GitNation portal`s users

Aleksei Zhilyuk — Wed, 08 Nov 2023 08:46:59 +0000

Avatars are an integral part of any social portal. It is important for people to see the person they are interacting with, photography creates a personal connection and enhances the user experience. But what if we go further and use AI and WebGL to turn every user photo into an interactive 3D avatar?

Requirements for the solution - to work on one photo uploaded by the user. Be cheap and reliable enough to process thousands of photos.

At the moment, there are several technologies that allow you to reconstruct 3D portrait data from a photograph of a person, but the only one reliable enough for production is the depth map.

What is a depth map?

The depth map is an image complementary to a photo where each pixel stores the depth of the scene.

The web does not have built-in components for displaying images with a depth map. This requires WebGL, some JS and a custom shader. Fortunately, there are ready-made components like react-depth-map

How to get a depth map for a photo?

Previously, the only way to get a depth map was to scan the surface with laser scanning at the time of photographing. Luckily, advances in AI and ML have made it possible to reconstruct pixel depth from a single photo. You can read more about the method here, and the practical implementation of the neural network is at the PyTorch hub.

You can play online and make a depth map for your photo on Huggling face. I tested Midas on 10 user photos and it became clear that the reliability is high enough, it can work.

Implementation

The chosen solution was to store the path to the depth map in a separate field in the database, generate a depth map daily with a cron job, save the URL to the file in that field, and clear its value when a user uploads a new photo. If the field is empty, then there is no depth map for this user and we display the standard <image> tag.

Run Midas as a Cron job

This was a remarkable problem as the neural network is extremely slow without a GPU. The first thing I tried was to buy a GPU instance from https://runpod.io and run the project there. This approach lead to dozens of problems with the installation and after their resolution, the cost of renting a GPU instance still remained a problem.

Having searched a little, I came across a wonderful service https://replicate.com/ - these guys have already deployed all known neural networks, including Midas, into the cloud. Look at the entire list, you may find a solution to your other problems!

The ultimate cron job for generating avatars is a Node.js script that runs daily:

for (const user of usersWithoutDepthMaps) {
    queue.enqueue(async () => {
        await updateDepthMapForUser(user);
    });
}

function updateDepthMapForUser(user: User) {
    const output = await replicate.run('cjwbw/midas:a6ba5798f04f80d3b314de0f0a62277f21ab3503c60c84d4817de83c5edfdae0', {
        input: {
            model_type: 'dpt_beit_large_512',
            image: user.avatar,
        },
    });
    await downloadFile(output, getDetphMapFileName(user));
    const depthMapUrl = await uploadFileToCloudianryNode(getDetphMapFileName(user));

    await db.user.update({
        where: {
            id: user.id,
        },
        data: {
            avatarDepthMap: depthMapUrl as string,
        },
    });

    console.log('Updated depth map for', user.name);
}

Display

We have a depth map. How to display them? Because The GitNation portal is written in Next.js it makes sense to use react-depth-map. The component using it looks something like this:

export function Public3dAvatar({ user }) {
    return (
        <ImageDepthMap
            key={user.id}
            originalImg={getSquareCloudinaryImage(user.avatar)}
            depthImg={getSquareCloudinaryImage(user.avatarDepthMap)}
            verticalThreshold={35}
            horizontalThreshold={35}
            multiplier={1}
        />
    );
}

Unfortunately, out of the box, react-depth-map gives a strange artifact where the avatar splits into two:

The reason is that the shader is written in such a way that the back of the scene is displaced in the opposite direction from the foreground. This works well for depth maps with multiple depth layers, but often breaks avatars. The solution was to fork the component and fix the pixel shader, greatly reducing the back clipping.

And the final result:

Conclusion

Seeing the progress of AI technology is an absolute excitement! It is hard to predict how they will change the technology in the next few years, yet they have already given a fresh, previously impossible-to-have look to our portal, all speakers have received nearly free 3d avatars, and users have received a fresh design solution.

Also here is the page where you can unlock the insights of Adron - a seasoned expert in C#, JavaScript, Go, and a variety of other languages.
Gain a unique polyglot perspective and discover the optimal technology stacks for your projects.