Forem: Wlad Radchenko

Optimizing and Running Neural Networks on React Native: A Grass Case Study

Wlad Radchenko — Thu, 08 Jan 2026 11:53:22 +0000

During the long New Year holidays, I decided to start a new pet project. I wanted to experiment with something interesting involving neural networks, and the idea of a mobile app that could recognize plants — identifying species, age, and even diseases — directly on a mobile device without a server, while also providing recommendations on care and treatment, seemed exciting and new to me.

As I dug in, I realized that there are already quite a few similar apps. However, that didn’t diminish the value of the experiment: I wanted to understand how neural networks can be optimized for mobile devices and run directly on my phone. In this article, I’ll share what I managed to achieve — from selecting models to optimizing them and integrating them into React Native — as well as my experience with various methods for plant classification, disease detection, and estimating age and leaf count.

This is GitHub code and Hugging Face models.

SCOLD Model for Cross-Modal Disease Search

The first tool I decided to experiment with was the SCOLD model from Hugging Face. It was designed for cross-modal search, meaning it can match an image with a textual description. For example, you can feed in a photo of a corn leaf and a text description of a disease, and the model will estimate how well the text matches the image. SCOLD was trained on the LeafNet dataset.

Despite its interesting concept, I wasn’t entirely sure how to apply it in practice. On top of that, the model itself was about 1GB in size, which immediately makes running it on a mobile device challenging. I decided to modify the model slightly so that it could identify the plant disease directly.

Since I was only interested in working with images, I removed the Roberta text encoder and kept only the image processing module based on the Swin Transformer [source code]. As a result, I ended up with a much smaller model that outputs a 512-dimensional feature vector for each image.

Originally, this vector was meant to be compared with Roberta’s output for textual descriptions, but for mobile deployment, I decided on a different approach: instead of working with text, I compare the image vectors with vectors precomputed for the entire dataset.

This setup also introduced two additional files: a 248MB binary file with the embeddings and an 18MB JSON file containing disease labels. This approach preserves the original task of disease identification without needing to process text, while significantly reducing the model size for mobile use.

from timm import create_model
import torch
import torch.nn as nn
import numpy as np
from transformers import RobertaModel

EMBEDDING_DIM = 512


class ImageEncoder(nn.Module):
    def __init__(self):
        super(ImageEncoder, self).__init__()
        # Load the Swin Transformer with features_only=True
        self.swin = create_model("swin_base_patch4_window7_224.ms_in22k", pretrained=True, features_only=True)
        for param in self.swin.parameters():
            param.requires_grad = True

        # Get the feature size of the final stage
        self.swin_output_dim = self.swin.feature_info.channels()[-1]  # Last stage: 1024 channels

        # Define FC layer
        self.fc1 = nn.Linear(self.swin_output_dim * 7 * 7, EMBEDDING_DIM)  # Flattened input size
        nn.init.xavier_uniform_(self.fc1.weight)
        nn.init.zeros_(self.fc1.bias)
        for param in self.fc1.parameters():
            param.requires_grad = True

    def forward(self, x):
        # Extract features from Swin
        swin_features = self.swin(x)[-1]  # Use the last stage feature map (e.g., [B, 1024, 7, 7])

        # Flatten feature map
        swin_features = swin_features.view(swin_features.size(0), -1)  # Shape: (B, 1024*7*7)

        # Pass through FC layer
        output = self.fc1(swin_features)  # Shape: (B, embedding_dim)
        return output


class LVL(nn.Module):
    def __init__(self):
        super(LVL, self).__init__()
        self.image_encoder = ImageEncoder()
        self.text_encoder = nn.Identity()
        self.t_prime = nn.Parameter(torch.ones([]) * np.log(0.07))
        self.b = nn.Parameter(torch.ones([]) * 0)

    def get_images_features(self, images):
        image_embeddings = self.image_encoder(images)  # (batch_size, EMBEDDING_DIM)
        image_embeddings = nn.functional.normalize(image_embeddings, p=2, dim=-1)
        return image_embeddings

    def get_texts_feature(self, input_ids=None, attention_mask=None):
        """
        Plug
        :param input_ids: Tensor of shape (batch_size, seq_length)
        :param attention_mask: Tensor of shape (batch_size, seq_length)
        :return:
        """
        return None

    def forward(self, images, input_ids=None, attention_mask=None):
        """
        Args:
            images: Tensor of shape (batch_size, 3, 224, 224)
            input_ids: Tensor of shape (batch_size, seq_length)
            attention_mask: Tensor of shape (batch_size, seq_length)

        Returns:
            Image and text embeddings normalized for similarity calculation
        """

        image_embeddings = self.get_images_features(images)
        return image_embeddings

The next step was converting the model to the ONNX format, which is well-supported on mobile and allows for integrating neural networks directly into React Native. During this process, I had to account for the specifics of each model: setting up the correct dummy input to verify the input shape, specifying input and output names, defining dynamic_axes to support batch sizes, and sometimes adjusting the opset version to ensure compatibility with ONNX Runtime.

import torch
# pip install onnxscript onnxruntime onnxruntime-tools
from model import LVL


def export(model_path: str, output_name: str):
    device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
    # Create a dummy input with the same dimensions as the real data.
    dummy_input = torch.randn(1, 3, 512, 512, device=device)

    # Load model
    model = LVL()
    model.to(device)
    model.eval()

    #  git clone https://huggingface.co/enalis/scold
    state_dict = torch.load(model_path, map_location=device)

    # Leave only the keys that are in the new model (only image_encoder)
    model_dict = model.state_dict()
    filtered_dict = {k: v for k, v in state_dict.items() if k in model_dict}

    # Load only image_encoder weights
    model.load_state_dict(filtered_dict, strict=False)  # <- strict=False important!

    # Export
    torch.onnx.export(
        model,                      # Model
        dummy_input,                # Input
        output_name,           # Name
        export_params=True,         # Save weights
        opset_version=18,           # Version
        do_constant_folding=True,   # Optim
        input_names=['images'],     # Name of inputs
        output_names=['image_embeddings'],  # Name of outputs
        dynamic_axes={'images': {0: 'batch_size'},  # Dynamic support batch size
                      'image_embeddings': {0: 'batch_size'}}
    )
    print("Finish!")


if __name__ == "__main__":
    from argparse import ArgumentParser

    parser = ArgumentParser()
    parser.add_argument("--model_path", type=str, required=True, help="Path to model pth")
    parser.add_argument("--output_name", type=str, default="disease_detector.onnx", help="Path to save onnx model with name model.onnx")
    args = parser.parse_args()

    export(model_path=args.model_path, output_name=args.output_name)

After that, I further reduced the model size by converting it from float32 to float16, while keeping the input format as float32. As a result, the model size dropped from 1 GB to 231 MB, making it much more suitable for a mobile application.

import onnx
from onnxconverter_common import float16


def export(model_path: str, output_name: str):
    # Load model
    model_fp32 = onnx.load(model_path)
    # Export to float16
    model_fp16 = float16.convert_float_to_float16(model_fp32, keep_io_types=True)
    # Save
    onnx.save(model_fp16, output_name)


if __name__ == "__main__":
    from argparse import ArgumentParser

    parser = ArgumentParser()
    parser.add_argument("--model_path", type=str, required=True, help="Path model onnx")
    parser.add_argument("--output_name", type=str, default="fp16.onnx", help="Path to save fp16 mode with name model_fp16.onnx")
    args = parser.parse_args()

    export(model_path=args.model_path, output_name=args.output_name)

The disease_detection.onnx model itself and the code for running it can be found on GitHub.

Plant Classification from Images

It would be hard to imagine this project without a neural network for recognizing plants from photos, so the next step was to tackle exactly that. While looking for suitable data and models, I came across the PlantCLEF 2024 competition, which led me to the Pl@ntNet dataset. The dataset itself is huge, and training a model on the full dataset would require a lot of time and resources. Fortunately, instead of overloading my laptop, I found that a pre-trained model along with the code to run it had already been released on Zenodo.

The model takes a plant image as input and outputs a set of probabilities corresponding to specific plant species. Essentially, this is a standard image classification task without any extra complications. For my purposes, this was more than enough, so I followed the familiar path: exporting the model to ONNX and then converting it to float16 to reduce its size and make it more suitable for mobile deployment. The ready-to-use model and the code for running it are available in the project repository.

import torch
import timm
# pip install onnxscript onnxruntime onnxruntime-tools
import argparse


def load_class_mapping(class_list_file):
    with open(class_list_file) as f:
        class_index_to_class_name = {i: line.strip() for i, line in enumerate(f)}
    return class_index_to_class_name


def export(model_path: str, output_name: str):
    torch.serialization.add_safe_globals([argparse.Namespace])
    device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

    # Class and model after train.py
    class_mapping = load_class_mapping(args.class_mapping)

    # Load model
    model = timm.create_model('vit_base_patch14_reg4_dinov2.lvd142m', pretrained=False, num_classes=len(class_mapping))
    checkpoint = torch.load(model_path, map_location=device)
    model.load_state_dict(checkpoint['state_dict'])
    model.eval()

    # Dummy input (ViT Base 224x224 RGB)
    dummy_input = torch.randn(1, 3, 518, 518)

    # Export
    torch.onnx.export(
        model,                      # Model
        dummy_input,                # Input
        output_name,           # Name
        export_params=True,         # Save weights
        opset_version=18,           # Version
        input_names=['input'],      # Name of inputs
        output_names=['output'],    # Name of outputs
        dynamic_axes={"input": {0: "batch_size"}, "output": {0: "batch_size"}},
    )
    print("Finished!")


if __name__ == "__main__":
    from argparse import ArgumentParser

    parser = ArgumentParser()
    parser.add_argument("--class_mapping", type=str, required=True, help="Path to class mapping")
    parser.add_argument("--model_path", type=str, required=True, help="Path species mapping")
    parser.add_argument("--output_name", type=str, default="plant_classificator.onnx", help="Path to save onnx model with name model.onnx")
    args = parser.parse_args()

    export(model_path=args.model_path, output_name=args.output_name)

Later, after studying the dataset more closely, I realized that it mostly contains rare and exotic plants. It barely includes common ornamental flowers, vegetables, or fruits that one might expect in such an app—or at least typical bouquet flowers.

Because of this, I decided to go further and implement code for fine-tuning the model or training it from scratch. In practice, training from scratch turned out to be the preferable option, as it allows creating a smaller model, which is critical for mobile devices. Additionally, this approach gives users the flexibility to choose which model to load depending on their specific needs and the available resources on their device.

I moved the training process into a separate module and documented the dataset structure and the steps for running training in the project documentation. Training can be run with a batch size of 8 on a GPU with 8 GB of VRAM, or the batch size can be reduced if resources are limited. The dataset must be assembled manually by combining data from Kaggle and other open sources. This approach proved to be more flexible and aligns well with the concept of an experimental mobile ML project.

Regression Model for Estimating Plant Age and Leaf Count

The task of estimating a plant’s age and the number of its leaves may seem trivial at first, but in practice, I couldn’t find any ready-to-use models. There is a scientific work, GroMo: Plant Growth Modeling with Multiview Images, along with its corresponding repository, but the code there is somewhat messy and not easily scalable for practical use. Therefore, I decided not to adapt it, but to rewrite the solution from scratch, tailored to my own constraints.

In the original work, plants of different species, such as mustard, radish, wheat, and okra, were photographed from multiple angles and at various growth stages, with annotations indicating the plant’s age and the number of leaves. An important point is that visual features strongly depend on the species: leaf shape, size, and even the overall plant structure can vary significantly, and sometimes leaves are visually almost indistinguishable from stems.

In the original model, four images from different angles were fed into the network at once, whereas in my case, the input was simplified to a single photo, which is more aligned with a real mobile user scenario. There was also an idea to check whether the current number of leaves or stems is appropriate for the plant’s age—or if it’s below average—but I decided to leave that for future work.

At the core of this model is MobileNetV3 Large, used as a universal visual encoder. From the pretrained network, only the convolutional part is taken, which extracts compact and informative features from a leaf image. These features are then aggregated using Adaptive Average Pooling into a fixed-size vector of 960 dimensions, making the model independent of the input image size.

This shared feature vector is then used for two regression tasks simultaneously: estimating the number of leaves and the plant’s age. To accomplish this, two separate “heads” are added, each implemented as a small fully connected block. I named this model LeafNet.

import torch.nn as nn
from torchvision.models import mobilenet_v3_large

class LeafNet(nn.Module):
    def __init__(self):
        super().__init__()
        backbone = mobilenet_v3_large(weights="DEFAULT")
        self.encoder = backbone.features

        self.pool = nn.AdaptiveAvgPool2d(1)

        self.count_head = nn.Sequential(
            nn.Linear(960, 128),
            nn.ReLU(inplace=True),
            nn.Linear(128, 1)
        )

        self.age_head = nn.Sequential(
            nn.Linear(960, 128),
            nn.ReLU(inplace=True),
            nn.Linear(128, 1)
        )

    def forward(self, x):
        feat = self.encoder(x)
        pooled = self.pool(feat).flatten(1)
        count = self.count_head(pooled)
        age = self.age_head(pooled)
        return feat, count, age

Training was performed on a combined dataset without separating by plant species, allowing the model to learn from a wider variety of visual patterns. The total dataset size was around 380 GB. Full training took approximately three days, and during training, the model is automatically exported to ONNX format for use on mobile devices. The training documentation, along with all the code for running experiments, is available in the project repository.

Integrating and Running Neural Networks in React Native

In this part of the article, I’ll discuss running neural networks directly on a mobile device and the code required to make it work. I intentionally skip basic setup steps like installing Node.js, running an Android emulator, or setting up the development environment. Those are well-documented elsewhere and not directly relevant to the task at hand. Here, we’ll focus on the code and the limitations encountered when working with neural networks on mobile platforms.

For convenience, the project repository includes a ready-to-use APK for those who don’t want to build the app themselves (it’s large because it includes the models). However, for development and experimentation, I highly recommend building the project manually. Testing was performed on a Pixel 8 Pro emulator, so behavior on other devices may vary. Detailed build and run instructions are available in the repository under the mobile folder.

The first major problem, which cost me about a day of development, was choosing the right mobile stack. I initially started with Expo since it allows quickly spinning up a prototype. However, I discovered that running ONNX models in this environment is either unstable or completely impossible. Even building a custom Expo client didn’t help. Eventually, I abandoned Expo and switched to pure React Native, which proved to be the correct choice for further native integration and neural network usage.

Next, I ran into a fundamental limitation of mobile platforms: the JavaScript engine on Android and iOS is not designed to handle large binary files. Files like captions.json and especially embeddings.bin (248 MB) cannot simply be loaded into memory from JavaScript without risking OutOfMemory errors or extremely slow performance due to constant data copying. Yet, these files are essential for the disease detection model, where searches are performed over precomputed embedding vectors.

To solve this, all work with large data and vector search is moved to the native layer on Android by implementing a custom Kotlin native module, called FaissSearchModule. Its main purpose is to load the embedding file, store the data outside the JavaScript heap, and perform vector search entirely in native memory. Instead of loading embeddings.bin into RAM, a memory-mapped file is used via MappedByteBuffer. This allows the operating system to manage memory and load data on demand, without holding the entire file in the app’s memory. Essentially, the embedding file becomes part of the process’s virtual memory, which is crucial for handling large datasets on mobile devices.

The nearest-vector search is also performed entirely in native code. Only the query embedding is sent from JavaScript, and the results returned are compact: indexes, distances, and text labels for diseases. This minimizes the load on the JavaScript-to-native bridge and avoids transferring large arrays back and forth. A similar approach is used for captions.json: instead of standard JSON parsing, the file is read streamingly, preventing the need to load the entire file into memory. These files are located in the Android project directory for proper access.

package com.berkano

import com.facebook.react.bridge.*
import org.json.JSONArray
import java.io.*
import java.nio.ByteBuffer
import java.nio.ByteOrder
import kotlin.math.sqrt

/**
 * Native module for vector search using Faiss
 * Data is stored in native memory and does not enter the JavaScript heap
 */
class FaissSearchModule(
    private val reactContext: ReactApplicationContext
) : ReactContextBaseJavaModule(reactContext) {

    // Store data in native memory
    // Use MappedByteBuffer instead of FloatArray for large files
    // This allows avoiding loading all data into RAM at once
    private var embeddingsBuffer: java.nio.MappedByteBuffer? = null
    private var captions: List<String>? = null
    private var numVectors: Int = 0
    private var embeddingSize: Int = 512
    private var embeddingsFile: File? = null

    override fun getName(): String {
        return "FaissSearch"
    }

    /**
     * Loads embeddings from a binary file using a memory-mapped file
     * Does NOT load all data into RAM, reads on demand
     */
    @ReactMethod
    fun loadEmbeddings(embeddingsPath: String, promise: Promise) {
        try {
            val file = File(embeddingsPath)
            if (!file.exists()) {
                promise.reject("FILE_NOT_FOUND", "Embeddings file not found: $embeddingsPath")
                return
            }

            val fileSize = file.length()
            numVectors = (fileSize / 4 / embeddingSize).toInt() // 4 bytes per float

            // Use MappedByteBuffer — this does NOT fully load the file into RAM
            // The operating system manages memory and loads data on demand
            FileInputStream(file).use { fis ->
                val channel = fis.channel
                val byteBuffer = channel.map(
                    java.nio.channels.FileChannel.MapMode.READ_ONLY,
                    0,
                    fileSize
                )
                byteBuffer.order(ByteOrder.LITTLE_ENDIAN)
                embeddingsBuffer = byteBuffer
            }

            embeddingsFile = file

            val result = Arguments.createMap()
            result.putInt("numVectors", numVectors)
            result.putInt("embeddingSize", embeddingSize)
            result.putLong("fileSize", fileSize)

            promise.resolve(result)
            println("Mapped $numVectors vectors of size $embeddingSize from file (${fileSize / 1024 / 1024}MB)")
        } catch (e: OutOfMemoryError) {
            promise.reject(
                "OUT_OF_MEMORY",
                "Not enough memory to map embeddings file. File too large: ${e.message}",
                e
            )
        } catch (e: Exception) {
            promise.reject("LOAD_ERROR", "Failed to load embeddings: ${e.message}", e)
        }
    }

    /**
     * Reads a vector by index from the memory-mapped file
     * Does not load all data into memory
     */
    private fun getVector(index: Int): FloatArray {
        if (embeddingsBuffer == null) {
            throw IllegalStateException("Embeddings not loaded")
        }

        val startByte = index * embeddingSize * 4 // 4 bytes per float
        val vector = FloatArray(embeddingSize)

        embeddingsBuffer!!.position(startByte)
        embeddingsBuffer!!.asFloatBuffer().get(vector)

        return vector
    }

    /**
     * Loads captions from a JSON file
     * Uses streaming parsing for large files
     */
    @ReactMethod
    fun loadCaptions(captionsPath: String, promise: Promise) {
        try {
            val file = File(captionsPath)
            if (!file.exists()) {
                promise.reject("FILE_NOT_FOUND", "Captions file not found: $captionsPath")
                return
            }

            // Use BufferedReader for efficient reading of large files
            val captionsList = mutableListOf<String>()
            var buffer = StringBuilder()
            var inString = false
            var escapeNext = false

            BufferedReader(FileReader(file), 8192).use { reader ->
                var char: Int
                while (reader.read().also { char = it } != -1) {
                    when {
                        escapeNext -> {
                            buffer.append(char.toChar())
                            escapeNext = false
                        }
                        char == '\\'.code -> {
                            escapeNext = true
                        }
                        char == '"'.code -> {
                            if (inString) {
                                // End of string
                                captionsList.add(buffer.toString())
                                buffer = StringBuilder()
                                inString = false
                            } else {
                                // Start of string
                                inString = true
                            }
                        }
                        inString -> {
                            buffer.append(char.toChar())
                        }
                        char == ']'.code -> {
                            // End of array
                            break
                        }
                    }
                }
            }

            captions = captionsList
            val result = Arguments.createMap()
            result.putInt("count", captionsList.size)

            promise.resolve(result)
            println("Loaded ${captionsList.size} captions")
        } catch (e: OutOfMemoryError) {
            promise.reject(
                "OUT_OF_MEMORY",
                "Not enough memory to load captions. File too large: ${e.message}",
                e
            )
        } catch (e: Exception) {
            promise.reject("LOAD_ERROR", "Failed to load captions: ${e.message}", e)
        }
    }

    /**
     * Computes cosine distance between two vectors
     */
    private fun cosineDistance(vecA: FloatArray, vecB: FloatArray): Float {
        var dotProduct = 0f
        var normA = 0f
        var normB = 0f

        for (i in vecA.indices) {
            dotProduct += vecA[i] * vecB[i]
            normA += vecA[i] * vecA[i]
            normB += vecB[i] * vecB[i]
        }

        normA = sqrt(normA)
        normB = sqrt(normB)

        if (normA == 0f || normB == 0f) {
            return 1f
        }

        val similarity = dotProduct / (normA * normB)
        return 1f - similarity
    }

    /**
     * Finds top-K nearest vectors to the query
     * Search is performed in native memory, only results are sent to JS
     */
    @ReactMethod
    fun search(
        queryEmbedding: ReadableArray,
        topK: Int,
        promise: Promise
    ) {
        try {
            if (embeddingsBuffer == null) {
                promise.reject("NOT_LOADED", "Embeddings not loaded. Call loadEmbeddings first.")
                return
            }

            if (captions == null) {
                promise.reject("NOT_LOADED", "Captions not loaded. Call loadCaptions first.")
                return
            }

            // Convert query from ReadableArray to FloatArray
            val query = FloatArray(queryEmbedding.size())
            for (i in 0 until queryEmbedding.size()) {
                query[i] = queryEmbedding.getDouble(i).toFloat()
            }

            if (query.size != embeddingSize) {
                promise.reject(
                    "INVALID_SIZE",
                    "Query embedding size ${query.size} != $embeddingSize"
                )
                return
            }

            // Compute distances for all vectors
            // Vectors are read on demand from the memory-mapped file
            val distances = mutableListOf<Pair<Int, Float>>()

            for (i in 0 until numVectors) {
                // Read vector from file on demand
                val vector = getVector(i)
                val distance = cosineDistance(query, vector)
                distances.add(Pair(i, distance))
            }

            // Sort and take top-K
            distances.sortBy { it.second }
            val topResults = distances.take(topK)

            // Prepare results to send to JS
            val results = Arguments.createArray()
            for ((index, distance) in topResults) {
                val resultItem = Arguments.createMap()
                resultItem.putInt("index", index)
                resultItem.putDouble("distance", distance.toDouble())
                resultItem.putString("caption", captions!![index])
                results.pushMap(resultItem)
            }

            promise.resolve(results)
        } catch (e: Exception) {
            promise.reject("SEARCH_ERROR", "Search failed: ${e.message}", e)
        }
    }

    /**
     * Clears loaded data from memory
     */
    @ReactMethod
    fun clearCache(promise: Promise) {
        embeddingsBuffer = null
        embeddingsFile = null
        captions = null
        numVectors = 0
        promise.resolve(null)
    }

    /**
     * Checks whether data is loaded
     */
    @ReactMethod
    fun isLoaded(promise: Promise) {
        val result = Arguments.createMap()
        result.putBoolean("embeddingsLoaded", embeddingsBuffer != null)
        result.putBoolean("captionsLoaded", captions != null)
        result.putInt("numVectors", numVectors)
        promise.resolve(result)
    }
}

The FaissSearchPackage plays a supporting role in this setup. Its purpose is to register the native module in React Native and inform the JavaScript part of the app that FaissSearchModule is available. All the actual logic and computation happens inside the module itself; the package serves purely as an infrastructure layer to connect it to the React Native framework.

package com.berkano

import com.facebook.react.ReactPackage
import com.facebook.react.bridge.NativeModule
import com.facebook.react.bridge.ReactApplicationContext
import com.facebook.react.uimanager.ViewManager

class FaissSearchPackage : ReactPackage {
    override fun createNativeModules(
        reactContext: ReactApplicationContext
    ): List<NativeModule> {
        return listOf(FaissSearchModule(reactContext))
    }

    override fun createViewManagers(
        reactContext: ReactApplicationContext
    ): List<ViewManager<*, *>> {
        return emptyList()
    }
}

Besides working with the embedding files, there’s another problem that can’t be solved purely in JavaScript. All ONNX models in the project expect a tensor of fixed shape and format as input, whereas in React Native an image usually comes as a URI or a base64 string. Converting an image to a numeric tensor in JavaScript is possible, but in practice it either runs slowly or consumes a lot of memory, especially for camera images. For this reason, input preprocessing is also moved to the native layer.

For this, a separate native module, ImageDecoderModule, was implemented. Its role is to take an image URI, load it via ContentResolver, decode it into a Bitmap, resize it to the required dimensions, and convert it into a CHW-format tensor expected by the PyTorch-trained models exported to ONNX. At this stage, pixel values are normalized to the [0,1] range, and RGB channels are explicitly separated so that the resulting array exactly matches the model input.

Importantly, all of this logic runs entirely on the Android side before passing the data to JavaScript. React Native cannot directly handle FloatArray, so the tensor is converted into a WritableArray, which is then safely passed across the bridge. This approach ensures the array is created once in native code without intermediate copies or unnecessary allocations in the JS engine, significantly reducing memory usage and speeding up data preparation for inference.

package com.berkano;

import android.graphics.Bitmap
import android.graphics.BitmapFactory
import android.net.Uri
import com.facebook.react.bridge.*
import java.io.InputStream

class ImageDecoderModule(
  private val reactContext: ReactApplicationContext
) : ReactContextBaseJavaModule(reactContext) {

  override fun getName(): String {
    return "ImageDecoder"
  }

  @ReactMethod
  fun decodeToTensor(
    uriString: String,
    targetWidth: Int,
    targetHeight: Int,
    promise: Promise
  ) {
    try {
      val uri = Uri.parse(uriString)
      val inputStream: InputStream? =
        reactContext.contentResolver.openInputStream(uri)

      if (inputStream == null) {
        promise.reject("ERROR", "Cannot open image stream")
        return
      }

      var bitmap: Bitmap? = null
      try {
        bitmap = BitmapFactory.decodeStream(inputStream)
        if (bitmap == null) {
          promise.reject("ERROR", "Cannot decode image")
          return
        }
        bitmap = Bitmap.createScaledBitmap(bitmap, targetWidth, targetHeight, true)
      } finally {
        inputStream.close()
      }

      val width = bitmap.width
      val height = bitmap.height

      val pixels = IntArray(width * height)
      bitmap.getPixels(pixels, 0, width, 0, 0, width, height)

      // CHW: [3, H, W] — format used for passing data to JavaScript
      val tensor = FloatArray(3 * width * height)

      for (y in 0 until height) {
        for (x in 0 until width) {
          val color = pixels[y * width + x]

          val r = (color shr 16 and 0xFF) / 255f
          val g = (color shr 8 and 0xFF) / 255f
          val b = (color and 0xFF) / 255f

          val idx = y * width + x
          tensor[idx] = r
          tensor[width * height + idx] = g
          tensor[2 * width * height + idx] = b
        }
      }

      // Convert FloatArray to WritableArray for proper transfer to JavaScript
      // React Native cannot directly convert FloatArray, so WritableArray is used
      val writableArray = Arguments.createArray()
      for (value in tensor) {
        // pushDouble accepts double, so convert float to double
        writableArray.pushDouble(value.toDouble())
      }

      promise.resolve(writableArray)
    } catch (e: Exception) {
      promise.reject("ERROR_DECODING_IMAGE", e)
    }
  }
}

Just like with the vector search module, ImageDecoderModule is registered via a separate ImageDecoderPackage, which is then included in MainApplication.kt. Without this, React Native won’t recognize the module. Additionally, the required permissions must be explicitly declared in AndroidManifest.xml; otherwise, the app won’t be able to access images. All these details are available in the project repository, in the Android part of the mobile application.

package com.berkano;

import com.facebook.react.ReactPackage
import com.facebook.react.bridge.NativeModule
import com.facebook.react.bridge.ReactApplicationContext
import com.facebook.react.uimanager.ViewManager

class ImageDecoderPackage : ReactPackage {

  override fun createNativeModules(
    reactContext: ReactApplicationContext
  ): List<NativeModule> {
    return listOf(ImageDecoderModule(reactContext))
  }

  override fun createViewManagers(
    reactContext: ReactApplicationContext
  ): List<ViewManager<*, *>> {
    return emptyList()
  }
}

In the end, all heavy and memory-sensitive operations remain on the native side, while JavaScript only works with preprocessed data and inference results.

Now, for running the models on a mobile device, I created a separate file that handles preparing, copying, initializing models, and running inference. On the first run, the code checks whether the model files and associated assets exist in the local filesystem. If not, they are copied from the assets folder (on Android). The logic is implemented in the copyAssetIfNeeded function. On Android, it’s recommended to copy large files using copyFileAssets.

The prepareModelAssets function gathers all necessary paths to models, embeddings, and auxiliary files, returning an object with these paths for later use. ONNX models are initialized via initializeModel, which creates an ONNX session with NNAPI priority for GPU/NPU, falling back to CPU if needed.

Inference is performed in runInference, where a Float32Array containing the image tensor (prepared by the native modules, as described in the ImageDecoderModule) is passed in. The function constructs a tensor in the correct format for the model, e.g., [1, 3, 224, 224] for disease detection, runs the model, and returns a normalized embedding.

import * as ort from 'onnxruntime-react-native';
import RNFS from 'react-native-fs';
import { Platform } from 'react-native';

async function copyAssetIfNeeded(assetName: string, folder: string): Promise<string> {
  const localPath = `${RNFS.DocumentDirectoryPath}/${assetName}`;

  const exists = await RNFS.exists(localPath);
  if (!exists) {
    try {
      if (Platform.OS === 'android') {
        // Android: read from assets
        // Path must be relative to the assets folder: folder/assetName
        const assetPath = `${folder}/${assetName}`;
        console.log(`Attempting to copy from assets: ${assetPath} to ${localPath}`);
        await RNFS.copyFileAssets(assetPath, localPath);
        console.log(`Successfully copied ${assetName} to local FS: ${localPath}`);
      } else {
        // iOS: read from the main bundle
        const source = `${RNFS.MainBundlePath}/${folder}/${assetName}`;
        console.log(`Attempting to copy from bundle: ${source} to ${localPath}`);
        await RNFS.copyFile(source, localPath);
        console.log(`Successfully copied ${assetName} to local FS: ${localPath}`);
      }
    } catch (err: any) {
      console.error(`Failed to copy ${assetName} from ${folder}/${assetName}:`, err);
      console.error(`Error details:`, {
        message: err?.message,
        code: err?.code,
        platform: Platform.OS,
        localPath,
        assetPath: `${folder}/${assetName}`,
      });
      // Try an alternative path without the folder (in case the file is in the root of assets)
      if (Platform.OS === 'android') {
        try {
          console.log(`Trying alternative path: ${assetName}`);
          await RNFS.copyFileAssets(assetName, localPath);
          console.log(`Successfully copied using alternative path`);
        } catch (altErr) {
          console.error(`Alternative path also failed:`, altErr);
          throw new Error(
            `Failed to copy ${assetName}. Please verify that the file is located in assets/${folder}/`
          );
        }
      } else {
        throw err;
      }
    }
  } else {
    console.log(`${assetName} already exists at ${localPath}`);
  }

  return localPath;
}

export async function prepareModelAssets() {
  const modelPath = await copyAssetIfNeeded('disease_detection.onnx', 'models');
  const embeddingsPath = await copyAssetIfNeeded('embeddings.bin', 'files');
  const captionsPath = await copyAssetIfNeeded('captions.json', 'files');
  const classMappingPath = await copyAssetIfNeeded('class_mapping.txt', 'files');
  const speciesMappingPath = await copyAssetIfNeeded('species_id_to_name.txt', 'files');

  return { 
    modelPath, 
    embeddingsPath, 
    captionsPath,
    classMappingPath,
    speciesMappingPath,
  };
}

export interface ModelSession {
  session: ort.InferenceSession;
  inputName: string;
  outputName: string;
}

let modelSession: ModelSession | null = null;

/**
 * Initializes the ONNX model with GPU/NPU priority via NNAPI
 */
export async function initializeModel(): Promise<ModelSession> {
  if (modelSession) {
    return modelSession;
  }

  try {
    // Use the path obtained from prepareModelAssets
    const { modelPath } = await prepareModelAssets();

    // Use the local model path
    const modelUri = modelPath;

    // Session options with NNAPI priority (GPU/NPU)
    const sessionOptions: ort.InferenceSession.SessionOptions = {
      executionProviders: ['nnapi', 'cpu'], // NNAPI for GPU/NPU, CPU as fallback
      graphOptimizationLevel: 'all',
    };

    // Create the inference session
    // onnxruntime-react-native can work directly with file paths
    const session = await ort.InferenceSession.create(modelUri, sessionOptions);

    // Retrieve input and output names
    const inputNames = session.inputNames;
    const outputNames = session.outputNames;

    if (inputNames.length === 0 || outputNames.length === 0) {
      throw new Error('The model has no inputs or outputs');
    }

    modelSession = {
      session,
      inputName: inputNames[0],  // Assume the first input
      outputName: outputNames[0], // Assume the first output
    };

    console.log('Model successfully loaded');
    console.log('Input name:', modelSession.inputName);
    console.log('Output name:', modelSession.outputName);

    return modelSession;
  } catch (error) {
    console.error('Error while loading the model:', error);
    throw error;
  }
}

/**
 * Runs model inference on an image
 * @param imageTensor Image tensor in [1, 3, 224, 224] format
 * @returns Image embedding
 */
export async function runInference(
  imageTensor: Float32Array,
): Promise<Float32Array> {
  if (!modelSession) {
    throw new Error('Model is not initialized. Call initializeModel() first.');
  }

  try {
    // Create input tensor
    // Format: [batch, channels, height, width] = [1, 3, 224, 224]
    const inputTensor = new ort.Tensor('float32', imageTensor, [1, 3, 224, 224]);

    // Run inference
    const feeds = { [modelSession.inputName]: inputTensor };
    const results = await modelSession.session.run(feeds);

    // Retrieve output tensor
    const outputTensor = results[modelSession.outputName];
    const embedding = outputTensor.data as Float32Array;

    // Normalize the embedding
    const norm = Math.sqrt(
      Array.from(embedding).reduce((sum, val) => sum + val * val, 0),
    );
    const normalizedEmbedding = new Float32Array(embedding.length);
    for (let i = 0; i < embedding.length; i++) {
      normalizedEmbedding[i] = embedding[i] / norm;
    }

    return normalizedEmbedding;
  } catch (error) {
    console.error('Error during inference:', error);
    throw error;
  }
}

/**
 * Releases model resources
 */
export function disposeModel(): void {
  if (modelSession) {
    modelSession.session.release();
    modelSession = null;
  }
}

To manage multiple models, I created modelManager.ts. It allows registering different models (disease, plant, age) and managing their loading into memory. Only two models are kept in memory at the same time to avoid overloading the device. If the limit is reached, the least recently used model is unloaded.

import * as ort from 'onnxruntime-react-native';
import RNFS from 'react-native-fs';
import { Platform } from 'react-native';

export type ModelType = 'disease' | 'plant' | 'age';

export interface ModelInfo {
  type: ModelType;
  path: string;
  session: ort.InferenceSession | null;
  inputName: string;
  outputName: string;
  isLoaded: boolean;
  lastUsed: number;
}

// Maximum number of models allowed in memory at the same time
const MAX_MODELS_IN_MEMORY = 2;

// Model cache
const modelCache: Map<ModelType, ModelInfo> = new Map();

/**
 * Copies a file from assets to the local file system
 */
async function copyAssetIfNeeded(assetName: string, folder: string): Promise<string> {
  const localPath = `${RNFS.DocumentDirectoryPath}/${assetName}`;

  const exists = await RNFS.exists(localPath);
  if (!exists) {
    try {
      if (Platform.OS === 'android') {
        const assetPath = `${folder}/${assetName}`;
        console.log(`Copying ${assetName} from assets...`);
        await RNFS.copyFileAssets(assetPath, localPath);
        console.log(`Successfully copied ${assetName}`);
      } else {
        const source = `${RNFS.MainBundlePath}/${folder}/${assetName}`;
        await RNFS.copyFile(source, localPath);
      }
    } catch (err: any) {
      console.error(`Failed to copy ${assetName}:`, err);
      if (Platform.OS === 'android') {
        try {
          await RNFS.copyFileAssets(assetName, localPath);
        } catch (altErr) {
          throw new Error(`Failed to copy ${assetName}`);
        }
      } else {
        throw err;
      }
    }
  }

  return localPath;
}

/**
 * Unloads the least recently used model from memory
 */
function unloadLeastUsedModel(): void {
  let leastUsed: ModelInfo | null = null;
  let leastUsedTime = Date.now();

  for (const model of modelCache.values()) {
    if (model.isLoaded && model.lastUsed < leastUsedTime) {
      leastUsed = model;
      leastUsedTime = model.lastUsed;
    }
  }

  if (leastUsed) {
    console.log(`Unloading model: ${leastUsed.type}`);
    if (leastUsed.session) {
      leastUsed.session.release();
      leastUsed.session = null;
      leastUsed.isLoaded = false;
    }
  }
}

/**
 * Loads a model into memory
 */
async function loadModel(modelInfo: ModelInfo): Promise<void> {
  if (modelInfo.isLoaded && modelInfo.session) {
    modelInfo.lastUsed = Date.now();
    return;
  }

  // Check if the memory limit for loaded models has been reached
  const loadedModels = Array.from(modelCache.values()).filter(m => m.isLoaded);
  if (loadedModels.length >= MAX_MODELS_IN_MEMORY) {
    console.log('Memory limit reached, unloading least recently used model...');
    unloadLeastUsedModel();
  }

  try {
    console.log(`Loading model: ${modelInfo.type}`);

    const sessionOptions: ort.InferenceSession.SessionOptions = {
      executionProviders: ['nnapi', 'cpu'],
      graphOptimizationLevel: 'all',
    };

    const session = await ort.InferenceSession.create(modelInfo.path, sessionOptions);

    const inputNames = session.inputNames;
    const outputNames = session.outputNames;

    if (inputNames.length === 0 || outputNames.length === 0) {
      throw new Error('The model has no inputs or outputs');
    }

    modelInfo.session = session;
    modelInfo.inputName = inputNames[0];
    modelInfo.outputName = outputNames[0];
    modelInfo.isLoaded = true;
    modelInfo.lastUsed = Date.now();

    console.log(`Model ${modelInfo.type} loaded successfully`);
  } catch (error) {
    console.error(`Error loading model ${modelInfo.type}:`, error);
    throw error;
  }
}

/**
 * Initializes the model management system
 */
export async function initializeModelManager(): Promise<void> {
  // Prepare model paths
  const diseaseModelPath = await copyAssetIfNeeded('disease_detection.onnx', 'models');
  const plantModelPath = await copyAssetIfNeeded('plant_classification.onnx', 'models');
  const ageModelPath = await copyAssetIfNeeded('plant_analysis.onnx', 'models');

  // Register models
  modelCache.set('disease', {
    type: 'disease',
    path: diseaseModelPath,
    session: null,
    inputName: '',
    outputName: '',
    isLoaded: false,
    lastUsed: 0,
  });

  modelCache.set('plant', {
    type: 'plant',
    path: plantModelPath,
    session: null,
    inputName: '',
    outputName: '',
    isLoaded: false,
    lastUsed: 0,
  });

  modelCache.set('age', {
    type: 'age',
    path: ageModelPath,
    session: null,
    inputName: '',
    outputName: '',
    isLoaded: false,
    lastUsed: 0,
  });

  console.log('Model manager initialized');
}

/**
 * Retrieves a model (loads it if necessary)
 */
export async function getModel(modelType: ModelType): Promise<ModelInfo> {
  const modelInfo = modelCache.get(modelType);
  if (!modelInfo) {
    throw new Error(`Model ${modelType} not found`);
  }

  await loadModel(modelInfo);
  return modelInfo;
}

/**
 * Unloads a model from memory
 */
export function unloadModel(modelType: ModelType): void {
  const modelInfo = modelCache.get(modelType);
  if (modelInfo && modelInfo.session) {
    console.log(`Unloading model: ${modelType}`);
    modelInfo.session.release();
    modelInfo.session = null;
    modelInfo.isLoaded = false;
  }
}

/**
 * Unloads all models from memory
 */
export function unloadAllModels(): void {
  for (const modelType of modelCache.keys()) {
    unloadModel(modelType);
  }
}

/**
 * Checks whether a model is loaded
 */
export function isModelLoaded(modelType: ModelType): boolean {
  const modelInfo = modelCache.get(modelType);
  return modelInfo?.isLoaded ?? false;
}

For loading and running the models, I use the same approach with onnxruntime-react-native. After obtaining the modelInfo object via getModel:

const modelInfo = await getModel('age');

an image tensor is created:

const inputTensor = new ort.Tensor('float32', imageTensor, [1, 3, n, n]);

This tensor is fed into the model, and the output tensors are returned immediately.

const feeds = { [modelInfo.inputName]: inputTensor };
const results = await modelInfo.session.run(feeds);

The examples available in plantClassificationService.ts and diseaseSearchService.ts demonstrate how to feed embeddings, normalize them, and perform vector search using Faiss. This approach is useful not only for this app but also for any mobile project that deals with large models and vector search.

A few words about the project itself and its purpose. At first glance, it might seem like just a collection of neural networks for plants, running on mobile devices with some optimizations. In reality, it’s an experiment in transferring complex models to resource-constrained devices, managing memory efficiently, and building unified code for different types of tasks. I wrote it not only for readers but also as notes for myself.

If you’ve made it to the end of this article, it means you enjoy diving into the details and aren’t afraid of challenges. The project is open-source under the MIT license, and all code is available on GitHub, so anyone can try it out, fine-tune models, optimize performance, or suggest new features. Finally, I invite everyone interested to join as contributors: every idea, optimization, or small feature makes the project better and more interesting. Sometimes a small experiment leads to unexpected discoveries, and this project is all about experimenting and finding those discoveries.

How I Tried to Save My Google Play Developer Account — and Ended Up Making a App for Crypto

Wlad Radchenko — Mon, 19 May 2025 10:32:32 +0000

I got a surprise email from Google recently:

“Your developer account is inactive and may be closed…”

I created this account back when I was still a student, mainly to publish a couple of small side projects. Then I kind of forgot about it. Years passed. Now Google warned me I had 60 days to publish something new — or the account would be deleted. And honestly? I didn’t want to lose it. It held a little piece of my dev journey.

"Okay," I thought. "I’ll whip something up real quick. Should take, like, 10 minutes!"

And that’s how I ended up building a full open-source crypto portfolio app — with donuts and LLM generated prompts. Here’s how it all spiraled out of control.

From Quick Fix to Full-on Crypto App

My first plan? Just update an old app. But I had forgotten how to React Native. Then I figured, hey, maybe I’ll add some features. A bit of design polish. A few UX tweaks…

One weekend later, I’d rebuilt the UI from scratch, rewritten all the logic, and ended up with a neat little open-source app that builds randomized crypto portfolios and generates ChatGPT prompts for each coin.

How It Works

The app fetches crypto data from the CoinGecko API
Random coins are selected and diversified based on how much money you have
Each coin is displayed as a donut chart
You tap a button and get a prompt for ChatGPT or any LLM, like:
- What's the investment risk level for this token (low/medium/high) and why?
- Does it have good profit potential?
- Is now a good time to buy, or should I wait?
- How reliable/safe is the token based on its market cap, volume, and supply structure?
- Give a short, actionable recommendation.

And yep, the app saves your portfolio history and lets you share it with friends. I was inspired during the creation by this picture:

Under the Hood

⚠️ Warning: there's a lot of stream-of-consciousness code below. If you want to know what came out of it, scroll to the end of the article.

What was going through my head while coding:

I’ll walk you through how I built it, the traps I ran into with Expo + React Native, and whether I actually managed to save my developer account in the end.

Requirements

You’ll need:

Node.js (v14+)
Android Studio (for Android) or Xcode (for iOS)
React Native with Expo
And EAS Build for creating Android bundles
The full source is on GitHub.

If you’re a total npm newbie, don’t worry — just run:

git clone <project>
сd <project>
npm install
npm run android or npm run ios

Goodbye Old Splash Screen

The first thing I ditched was the outdated splash screen. I replaced it with a clean background and a login screen. Installed the needed packages and created a new SplashScreen.tsx.

npm install react-native-responsive-screen

And code in SplashScreen.tsx:

import { useRef } from 'react';
import { View, Text, Image, StyleSheet, TouchableOpacity, Animated } from 'react-native';
import { widthPercentageToDP as wp, heightPercentageToDP as hp } from 'react-native-responsive-screen';

const SplashScreen = ({ onHide }) => {
  const fadeAnim = useRef(new Animated.Value(1)).current;
  const scaleAnim = useRef(new Animated.Value(1)).current;

  // Animation after clicking a button
  const handleGetStarted = () => {
    Animated.parallel([
        Animated.timing(fadeAnim, {
        toValue: 0,
        duration: 500,
        useNativeDriver: true,
        }),
        Animated.timing(scaleAnim, {
        toValue: 2, // you can reduce it to 0.5 or 0.1 if you want a collapse, but I have an increase
        duration: 500,
        useNativeDriver: true,
        }),
    ]).start(() => onHide());
   };

  // Draw View  
  return (
    <Animated.View style={[styles.container, { opacity: fadeAnim, transform: [{ scale: scaleAnim }] }]}>
      <View>
        <Image 
            source={require('./assets/donuts/background.png')} 
            style={{ marginTop: hp('4.76%'), width: wp('109%'), height: hp('57.14%') }}
            resizeMode="contain">
        </Image>
      </View>
      <View style={styles.content}>
        <Text style={styles.title}>Doughfolio</Text>
        <Text style={styles.description}>
          Visualize your crypto portfolio with delicious donut charts
        </Text>
      </View>
      <TouchableOpacity style={styles.button} onPress={handleGetStarted}>
        <Text style={styles.buttonText}>Get Started</Text>
      </TouchableOpacity>
    </Animated.View>
  );
};


const styles = StyleSheet.create({
  container: {
    position: 'absolute',
    top: 0,
    left: 0,
    right: 0,
    bottom: 0,
    backgroundColor: '#FFD8DF',
    justifyContent: 'center',
    alignItems: 'center',
    zIndex: 1000,
  },
  content: {
    flex: 1,
    justifyContent: 'flex-end',
    alignItems: 'flex-start',
    paddingHorizontal: wp('7.27%'),
    marginBottom: hp('7.61%'),
  },
  title: {
    fontSize: wp('14%'),
    fontWeight: 'bold',
    color: '#FF6E76',
    marginBottom: hp('1.9%'),
    textAlign: 'left'
  },
  description: {
    fontSize: wp('5.09%'),
    color: '#FF6E76',
    textAlign: 'left',
  },
  button: {
    backgroundColor: 'white',
    borderRadius: 28,

    // Shadow for Android
    elevation: 10,

    // Shadow for iOS
    shadowColor: '#9B8084',
    shadowOffset: { width: 5, height: 5 },
    shadowOpacity: 0.1,
    shadowRadius: 10,

    paddingHorizontal: wp('14.72%'),
    paddingVertical: hp('2%'),
    width: '80%',
    marginBottom: hp('5.71%'),
    alignItems: 'center',
    justifyContent: 'center'
  },
  buttonText: {
    color: 'black',
    textTransform: 'uppercase',
    fontSize: wp('4.72%'),
    fontWeight: 'bold',
  },
});

// We set export to export SplashScreen to App.tsx
export default SplashScreen;

Feeling fancy? You can even add Lottie animations. Just grab a JSON animation from LottieFiles (you'll need an account), install Lottie, and embed the animation in your view.

npm install lottie-react-native

And integrate LottieView:

<LottieView source={require('./assets/animation.json')} autoPlay loop />

In App.tsx, I layered the SplashScreen over the main content and used splashVisible state to toggle its visibility.

export default function App() {
  const [splashVisible, setSplashVisible] = useState(true);

  const [fontsLoaded] = useFonts({
    'Roboto-Bold': require('./src/assets/fonts/Roboto-Bold.ttf'),
    'Roboto-Light': require('./src/assets/fonts/Roboto-Light.ttf'),
  });

  return (
    <View style={{ flex: 1 }}>
      <DonutChartContainer />
      {splashVisible && (
        <SplashScreen onHide={() => setSplashVisible(false)} />
      )}
    </View>
  );
}

Coin Data + Donuts

The interesting part starts in App.tsx, where the app fetches data:

// Function for getting data from CoinGecko API
async function fetchCryptoData() {
  const response = await fetch('https://api.coingecko.com/api/v3/coins/markets?vs_currency=usd&order=market_cap_desc&per_page=200&page=1');
  const data = await response.json();
  return data;
}

// Function to randomly select 10 cryptocurrencies
function getRandomCryptos(data, count = 10, maxIndex = 200) {
  const minSlice = Math.floor(Math.random() * (maxIndex - count + 1));  // from 0 to 90
  const maxSlice = minSlice + count;
  return data.slice(minSlice, maxSlice);
}

 const handleHistorySelect = (item: any) => {
  setData(item.data);
  // Updating totalValue via withTiming for smooth animation
  totalValue.value = withTiming(item.totalValue, { duration: 500 });
  // Recalculating percentages
  decimals.value = item.data.map(crypto => crypto.percentage / 100);
};

  ////////////////////////////
 // BREAKING THE TEMPLATE //
///////////////////////////

try {
  // Mix up the donut pictures
  setImages(getShuffledDonutImages());

  // Step 1: Get data from the API
  const cryptoData = await fetchCryptoData();

  // Step 2: Randomly select 10 cryptocurrencies
  const selectedCryptos = getRandomCryptos(cryptoData, 10);

  // Step 3: Generate random numbers for weight distribution
  const generateNumbers = generateRandomNumbers(n, amount);

  // Calculate the total sum of these numbers
  const total = generateNumbers.reduce((acc, currentValue) => acc + currentValue, 0);

  // Calculate percentages for each number
  const generatePercentages = calculatePercentage(generateNumbers, total);

  // Round off percentages and make them in the format 0.00
  const generateDecimals = generatePercentages.map((number) => {
    if (number != null && !isNaN(number)) {
      return Number(number.toFixed(0)) / 100;
    }
    return 0; // Yes, there can be null values from the API
  });

  totalValue.value = withTiming(total, { duration: 1000 });

  decimals.value = [...generateDecimals];

  // Generate an array of objects with data
  const arrayOfObjects = generateNumbers.map((value, index) => ({
    name: selectedCryptos[index].name,
    image: selectedCryptos[index].image,
    symbol: selectedCryptos[index].symbol,
    minPrice: selectedCryptos[index].ath,
    maxPrice: selectedCryptos[index].atl,
    price: selectedCryptos[index].current_price,
    marketCap: selectedCryptos[index].market_cap,
    marketCapChangePercentage24h: selectedCryptos[index].market_cap_change_percentage_24h,
    priceChangePercentage24h: selectedCryptos[index].price_change_percentage_24h,
    circulatingSupply: selectedCryptos[index].circulating_supply,
    maxSupply: selectedCryptos[index].max_supply,
    totalVolume: selectedCryptos[index].total_volume,
    value,
    percentage: generatePercentages[index],
    decimals: generateDecimals[index] / 100,
    color: colors[index], // Random color generation
    url: 'https://www.coingecko.com/en/coins/' + selectedCryptos[index].id,
  }));

  // Output data to the console
  setData(arrayOfObjects);
  await addToHistory(arrayOfObjects); // Save data + total amount
} catch (error) {
  console.error('Failed to data generation:', error);
}

When setData is called, we’re updating the data state inside the DonutChartContainer — which then redraws the chart.

const [data, setData] = useState<Data[]>([]);

Bonus: there’s also a useHistory.ts hook that saves every generated portfolio to AsyncStorage under the key cryptoDonutHistory. Firstly installation:

npm install @react-native-async-storage/async-storage

And code of useHistory.ts:

// src/hooks/useHistory.ts
import AsyncStorage from '@react-native-async-storage/async-storage';
import { useState, useEffect } from 'react';

interface HistoryItem {
  date: string;
  data: any[];
  totalValue: number; 
}

export const useHistory = () => {
  const [history, setHistory] = useState<HistoryItem[]>([]);

  const loadHistory = async () => {
    try {
      const saved = await AsyncStorage.getItem('cryptoDonutHistory');
      if (saved) {
        const parsed = JSON.parse(saved);
        setHistory(parsed);
      }
    } catch (e) {
      console.error('Failed to load history', e);
    }
  };

  interface HistoryItem {
    date: string;
    data: {
      name: string; 
      value: number; // Price in $
      percentage: number;
      color: string;
      image: string;  // Image for coin
      symbol: string; //  Symbol of coin
    }[];
    totalValue: number;
  }

  const clearHistory = async () => {  // Clear AsyncStorage for cryptoDonutHistory
    try {
      await AsyncStorage.removeItem('cryptoDonutHistory');
      setHistory([]);
      return true;
    } catch (e) {
      console.error('Cleaning error:', e);
      return false;
    }
  };

  const addToHistory = async (newData: any[]) => {  // Added data in AsyncStorage
    const totalValue = newData.reduce((sum, item) => sum + item.value, 0);
    try {
      const newItem = { 
        date: new Date().toLocaleString(),
        data: newData,
        totalValue,
      };
      const updatedHistory = [newItem, ...history];
      await AsyncStorage.setItem('cryptoDonutHistory', JSON.stringify(updatedHistory));
      setHistory(updatedHistory);
    } catch (e) {
      console.error('Failed to save history', e);
    }
  };

  useEffect(() => {
    loadHistory();
  }, []);

  return { history, addToHistory, clearHistory };
};

Localized Number Formatting

In the US, we write numbers like $324,654,765. But in many other countries, the comma is a decimal separator. Which means your users might read that as $324 and some cents. Not ideal.

To fix this, I formatted all numbers using the device’s locale — so users see values in the format they expect.

export const formatNumber = (value, { isCurrency = false, currency = 'USD', minimumFractionDigits = 0, maximumFractionDigits = 2 } = {}) => {
  if (value == null || isNaN(value)) return '∞';

  return value.toLocaleString(undefined, {
    style: isCurrency ? 'currency' : 'decimal',
    currency,
    minimumFractionDigits,
    maximumFractionDigits,
  });
};

// Also uses safeToFixed to safely round numbers.
export const safeToFixed = (value, decimals = 2) => {
  if (isNaN(value) || value == null) {
    // If the value is not a number or null, return a string with the default value
    return '0.00';
  }
  return value.toFixed(decimals);
}

The Finishing Touches

With 4 hours left before midnight, I focused on the visuals. I found the tastiest-looking donut images on Vecteezy (free license), designed some Play Store slides in Corel Vector, and uploaded everything.

I hit “Submit” just after midnight, watched the build go through, and went to bed thinking I was safe from the next inactivity purge for a while. Here’s the app on Google Play if you're curious.

But Then, Google Strikes Again

Next morning?

“Your developer account may be at risk due to inactivity.”

Come on, Google! I just published an app! I followed the rules!

In the Play Console, the message basically said:

Confirm your email and phone number (✅ done)
Publish a new app or release an update (✅ also done)

So yeah. Either they didn’t register my app as “active” yet, or I’ll need to wait and see if the red banner disappears. But hey — I now have:

An open-source project I’m proud of
A maybe-doomed dev account
And a ton of ideas for the next version

Thinking of adding a map with donut shops, built-in LLM chat, and even more data in the future. I'm planning how I'll implement new features:

Let’s Trade Stories

Got your own horror story about Google Play or dev accounts going rogue? Drop it in the comments. Share your thoughts, join the open-source repo — and if your story’s spicy enough, I owe you a donut.

GitHub repo: https://github.com/wladradchenko/doughfolio.wladradchenko.ru
Google Play: https://play.google.com/store/apps/details?id=com.wladradchenko.donut
Made with: ☕, 🍩, and 😅

Why My Open Source Project Wunjo Can’t Reach 1K Stars? 😢

Wlad Radchenko — Tue, 25 Mar 2025 08:09:50 +0000

"Build it, and they will come..." – They said.
"Release an awesome AI-powered tool, and the stars will rain down on you!" – They said.

P.S. Now there is no trust in articles and posts about how to earn a lot of stars on GitHub and that a beautiful README matters a lot. All of this is fiction that does not work today. This myth was 💥 Busted!

Well, here I am, stuck at 979 stars 😩, and that number has barely moved for weeks.

I’ve been building Wunjo, an Open Source AI-powered video editing tool that can today automatically cut, highlight, and transform videos with a simple text prompt. Sounds cool, right? Yet, getting to 1K stars on GitHub feels like an endless grind. This is a set of tools in software to optimization process of video, photo editing and API (API Docs) inside for other pet-projects.

I don’t enjoy manual video editing—it takes forever. Finding the right cuts, trimming, syncing audio… it’s exhausting. That’s why I built Wunjo:

✅ Upload any video (even hours long!)
✅ Describe what you want: "Make a highlights reel," "Cut out dull moments," "Find all goals & red cards"
✅ AI does the rest – detects key moments, edits them together, and delivers a polished video. You also can settings duuration, for example to get shorts for YouTube or TikTok.

Wunjo can even analyze specific fragments to answer:
❓ "What’s happening in this scene?"
❓ "What are people wearing?"
❓ "How is the light falling?"

Future updates might allow editing videos based on sound, for making interview shorts, news clips, and more.

But… Where Are My Stars?

I’m constantly improving the project, adding features, optimizing performance, and sharing updates. But that 1K star milestone just keeps teasing me. Only 21 stars left, yet it’s been like this for WEEKS.

So, fellow devs, I need your wisdom:

💬 How do you promote your Open Source projects?
💬 Have you struggled with GitHub star plateaus?
💬 Any tips for getting more eyes on a project?

Let’s talk! And, well… if Wunjo sounds useful, maybe drop a star? ⭐😉

👉 GitHub Repo and Patreon.

P.S. Also, check out the project pages on Product Hunt, Product Radar, and DevHunt.

Your coding year in review

Wlad Radchenko — Mon, 09 Dec 2024 18:03:50 +0000

Exploring the internet, I stumbled upon GitHub Unwrapped—a fascinating tool that automatically creates a video based on your GitHub activity over the past year. It showcases your most-used languages, those spontaneous late-night coding sessions, stars earned, and more. Plus, it’s open source.

This gem is built using Remotion, another open-source project that enables automating video creation with React for web. The documentation is good, but you need to figure it out. I thought it was so cool that I had to share it n dev.to!

At the end of the video, I added a little extra—an update about my own open-source pet project, which contributed heavily to my stats and the video itself. Fun fact: Wunjo was recently named Product of the Week on Product Radar with record number votes from Makers!

DevHunt Weekly: Unfair Rankings or Just a Glitch?

Wlad Radchenko — Tue, 26 Nov 2024 07:40:44 +0000

Last week, I launched my open source project, Wunjo Community Edition CE, on DevHunt.

I’m reflecting on the results of this week’s DevHunt in this post, as it’s a platform where open-source projects and tools for developers are launched and compete. Since Dev.to is a place where people often share about development, useful resources, and tools for IT professionals, I believe many of you might consider launching your projects on DevHunt. I want to share my disappointing experience so you can be aware of potential pitfalls.

I already wrote thoughts about it, the positive and negative sides that I felt as a user, you can read them on dev.to.

The voting lasted all week, and I kept a close eye on the rankings. With just 12 hours to go (Voting ends on Monday), I received an email stating that my project was in second place, while another project, WebDraw on DevHunt, was down at 10th place. The position of the project in the letter indicates its place in the current voting by upvote.

Two hours before the voting ended, I checked again—Wunjo was on second rank, and WebDraw wasn’t even in the top five. However, when the results were announced, I was shocked to find that WebDraw had surged into third place, while Wunjo was bumped to fourth.

Here’s where things don’t add up

Votes and Impressions Tell a Different Story

Both Wunjo and WebDraw have the same number of upvotes, but the impressions for Wunjo were 2896, compared to WebDraw's 1362. Impressions, as I understand them, are based on project views and retention in the top list, meaning they reflect actual engagement. This discrepancy raises serious questions about the legitimacy of WebDraw’s sudden surge.

Timing of the Shift

The bulk of WebDraw’s movement happened in the last two hours of the voting period. For a project that had little activity throughout the week to suddenly leapfrog multiple projects at the finish line—it just doesn’t feel organic.

The Notification That Confused Me Further

After the results were announced, I received an email with the subject "Celebrating Your Remarkable Third Place Win on DevHunt!" (screenshot attached). But when I checked the platform, there was no badge, no acknowledgment of third place—just disappointment.

I even reached out to DevHunt founder John Rush on X (formerly Twitter) but haven’t received a response yet.

Disclaimer: This post is not a call to avoid launching your projects on DevHunt—I’m simply reflecting on my experience and sharing what happened during my recent launch. I hope this sparks a discussion and maybe even some much-needed change.

Why This Matters

This experience left me disheartened, not just because Wunjo didn’t receive the recognition it deserved, but because this undermines the transparency and trust that platforms like DevHunt rely on. Even if the situation gets corrected, the moment for social sharing, celebrating, and engaging with the community has already passed. Those who followed the voting will only remember the unfair results.

It’s not about winning—it’s about fairness.

My Questions to the Community

Have you ever experienced something similar? How did it make you feel?
What do you think about platforms like DevHunt? Should they improve transparency in their voting processes?

Let’s discuss! I’m curious to hear your thoughts and ideas.

For me, this has been a lesson in resilience. Despite the disappointment, I still believe in Wunjo and its potential. I hope this post helps bring awareness to the importance of fair competition, especially in spaces meant to uplift and celebrate creators.

Looking forward to your thoughts and stories!

Update: Justice Restored!

I’m happy to share that fairness has prevailed—Wunjo is now officially among the top three projects on DevHunt! 🏆🎉

Thank you to the Dev.to and DevHunt communities, as well as John Rush, for addressing this issue. Your support means a lot, and it gives me hope that transparency and fairness are valued on platforms like this.

Let’s continue building a positive and supportive environment for creators!

How Else Can You Promote Your Open Source Product?

After launching on Product Hunt, I saw my first sales and got a wave of new users — definitely worth it. But surprisingly, Product Radar brought in even more traffic and conversions, and it was way simpler to launch there — no waiting for a feature spot or special scheduling. Beyond that, platforms like BetaList and AppSumo can also give your product strong early exposure and help you grow your user base fast.

Automate Saving the Planet... Or Just Your Computer's Energy 🐍

Wlad Radchenko — Tue, 19 Nov 2024 16:42:50 +0000

I came across @oliverbennet is post "What's a Quickie Post?" and couldn’t resist giving it a try Quickie Post myself. So, here’s a Python twist on a global problem:

Problem: Fighting climate change is a serious challenge, but did you know Python is playing a role in it?

Imagine you’re writing code on Python to help reduce energy waste:

import os

def save_energy():
    if os.name == "nt":  # Windows
        os.system("shutdown /s /t 1")
    elif os.name == "posix":  # macOS/Linux
        os.system("pmset sleepnow")
    else:
        print("Unsupported OS! 🌍💻")
    print("Computer shutting down to save energy! 🌱")

# Save the planet
save_energy()

Great, let's try something more sophisticated. We’re all aware that high CPU or GPU usage translates to more energy consumption. So, this Python script uses APScheduler to periodically check system load, and if it’s too high, it shuts down or puts the computer to sleep. 🌍

import os
import psutil  # For monitoring CPU usage
import GPUtil  # For monitoring GPU usage
from apscheduler.schedulers.blocking import BlockingScheduler

def save_energy():
    """Shutdown or put the system to sleep if CPU or GPU load is too high."""
    print("Energy-saving monitor started! 🌿")
    if os.name == "nt":  # Windows
        os.system("shutdown /s /t 1")
    elif os.name == "posix":  # macOS/Linux
        os.system("pmset sleepnow")
    else:
        print("Unsupported OS! 🌍💻")
    print("Computer shutting down to save energy! 🌱")

def check_system_load():
    """Check CPU and GPU load and trigger save_energy if either is too high."""
    # Check CPU load
    cpu_load = psutil.cpu_percent(interval=1)
    print(f"Current CPU Load: {cpu_load}%")

    # Check GPU load
    gpus = GPUtil.getGPUs()
    if gpus:
        gpu_load = max(gpu.load * 100 for gpu in gpus)  # Get max load of all GPUs
        print(f"Current GPU Load: {gpu_load}%")
    else:
        gpu_load = 0
        print("No GPUs detected.")

    # Trigger energy-saving if CPU or GPU load is too high
    if cpu_load > 90 or gpu_load > 90:  # Adjust thresholds as needed
        print("High system load detected! Initiating energy-saving protocol... 🚨")
        save_energy()

# Set up the scheduler
scheduler = BlockingScheduler()
scheduler.add_job(check_system_load, 'interval', minutes=1)  # Check every minute

How can we save the planet in other programming languages? Let's discuss!

Exploring dev.to's Hidden Features: Upload a Video. What?

Wlad Radchenko — Tue, 19 Nov 2024 16:06:24 +0000

I decided to dive deeper into dev.to 🛠️ and clicked on the Dashboard, where I was surprised to see a button labeled "Upload a video." 😮 Curious, I couldn't resist testing it out by uploading one of my own videos from this dev.to post 🎥 to see what happens next.

Don't be hard on me, I don't have any other videos.

By the way, the video showcases my open-source project. 🖥️ I created the facial effects, motion, and generation entirely with my neural network-powered program 🤖 (aside from the text and video cuts). Now I’m left wondering—where exactly will this video show up? 🤔

Does anyone use the "Upload a video" feature on dev.to and have thoughts on how it should shape the platform’s future?

Should it be more like, “Hey, I’m scrolling through tech and news videos to unwind and kill time,” or should it be a serious space for learning with tutorials and deep dives into open-source tools and projects? Write in the comments your opinion on what dev.to will be like in the future. And who is the ideal publisher now.

[ROAST] Launching on DevHunt Doesn't Work? Here's What I Found!

Wlad Radchenko — Tue, 19 Nov 2024 11:08:07 +0000

Let's talk about DevHunt. As a platform for launching open-source projects, I decided to give it a spin and see how it performs. Here are some of its key features: Founded over a year ago by John Rush (more details on dev.to), DevHunt aims to help open-source projects share their stories. If you want a free launch, your project will go live in six months. For a faster release, you’ll need to pay, or you can sponsor the platform to always appear in the top banner.

By the way, the platform itself has open source code on GitHub ❤️.

The concept of a delayed launch seems to be based on the idea that founders of new projects will use that time to connect, build a community, and gather support contacts on LinkedIn and X. This one-on-one interaction should ideally happen within the platform itself, since it’s meant to be a community for developers, rather than outsourcing engagement to other infrastructures like LinkedIn and X.

But here’s the catch: It doesn’t work! Here's why, based on my experience with launches on the best plarform Product Hunt (which I’m still exploring) and new cool alternative platform, Product Radar, as well as three weeks of observing DevHunt.

For a platform boasting “100k+ developers found Dev Tools here”, the activity is shockingly low.

Engagement is minimal, and when you comment on a project, your comment is likely to go unanswered. In fact, I couldn’t even find a way to reply to comments.

Many people seem to forget about their launches, and the platform doesn’t even send reminders. On the day of my launch or one week before, I only received updates about the projects that won last week. (Today is November 19, 2024)

In the evening, if you have account X linked, you will be indirectly notified about the launch via a post from the DevHunt channel during their roundup of newly launched projects. For example.

One thing I genuinely appreciate about DevHunt is that even with a six-month wait for a launch, the interface and functionality of this open-source project have evolved. And DevHunt does let you edit your description—well, sort of. There’s a quirky bug: If you update the description, save it, then change the image, your project card might still display the old description.

Weird, but okay! Just tweak the sequence, and it works.

Speaking of winners, the "Past winners 👑" section doesn’t actually show last week’s launches but rather all previous winners, making it hard to track real activity.

Real activity is clear if you go and look at the projects every week, or by mailing.

There’s also a lack of subcategories for search and no smart search feature (like a bot) to help you find specific tools, like a video generator. And what’s the purpose of projects on DevHunt anyway? Is it to recruit team members, find investors, share your story, or just get your project roasted? I wanna see those target of project as category in search.

Also, what does the "Impressions" badge mean? Hovering over it doesn’t explain anything.

I also like the user activity transparency. You can see which projects a user has supported or commented on. That’s a real advantage for account owners who want to revisit tools they’ve favorited. Speaking of favorites, though, there’s one big gap: there’s no dedicated "Favorites" tab. Since I can’t upvote projects after their launch period ends, how do I bookmark them for later? There’s just no real incentive to keep me coming back to DevHunt.

This user activity feature could also be super valuable for thought leaders in fields like AI. Anyone could easily check which projects these leaders are interested in and use in their work.

Honestly, I think DevHunt has huge potential if developed further. Imagine video breakdowns of projects, behind-the-scenes stories of launches, or how DevHunt impacts projects over time (for example on DevHunt or Patreon). Maybe even microblogs for each project, where teams can share updates. This would only work, though, if the platform has solid user retention and good SEO, so every update makes it easier to find these tools.

And hey, why not add gamification through achievements to keep people engaged, instead of just being a platform for cold email blasts?

And let’s not forget: it’s a free and open platform, and that’s pretty awesome.

Despite the issues, I still like the idea of a fresh community for developer tools. And I wish John Rush his DevHunt development, the main thing is not to abandon the project, but to make it the best tool. Thank you.

Will I keep exploring new platforms? Absolutely!

Practical Experience: Integrating Over 50 Neural Networks Into One Open-Source Project

Wlad Radchenko — Wed, 13 Nov 2024 09:44:43 +0000

A year and a half ago, I embarked on an open-source project that has since grown and evolved significantly. Inspired by the AUTOMATIC1111 project, which was just starting to gain traction at the time, I kept adding new features and capabilities. Today, my project integrates over 50 different neural networks, each handling a unique task. In this article, I want to share some practical tips and key takeaways from my journey. I hope they prove helpful to you and motivate you to refactor your code.

My open-source project focuses on creating and editing video, images, and audio using neural networks. Often, different methods can achieve similar outcomes, but ensuring consistency across the project has been a major challenge. As I integrated open-source solutions, optimized them, and added new functionality, maintaining a unified approach became essential. For instance, features like face swapping, lip synchronization, and portrait animation all require facial recognition. Rather than using separate methods for each, as was common in the original solutions, I opted for a single, shared model for facial recognition. Consequently, the 50+ neural networks are organized such that each one serves a unique purpose without redundancy.

During development, I made a key decision: to avoid TensorFlow and any related frameworks, focusing solely on PyTorch and ONNX Runtime.

For those curious about the specific features or the neural networks I used, I have included several links: a YouTube playlist documenting the project's evolution and a short video created using my software.

Each model in this project is diverse and complex, performing tasks like image and video generation, facial recognition, segmentation, and much more. There are no simple solutions; every neural network fulfills a distinct role.

Let’s dive into the insights

Disclaimer: I am sharing my personal experience, the life hacks I use, and the challenges I've encountered during development. Since libraries and frameworks are constantly being updated, please ensure that you verify compatibility and check for any changes in the latest versions. The tips provided here reflect my approach at the time of writing and may require adjustments as new updates and features become available.

Tip 1: One Model is One Task

One of the first surprises I encountered was that you can’t load a single model into VRAM and use it simultaneously for multiple tasks. Each model must be loaded separately for its specific task. This realization laid the groundwork for many of the strategies I developed later on.

Tip 2: Managing a Task Queue

My application is built on Flask, which means users don’t have to wait for tasks to finish processing. They can initiate multiple tasks simultaneously, potentially putting a heavy load on memory. To prevent memory overflow, I introduced artificial delays between task executions, with random intervals to minimize the chances of multiple tasks starting at the same time. This approach also ties into Tip 3.

Tip 3: Monitoring Memory Usage

Before launching a task, I measure the available memory on the device. If the memory falls below the threshold needed for the model, I intentionally delay the task execution. This proactive approach helps ensure that the system remains stable and tasks don’t fail due to insufficient memory.

import torch
import psutil

def get_vram_gb(device="cuda"):
    if torch.cuda.is_available():
        properties = torch.cuda.get_device_properties(device)  # Get the values for a specific GPU, which is our device
        total_vram_gb = properties.total_memory / (1024 ** 3)
        available_vram_gb = (properties.total_memory - torch.cuda.memory_allocated()) / (1024 ** 3)
        busy_vram_gb = total_vram_gb - available_vram_gb
        return total_vram_gb, available_vram_gb, busy_vram_gb
    return 0, 0, 0


def get_ram_gb():
    mem = psutil.virtual_memory()
    total_ram_gb = mem.total / (1024 ** 3)
    available_ram_gb = mem.available / (1024 ** 3)
    busy_ram_gb = total_ram_gb - available_ram_gb
    return total_ram_gb, available_ram_gb, busy_ram_gb

Tip 4: Handling “CUDA Out of Memory” Errors

In addition to delaying task execution, I implemented checks for the most common error: “CUDA out of memory.” The solution is straightforward: if this error occurs, the system clears unnecessary data from memory and retries the process. This approach ensures that tasks can still complete successfully, even under memory constraints.

min_delay = 20
max_delay = 180
try:
    # Launch the method with a neural network
except RuntimeError as err:
    if 'CUDA out of memory' in str(err):
        # Clear memory
        sleep(random.randint(min_delay, max_delay))
        # Clear memory again
        # Launch the method again
    else:
        raise err

Tip 5: Organizing Backend Modules
The backend of my application is organized into modules, categorized by specific properties: altering videos or images, generating videos or images, and modifying audio. Each module is also classified based on whether it handles frontend or backend tasks. Models that need to provide immediate results to users, such as segmentation, txt2img, and img2img, are prioritized differently from those that process larger, time-intensive tasks in the background.

Frontend models, like those used with:

await ort.InferenceSession.create(MODEL_DIR).then(console.log("Model loaded"));

...are not part of this backend task management. As a result, I have to preload models into memory for quick response times, ensuring that different users don’t simultaneously access the same model (as discussed in Tip 1). Additionally, these preloaded models are reserved for tasks requiring rapid feedback, and they are not used for long-running processes, to avoid violating the constraints outlined in Tip 1.

Tip 6: Managing Memory-Intensive Models

Models designed for long-running tasks can be highly demanding, often consuming all available VRAM. From an optimization standpoint, frequently loading and unloading such models is inefficient, though sometimes necessary. To mitigate this, I use a strategy involving "micro models" — lightweight models that take up less memory but still require time for loading and unloading.

When processing tasks, we group them based on the method’s processing duration. Tasks from the same group are handled using these micro models, forming a queue before loading into a larger, memory-intensive model. Remember Tips 3 and 4? We have two strategies: estimating memory usage before loading the model or launching the model and handling a "CUDA out of memory" error.

Need to clear ~~cash~~ cache!

When we encounter this error, we clear VRAM of unnecessary models, including those used for rapid responses, and clean up any residual data. This approach ensures that memory-intensive models can run efficiently without disrupting other tasks.

if torch.cuda.is_available():  # If CUDA is available, because the application can work without CUDA
    torch.cuda.empty_cache() # Frees unused memory in the CUDA cache
    torch.cuda.ipc_collect() # Performs garbage collection on CUDA objects accessed via IPC (interprocess communication)
gc.collect() # Calls Python's garbage collector to free memory occupied by unused objects

Tip 7: Clearing Memory After Task Completion

After each task is completed, it's crucial to free up memory by removing variables and unloading models that are no longer needed. This can be done using:

del ...

This practice helps maintain efficient memory usage and prevents unnecessary VRAM and RAM consumption, ensuring the system stays optimized for subsequent tasks.

Tip 8: Layer-Wise Model Loading

To manage limited VRAM, models can be loaded layer by layer, distributing them between the GPU and CPU or even across multiple GPUs. However, all components of a single layer must reside on the same GPU. This method is particularly useful for tasks like image and video generation but can also be applied to other resource-intensive processes. By strategically loading models in this manner, you can maximize memory efficiency while still enabling complex operations.

device_map = {
    'encoder.layer.0': 'cuda:0',
    'encoder.layer.1': 'cuda:1',
    'decoder.layer.0': 'cuda:0',
    'decoder.layer.1': 'cuda:1',
}
# Or
device_map = {
    'encoder.layer.0': 'cuda',
    'encoder.layer.1': 'cpu',
    'decoder.layer.0': 'cuda',
    'decoder.layer.1': 'cpu',
}

Tip 9: Memory Optimization Techniques
Don’t forget to use enable_xformers_memory_efficient_attention() if your model's pipeline supports it. This method can significantly reduce memory usage. Additionally, there are other optimization techniques detailed in the documentation, such as enable_model_cpu_offload(), enable_vae_tiling(), and enable_attention_slicing(). In my project, these methods are especially useful for tasks like video restyling. However, for video generation, I rely on different, more specialized optimization strategies.

if vram < 12:
    pipe.enable_sequential_cpu_offload()
    print("VRAM below 12 GB: Using sequential CPU offloading for memory efficiency. Expect slower generation.")
elif vram < 20:
    print("VRAM between 12-20 GB: Medium generation speed enabled.")
elif vram < 30:
    # Load essential modules to GPU
    for module in [pipe.vae, pipe.dit, pipe.text_encoder]:
        module.to("cuda")
    cpu_offloading = False
    print("VRAM between 20-30 GB: Sufficient memory for faster generation.")
else:
    # Maximize performance by disabling memory-saving options
    for module in [pipe.vae, pipe.dit, pipe.text_encoder]:
        module.to("cuda")
    cpu_offloading = False
    save_memory = False
    print("VRAM above 30 GB: Maximum speed enabled for generation.")

Tip 10: Efficient Frame Handling

Storing frames in memory can be a double-edged sword. On powerful machines with constraints on resolution or content duration, keeping everything in memory can be beneficial for speed. However, many users of my project run it on lower-end devices, often processing hour-long, high-resolution videos. To accommodate this, I rewrote all methods to work with the current frame and values, saving data to the hard drive rather than keeping it in memory. By accessing data as needed and storing only file references in a list, I managed to make the process more efficient and hardware-friendly. Additionally, using generators or chunked processing helps manage large datasets, a strategy I leverage in modules like face replacement.

Tip 11: Frame Resolution Adjustments

Depending on the model, I sometimes need to resize frames to dimensions that the user's device can handle. After processing, I restore the frame size using basic resizing techniques or more advanced upscaling methods. This step is crucial for ensuring compatibility across a wide range of hardware setups.

Tip 12: Are Models Always Synchronous?

This statement isn’t set in stone, as the world of AI is ever-evolving, but here’s my experience: I haven’t seen significant benefits from using asynchronous methods with models. The exceptions are data processing operations not directly related to the model and requests for downloading or validating model versions. Otherwise, models operate synchronously, and that's been sufficient for most scenarios I’ve encountered.

Tip 13: Library Version Compatibility

Managing library versions, especially for packages like torch, torchvision, torchaudio, and xformers, is critical. Here’s how to ensure everything works seamlessly:

Check Your CUDA Version

Run:

   nvcc -V

Then visit the PyTorch download page to understand version compatibility. For instance, if your CUDA version is 11.8 (cu118), remember that it can support older versions of torch. Even CUDA 12.6 can work with a torch version designed for cu118.

Align Library Versions

Typically, torch and torchaudio share the same version (e.g., 2.4.1), while torchvision may differ (e.g., 0.19.1). You can infer version compatibility, like torch 2.2.2 with torchvision 0.17.2. Understanding these dependencies is essential.

Additionally, you can download .whl files from official sources and unpack them manually or by pip. This step is crucial for my project because it installs via an installer. For Windows users, it fetches torch, torchaudio, and torchvision according to their selected options, displaying download status before unpacking.

Check xformers Compatibility

Visit the xformers GitHub repo to ensure compatibility with your torch and CUDA versions. Support for older versions can be dropped, so staying updated is vital, especially if you're running CUDA 11.8 and want to leverage xformers for limited VRAM.

Optional: Flash-Attn Installation

Flash-attention can boost performance, and you can install it efficiently using:

   MAX_JOBS=4 pip install flash-attn

Adjust the number of jobs to suit your setup. Here’s how I use it:

try:
    from flash_attn import flash_attn_qkvpacked_func, flash_attn_func
    from flash_attn.bert_padding import pad_input, unpad_input, index_first_axis
    from flash_attn.flash_attn_interface import flash_attn_varlen_func
except ImportError:
    flash_attn_func = None
    flash_attn_qkvpacked_func = None
    flash_attn_varlen_func = None

Tip 14: Ensuring CUDA is Available for ONNX Runtime

To verify CUDA support in ONNX Runtime, run this code:

access_providers = onnxruntime.get_available_providers()
if "CUDAExecutionProvider" in access_providers:
    provider = ["CUDAExecutionProvider"] if torch.cuda.is_available() and self.device == "cuda" else ["CPUExecutionProvider"]
else:
    provider = ["CPUExecutionProvider"]

For CUDA 12.x, unlike version 11.8, you’ll need to install cuDNN 9.x on Linux (though this might not be necessary on Windows). Be cautious: sometimes onnxruntime-gpu installs without CUDA support. Once you ensure your torch version is CUDA-compatible, it's a good idea to reinstall onnxruntime-gpu:

pip install -U onnxruntime-gpu

Tip 15: Handling Library Version Conflicts

What if some models work only with older libraries, while others need the latest ones? I faced this in gfpganer, which required an old torchvision version, but video generation needed new torch libraries. Here’s how I solved it:

try:
    # Check if `torchvision.transforms.functional_tensor` and `rgb_to_grayscale` are missing
    from torchvision.transforms.functional_tensor import rgb_to_grayscale
except ImportError:
    # Import `rgb_to_grayscale` from `functional` if it’s missing in `functional_tensor`
    from torchvision.transforms.functional import rgb_to_grayscale
    import types
    import sys

    # Create a module for `torchvision.transforms.functional_tensor`
    functional_tensor = types.ModuleType("torchvision.transforms.functional_tensor")
    functional_tensor.rgb_to_grayscale = rgb_to_grayscale

    # Add this module to `sys.modules` so other imports can access it
    sys.modules["torchvision.transforms.functional_tensor"] = functional_tensor

This approach imports the modified methods from newer versions, ensuring compatibility across different models and libraries.

Tip 16: Watch Out for Warnings

Always pay attention to warnings in your project. These often hint at breaking changes in future library versions. Proactively address these warnings by updating or adding parameters as needed, preventing inconsistencies when you eventually upgrade. Keeping your codebase in sync with evolving libraries is crucial for long-term stability.

A reaction to console messages Warning from Torch:

Tip 17: GPU Management in a Cluster

When working with a cluster of multiple machines, remember that you can't combine the VRAM from separate GPUs. However, if your GPUs are on the same local network, libraries like Ray allow centralized GPU management from a single controller. Note: VRAM summing doesn't work except on a single machine with multiple GPUs. Here, techniques from Tip 8 apply, but VRAM still isn't cumulative across devices.

Tip 18: Model Compilation with `torch.jit`

Using torch.jit to compile models can greatly speed up execution. Try torch.jit.trace() or torch.jit.script() to convert your model into an optimized format, ideal for repeated calls:

import torch

# Tracing a model example
model = ...  # your model
example_input = ...  # an input sample for the model
traced_model = torch.jit.trace(model, example_input)

# Use traced_model for faster execution
output = traced_model(example_input)

This method shines when the same model is used repeatedly across various tasks.

Tip 19: Profiling for Performance Optimization

Tools like torch.profiler are invaluable for pinpointing bottlenecks in your model's performance. By profiling, you can see which operations consume the most time or memory and adjust your code for efficiency:

import torch
from torch.profiler import profile, record_function

with profile(profile_memory=True) as prof:
    with record_function("model_inference"):
        output = model(input_data)

print(prof.key_averages().table(sort_by="cuda_time_total", row_limit=10))

This helps allocate resources better and focus on optimizing the right sections of your code.

A Heartfelt Conclusion

And there we have it: 19 tips to supercharge your neural network projects! But I believe there's room for one more: your Tip 20. Drop your favorite optimization or development trick in the comments to complete this list together!

I have a dream: to see 4,096 stars on my GitHub project. Your support fuels my passion, drives me to improve code, develop new techniques, and share my experiences. If my work has been helpful, please star the project. Your encouragement means the world and inspires me to keep creating. And don’t forget to share your neural network projects on GitHub in the comments! 🖐

My Journey to dream: 4096 ⭐ on GitHub

Wlad Radchenko — Mon, 04 Nov 2024 11:51:20 +0000

I have a dream — to reach 4096 ⭐ on GitHub! To make it happen, I'm developing an open-source project called Wunjo. I'm constantly improving it and coming up with new features. This program enables you to create and edit videos, images, and audio using neural networks. It already has over 50 integrated models full model list that work together within a single app.

I've been working on this project solo for over a year now. I create video tutorials on how to use the app, and I’m always responsive to Issues and Discussions. I’ve also started a Boosty page for the app and a Telegram channel to gather even more feedback.

Wunjo is built for content creators, editors, designers, and photographers. With this app, you can generate videos, restyle them, animate portraits, swap faces, animate lips, remove objects and backgrounds, clone speech, and enhance content quality. I've drawn on various approaches from GitHub, blending and optimizing them, and replacing parts of the code to improve performance. Soon, I'll be launching on DevHunt (14 November), and I would love your support.

Additionally, I've already launched and shared my project on platforms like Product Hunt and Product Radar. Interestingly, the latter turned out I think will be even more effective for product promotion in future, so I highly recommend checking it out.

If you have any ideas or experience on how to get more than four thousand stars for a project, please write it in the comments. I will be very grateful to you.

And some review about how it works...

Join me on GitHub on this journey to my dream!