Forem: tedTecch

Integrate computer vision with flutter (using Google Teachable)

tedTecch — Sun, 26 Mar 2023 18:31:10 +0000

Introduction

In recent years, computer vision AI has become increasingly popular in the development of mobile applications. With computer vision AI, mobile apps can perform a variety of tasks, such as image recognition, object detection, emotion detection and facial recognition. Google Teachable is a platform that allows users to create and train their own machine learning models, without requiring any programming knowledge. By integrating computer vision AI with Google Teachable in a Flutter app, developers can create intelligent mobile applications that can recognise and classify images with high accuracy.

In this article, we will explore the process of integrating computer vision AI with Google Teachable in a Flutter app. We will discuss the steps required to create and train a machine learning model using Google Teachable, and how to export it as a TensorFlow Lite model that can be used in a Flutter app. We will also demonstrate how to use the TensorFlow Lite interpreter to perform image(fruit) classification in a Flutter app, and how to build a user interface that captures and processes images.

Finally, we will discuss the benefits of using computer vision AI in mobile apps, and the potential applications of this technology in the future.

Getting Started

In this section, we will discuss how to integrate computer vision AI into a Flutter app using Google Teachable. Specifically, we will cover the initial steps of creating a new Flutter project and installing the necessary packages and dependencies.

Creating a New Flutter Project
Before we can begin integrating computer vision AI into our Flutter app, we need to create a new Flutter project. This can be done using the following command in the terminal:

flutter create <project-name>

This command will create a new Flutter project with the specified name. Once the project is created, navigate to the project directory using the cd command in the terminal.

Installing Necessary Packages and Dependencies
Now that we have created our new Flutter project, we need to install the necessary packages and dependencies for integrating computer vision AI. The packages required for this integration are:

camera: This package provides access to the device's camera, which we will need for capturing images for computer vision AI analysis.
tflite: This package provides support for TensorFlow Lite models, which are used for machine learning and computer vision AI.

To install these packages, add them to the pubspec.yaml file in the project directory and run the following command in the terminal:

flutter pub get

Integrating Google Teachable with Flutter App

Google Teachable is a platform that allows developers to train custom machine learning models without writing any code. With Google Teachable, we can easily create and train machine learning models using a simple drag-and-drop interface. This makes it an excellent tool for adding computer vision AI capabilities to our Flutter app.

Setting up a Google Teachable Account
To get started with Google Teachable, we need to create an account. To do this, visit https://teachablemachine.withgoogle.com/ and sign in with your Google account. Once you have signed in, you will be taken to the Teachable Machine dashboard where you create a new image project just like the image below:

Creating a Custom Machine Learning Model in Google Teachable
Now that we have set up our Google Teachable account and a new project, we can create a custom machine learning model. For the purpose of this tutorial, we will be creating an image classification model that can identify different fruits(strawberries, oranges, apples).

To create the model, we will use the Teachable Machine webcam feature to capture images of different fruits. The images of these fruits would be in separate folders or class, you can do this by renaming the folders(class) according to the category of each fruit(strawberries, oranges, apples). Alternatively, we could upload images of the fruits we want to classify.

Once we have captured enough images for each fruit(300 images in this case for each category), we can train the model by clicking on the Train Model button. This will train the machine learning model and generate a classification model and a set of classification labels. The training can take some time so be patient till its done.

Exporting the Model as a TensorFlow Lite Model
After training the model, we need to export it as a TensorFlow Lite model. To do this, we can select the TensorFlow Lite export option from the Export Model dropdown menu, then click the download model button. This will generate a TensorFlow Lite model file that we can integrate into our Flutter app.

Integrating the TensorFlow Lite Model in Flutter App
Now that we have exported the TensorFlow Lite model from Google Teachable, we can integrate it into our Flutter app. To do this, we need to unzip the downloaded file and copy the model and label file to our assets folder in our flutter app.

Once we have loaded the TensorFlow Lite model in our Flutter app, we can use it to perform image classification on images captured from the device's camera using the camera package. We can then display the classification results in the Flutter app UI.

Implementing Computer Vision AI in Flutter App

Using TensorFlow Lite Interpreter in Flutter app to perform image classification
First, we need to initialize our camera in Flutter. We do this by adding a CameraController to our initState() method. We set up the camera using the availableCameras() method from the camera package. We then set the back-facing camera as our default camera and initialize the camera using the initialize() method. We start the image stream from the camera using the startImageStream() method, which is called when the camera is initialized. This allows us to capture frames from the camera continuously.

  CameraImage? cameraImage;
  CameraController? cameraController;

  @override
  void initState() {
    super.initState();
    loadCamera();
  }

  loadCamera() async {
    final cameras = await availableCameras();
    final frontCamera = cameras.firstWhere(
        (camera) => camera.lensDirection == CameraLensDirection.back);
    cameraController = CameraController(
      frontCamera,
      ResolutionPreset.medium,
      enableAudio: false,
    );

    cameraController!.initialize().then((_) {
      if (!mounted) {
        return;
      } else {
        setState(() {
          cameraController!.startImageStream((imageStream) {
            cameraImage = imageStream;
          });
        });
      }
    });
  }

Next, we need to use the Tflite package to perform image classification. We load our model and labels using the loadModel() method. We then use the runModelOnFrame() method to classify each frame from the camera stream. This method takes in the bytesList, image height, image width, imageMean, imageStd, rotation, numResults, and threshold as parameters. We set the asynch parameter to true so that the model runs on a separate thread, preventing the UI from freezing.Finally, we are using the forEach() function to loop through the predictions and update the UI with the predicted output.

  String output = '';

  runModel() async {
    if (cameraImage != null) {
      var prediction = await Tflite.runModelOnFrame(
        bytesList: cameraImage!.planes.map((plane) {
          return plane.bytes;
        }).toList(),
        imageHeight: cameraImage!.height,
        imageWidth: cameraImage!.width,
        imageMean: 127.5,
        imageStd: 127.5,
        rotation: 90,
        numResults: 2,
        threshold: 0.1,
        asynch: true,
      );
      prediction!.forEach((element) {
        setState(() {
          output = element['label'];
        });
      });
    }
  }

  loadModel() async {
    await Tflite.loadModel(
      model: 'assets/model_unquant.tflite',
      labels: 'assets/labels.txt',
    );
  }

*** Building a user interface to capture and process images**

Next, we will build the user interface to capture and display images. In the Scaffold() widget, we will create an AppBar() widget with the title "Fruit Classification". We will then use the SafeArea() widget to avoid overlapping the UI with the device's system UI. We will create a Container() widget with a height of 70% of the device height and width of the device width to hold the camera preview. We will then check if the camera controller is initialized and use the AspectRatio() widget to set the aspect ratio of the camera preview. Finally, we will add a Text() widget to display the output label from the machine learning model.

Scaffold(
  appBar: AppBar(
    title: const Text("Fruit classification "),
  ),
  body: SafeArea(
    child: Padding(
      padding: const EdgeInsets.all(16.0),
      child: Column(
        crossAxisAlignment: CrossAxisAlignment.start,
        children: [
          Container(
            height: MediaQuery.of(context).size.height * 0.7,
            width: MediaQuery.of(context).size.width,
            child: !cameraController!.value.isInitialized
                ? Container()
                : AspectRatio(
                    aspectRatio: cameraController!.value.aspectRatio,
                    child: CameraPreview(cameraController!),
                  ),
          ),
          Text(
            output,
            style: const TextStyle(
                fontWeight: FontWeight.bold, fontSize: 20),
          )
        ],
      ),
    ),
  ));

That concludes the logic, when you run your project, you should have something similar to the video below.

Conclusion

In conclusion, this article has demonstrated how to integrate computer vision AI into a Flutter app using Google Teachable. We have explored how to create a custom machine learning model in Google Teachable, export the model as a TensorFlow Lite model, and integrate it into a Flutter app using the TensorFlow Lite Interpreter. We have also built a user interface to capture and process images and displayed the results of the classification in real-time.

Future possibilities and improvements
As for future possibilities and improvements, there are various ways to enhance the user experience by integrating computer vision AI with other technologies such as augmented reality (AR). For instance, we could build an AR app that displays additional information about the classified fruits, such as their nutritional value and recipes.

Importance of computer vision AI in mobile app development
The importance of computer vision AI in mobile app development cannot be overstated. It opens up endless possibilities for creating innovative and intelligent mobile apps that can recognize and understand the world around them. With the help of Google Teachable and Flutter, developers can easily integrate computer vision AI into their mobile apps and enhance the user experience.

You can find the complete code for this project on the following link: Repo

CRYPTO WALLET DAPP WITH FLUTTER AND SOLIDITY

tedTecch — Sat, 21 May 2022 15:27:35 +0000

Introduction

The evolution of web3 has been growing at a very fast pace, with a lot of products and tools coming out making it easy to understand and perform transactions on the blockchain. The use of blockchain technologies to develop web apps and mobile apps, also known as decentralised applications (Dapps), like our crypto wallets (trust wallet, metamask), pancake swap, etc., is also growing at a very fast pace. Therefore, anyone who intends to stay current and ahead of the growth should learn how to use and integrate this technology.

In this article, I’ll show you how to connect your Solidity smart contract to a flutter application and perform basic CRUD operations. This article is a continuation of my previous article on creating an ERC20 token on a polygon network. In the article, I explain some basic terminologies that won't be repeated in this article, so check it out Here.

We’ll cover the following in details:

Connecting to a smart contract
Getting the balance of an address
Fetching data from the blockchain, e.g., name, symbol, e.t.c.
Transfer our token from one address to another
Mint tokens to our address

Prerequisites

To understand and follow through this article, you need to have:

Flutter pre-installed
Basic understanding of flutter and dart
Code editor
Web Browser(chrome)

Getting started

After writing, compiling, and deploying the smart contract, we need a blockchain server to run it on. There are some platforms that offer this service, like Infura, Alchemy, etc. We will be using Alchemy for its simplicity and also because it provides us access to different blockchain networks.
Simply create an account and create a new app on the platform, enter project details, then you can select the chain, in this case, polygon and network in polygon Mumbai.

After creating the app, click on "view key," which would give us a key and two links, as in the image below. You can pick any of the links, but we would use the http link by copying it.

The next thing is to create a new Flutter project and develop a basic design for our app with three buttons: send, receive, and a floating action button (to refresh our screen). The send and receive buttons provide a cupertino alert dialog where we can input an address and amount.
But before we start coding the logic of the app, we need to add some packages.

The first package we require is web3dart, which is a dart library that connects to the Ethereum blockchain. It connects to an Ethereum node to send transactions, interact with smart contracts, and much more. And the second package is http, which is used for making https requests in a flutter.

We start by creating the following variables below. We also paste our metamask address, the http link provided by Alchemy in the blockchain url, and declare a variable dec, which represent 10^18 which is the erc20 decimal standard.


 late Client httpClient;
  late Web3Client ethClient;
  //Polygon address
  final String myAddress = "0x4818569AA9dE13d3cC1D702Cd10a95932799a674";
  //url from alchemy
  final String blockchainUrl = "https://polygon-mumbai.g.alchemy.com/v2/xhOSAQIFW6H_-NjxcSrpa1vJbwckXTUC";
  bool data = false;
  int myAmount = 0;
  int amt = 0;
  var addressTo = "";
  var dec = pow(10, 18);
  var mydata;
  var mybalance;
  late String transHash;
  var balance;
  var name;
  var symbol;

After that, we head back to our Remix IDE, and in the Solidity compiler tab, we copy the ABI of our smart contract as shown in the image below, marked by the red arrow. Also, ensure the contract file is our created contract as marked with the blue arrow.

Then in our flutter project, we create an assets folder in our root folder and within it we create a file named ‘token.json’, in this file, we paste the copied abi. We can then move to our pubspec.yaml file and uncomment, adding assets to our application looking like the code below. Then we can run flutter pub get command.

 assets:
    - assets/

In our initState callback, we will initialize our httpClient and Web3client, as well as call a method to retrieve the name, symbol, and balance values from our smart contract:

@override
  void initState() {
    httpClient = Client();
    ethClient = Web3Client(blockchainUrl, httpClient);
    super.initState();
    getName();
    getSymbol();
    getBalance(myAddress);
  }

Connect Smart Contract

We declare a contract using the DeployedContract class from our web3dart package, which takes in the ABI file, the name of our smart contract, which in our case is FirstToken, and the contract address and returns it from our function.

Future<DeployedContract> getContract() async {
    String abiFile = await rootBundle.loadString("assets/token.json");
    String contractAddress = "0x50338cAF974F2ec1869020e83eF48E36aCE93caf";
    final contract = DeployedContract(
        ContractAbi.fromJson(abiFile, "FirstToken"),
        EthereumAddress.fromHex(contractAddress));
    return contract;
  }

Fetch Data From TheBlockchain

The next function, query is used to call a function inside our smart contract, I used it mainly for read operations. This function connects to our smart contract using the call extension from the web3dart EthereumClient class, we will be calling this function in another function.

Future<List<dynamic>> query(String functionName, List<dynamic> args) async {
    //
    final contract = await getContract();
    final ethFunction = contract.function(functionName);
    // This line below doesn't work.
    final result = await ethClient.call(
        contract: contract, function: ethFunction, params: args);
    // print(result.toString());
    return result;
  }

Write To The Blockchain

In this submit function we are performing a write operation to the blockchain therefore it requires our private key which you can get from metamask, but in a normal wallet, this private key is stored securely in the app when you import or generate a wallet, take note that you should not reveal your private key under any circumstances.

Also, we used a Transaction.callContract to pass in our gotten contract, the function we are requesting in our smart contract, the parameters imputed in the function and maximum gas fee, after we used sendTransaction from web3dart to pass in our private key, the transaction object and fetchChainIdFromNetworkId.

Future<String> submit(String functionName, List<dynamic> args) async {
    DeployedContract contract = await getContract();
    final ethFunction = contract.function(functionName);
    snackBar(label: "Recording tranction");
    EthPrivateKey key = EthPrivateKey.fromHex(
        "5288***********************************60ffbd08");
    Transaction transaction = await Transaction.callContract(
        contract: contract,
        function: ethFunction,
        parameters: args,
        maxGas: 100000);
    print(transaction.nonce);
    final result = await ethClient.sendTransaction(key, transaction,
        fetchChainIdFromNetworkId: true, chainId: null);
    print(result);
    ScaffoldMessenger.of(context).removeCurrentSnackBar();
    snackBar(label: "verifying transaction");
    //set a 20 seconds delay to allow the transaction to be verified before trying to retrieve the balance
    Future.delayed(const Duration(seconds: 20), () {
      ScaffoldMessenger.of(context).removeCurrentSnackBar();
      snackBar(label: "retriving balance");
      getBalance(myAddress);
      ScaffoldMessenger.of(context).clearSnackBars();
    });
    return result;
  }

We set a 20-sec delay before calling our getBalance function to ensure that the transaction has been executed because it takes a while, we also add some snackBar to make sure a user knows what is going on.

Moving forward we have the main getBalance, getName and getSymbol function, these functions calls the query read request function explained above, pass the function name as it is written in solidity and save it as a variable in our flutter app.

Future<void> getBalance(String targetAddress) async {
    EthereumAddress address = EthereumAddress.fromHex(targetAddress);
    // print('In getGreeting');
    List<dynamic> result = await query('balanceOf', [address]);
    print('In getGreeting');
    print(result[0]);
    mybalance = result[0];
    var div = BigInt.from(dec);
    balance = BigInt.from(mybalance / div);

    print("balance: $balance");
    data = true;
    setState(() {});
  }
  Future<void> getName() async {
    // print('In getGreeting');
    List<dynamic> result = await query('name', []);
    print(result[0]);
    name = result[0];
    data = true;
    print(name);
    setState(() {});
  }
  Future<void> getSymbol() async {
    List<dynamic> result = await query('symbol', []);
    print(result[0]);
    symbol = result[0];
    data = true;
    setState(() {});
  }

Finally, we have our ‘sendCoin’ and ‘recieveCoin’ functions, these functions use the ‘submit’ function explained earlier passing the function name as it is in solidity, also the address it should be sent to and the amount.

Future<String> reciveCoin() async {
    EthereumAddress addressTo =
        EthereumAddress.fromHex("0x4818569AA9dE13d3cC1D702Cd10a95932799a674");
    var bigAmount = BigInt.from(myAmount);
    var response = await submit('mint', [addressTo, bigAmount]);
    print('Recieved');
    transHash = response;
    setState(() {});
    return response;
  }
  Future<String> transferCoin() async {
    var amount = BigInt.from(amt *dec);
    EthereumAddress to = EthereumAddress.fromHex(addressTo);
    print("amo: $amount");
    var response = await submit('transfer', [to, amount]);
    print('Transfered');
    transHash = response;
    setState(() {});
    return response;
  }

That concludes our logic, when you run your project, you should have something like this.

Conclusion

This smart integration has covered enough to help you understand and execute read and write operations to the block chain using flutter which is relatively new, so keep exploring and experimenting. Keep in mind there is no learning without practice.
Check out the link below to access the project's repository.
https://github.com/lyon-zas/ERC-20-wallet

Create an ERC-20 token on Polygon

tedTecch — Mon, 16 May 2022 22:12:23 +0000

The adoption of cryptocurrencies and block-chain technology has been amazing in the past few years, leading to more coins being created with different use-cases from Defi to exchanges etc.
In this article, I would teach you how to create an ERC-20 token using solidity and deploy it on the polygon block-chain network.
But before we start, Let’s define some of the terms we will come across in this article.

What is ERC-20?

Erc-20 is a technical standard used in a smart contract on the Ethereum blockchain to provide and implement a list of rules that all Ethereum based tokens must follow according to Investopedia.

What is a smart contract?

A “contract” simply means an agreement between two or more people and the word “smart” means automated or independent, therefore a smart contract is an automated document or instruction that executes an agreement running independently on a block-chain network.

According to Wikipedia, a Block-chain is a growing list of records called blocks that are securely linked together using cryptography, each block contains a cryptographic hash of the previous block, time and transaction data.

What is Polygon?
Polygon is a secondary layer solution that is compatible with and complements the Ethereum block-chain. Polygon complements the Ethereum block-chain by providing additional features relating to security, block-chain sovereignty, fast and cheap transactions.

Getting Started

To begin, we must first download and install MetaMask. So, go ahead and click this link to add the MetaMask extension to your browser (preferably Chrome).
After that, create and configure your cryptocurrency wallet. Because this is a learning exercise, click here, scroll down and click “add Mumbai network” to add the polygon test network and switch your network from Ethereum Mainnet to Matic Mumbai as seen in the image below.

The next thing is to get some test Matic to handle gas fees when performing write operations to our smart contract. Click this link and paste your wallet address to get some as seen in the images below.

Step 1: Create Remix File
To write the code we would use Remix by clicking on remix.ethereum.org, Remix is an online integrated development environment(IDE) for solidity. Click on the contract folder by the left and create a file called token.sol, the “sol” here stands for solidity.

Step2: Using OpenZeppelin
OpenZeppelin provides the ERC-20 token standard we discussed above and other templates for different tokens. Therefore we can utilise the wizard feature on their website to create the token even without prior knowledge of solidity.
In the Wizard page here, we add the name “First Token”, and the symbol “FRT” for our token, we also make it mintable meaning you can create a supply for the token. you can see this in the image below.

Explanation:

// SPDX-License-Identifier: MIT
pragma solidity ^0.8.7;

In the first line, we added the license type, followed by the second line that declares the version of solidity we are using, which is 0.8.7 in this file.

import "@openzeppelin/contracts/token/ERC20/ERC20.sol";
import "@openzeppelin/contracts/access/Ownable

Lines 4 and 5 import the openZeppelin’s ERC-20 and ownable packages respectively, Then declare a contract followed by the name “First Token” we also specify that the contract is ER20 and it’s ownable.

constructor() ERC20("First Token", "FRT") {}

next, we declared the constructor to initialize the ERC20 name and symbol whenever we create an instance.


function mint(uint256 amount) public onlyOwner {
        _mint(msg.sender, amount* 10**uint(decimals()));
    }

Final we have a function that collects the address the token will be minted to and the amount to be minted, then a public modifier which means the function can be called outside the contract and another modifier only owner indicating only the owner(the address that deployed the contract) can call the function, it then passes this input details to the _mint() function.

// SPDX-License-Identifier: MIT
pragma solidity ^0.8.4;
import "@openzeppelin/contracts/token/ERC20/ERC20.sol";
import "@openzeppelin/contracts/access/Ownable.sol";
contract FirstToken is ERC20, Ownable {
    constructor() ERC20("First Token", "FRT") {}
    function mint(uint256 amount) public onlyOwner {
        _mint(msg.sender, amount* 10**uint(decimals()));
    }
}

Step4: Compile the contract
Using remix makes it so easy to compile our smart contract, you can do this by clicking on the solidity compiler which is the third icon on the left navbar. Next, we ensure that our solidity version in the second line in our contract is the same as our compiler version to prevent any errors, then you click the “compile token.sol” button, subsequently you can just “CTRL + S” to compile which saves time.

Step5: Deploy Contract
Now that our contract is compiled we can deploy it to Ethereum, polygon blockchain, Binance etc. Here we would deploy on the polygon Mumbai testnet because this would cost some gas fees to deploy, also testnet is a great way for developers to learn without any cost.

To deploy the contract, we move to the last icon on the left navbar. First, we change the deployment environment JavaScript VM(london) to injected web3, and approve the "connect” metamask wallet address to remix. The next thing we check is if the contract we are about to deploy is the “token.sol” you can do this by checking the above the deploy button.

Finally, we can now click the deploy button, confirm the transaction on metamask and wait for the traction to be approved which would take some seconds.
Now that our token has been successfully deployed we are going to mint 1000 FirstToken and confirm the transaction, we can also check the total supply of this token feel free to play around with other functions. Note the orange buttons are written operations that would cost a gas fee and the blue buttons are read operations which won't cost any gas fees.

View Deployed Token In Wallet
Now, that we have our token deployed on the polygon network, it would be amazing to view this token in our wallet just like any other token right?
To view, this token opens up metamask, click on the assets tab and
“import tokens” at the bottom of our MetaMask app. On the provided page we can fill out some empty fields but we would need only the contract address and the rest would the filled with the help of MetaMask.
You can get the contract address by clicking on the copy icon in the remix as seen in the image below.

Click on the “add Custom Token” button and we have our token imported successfully. As you can see from the image below, we have “1000 FRT” which we minted earlier and you can also see the history of the transaction, we can also transfer this coin to your friends with just their address.

Conclusion

In this article, we have covered some blockchain terminologies, solidity, how to create an erc20 token and also viewing this token in your MetaMask wallet.
I hope this article has been helpful to you in your tech journey, remember there is no learning without practice so make sure your practice.
Thank you.

In the second part of this article, I will be integrating this smart contract into a flutter application.
You can read the second article here