Forem: Michael Olayemi

Secure Shell (SSH) Crates in Rust

Michael Olayemi — Fri, 18 Oct 2024 02:47:11 +0000

Overview

Secure Shell (SSH) allows us to communicate with a remote server on our local machine for various operations.
One of the common operations is reading a file remotely.
Let's see some third-party SSH crates (dependencies) available in Rust.

Rust SSH Crates

We can use any of these crates in our future Rust project as an SSH client.

Multiple Page Web Scraper in Rust.

Michael Olayemi — Fri, 24 Jun 2022 18:56:05 +0000

Data is the new gold according to some of my folks. To some extent maybe they are right. Getting data especially massive data could be expensive. In this article, I will show you step-by-step how to build a multiple-page web scraper using rust programming language.

Imagine you wanted to get the price of an iPhone x from an eCommerce website with multiple pages. You can achieve that manually but it will take a lot of time. Writing a code for the purpose will make it easy. In seconds you can scrape over 10 pages.

Create a new project using cargo new multipage. We are going to the need the following dependencies, add them to your cargo.toml under dependencies:

reqwest={version="0.11.9", features=["blocking"]}
select="0.5.0"
anyhow="1.0.56"`

Reqwest is an http client and it will be useful for making http calls to the web pages will intend to scrap some data from.

Select is the heart of this program. It's a cool crate that has functions to scrap HTML web pages.

Anyhow will majorly help in handling errors.

let's dive in:
Firstly, we are going to inspect the web page we wanted to scrap. If you are using Chrome, right-click and pick the inspect option. inspect the HTML codes until you get to the elements of the web page you intended to scrap. I used this web link as a practice:
"https://www.jumia.com.ng/catalog/?q=iphone&viewType=grid&page=1#catalog-listing"
The website is an eCommerce website. After inspecting the page, I got the HTML code for the catalog-listing:

<a class="core" href="/castillo-castillo-de-liria-red-75cl-x1-81158848.html" data-id="CA254FF1EJDRYNAFAMZ" data-name="Castillo De Liria RED 75cl x1" data-price="3.85" data-brand="Castillo" data-category="Grocery/Beer, Wine &amp; Spirits/Wine/Red Wine" data-dimension23="" data-dimension26="18"
 data-dimension27="4.6" data-dimension28="1" data-dimension37="0" data-dimension43="CP_MT38|CP_MT72|CP_MT73|CP_MT76|CP_MT77|CP_MT78|CP_MT80|CP_MT81|CP_MT84|CP_MT92|Camp_45|Camp_70|Camp_72|FDY2020|FDYJE|Merch_148|Merch_150" data-dimension44="0" data-list=""
  data-position="1" data-track-onclick="eecProduct" data-track-onview="eecProduct" 
  data-track-onclick-bound="true"><div class="img-c"><img data-src="https://ng.jumia.is/unsafe/fit-in/300x300/filters:fill(white)/product/84/885118/1.jpg?0770" src="https://ng.jumia.is/unsafe/fit-in/300x300/filters:fill(white)/product/84/885118/1.jpg?0770"
   class="img" width="208" height="208" alt="">
   <img data-src="https://ng.jumia.is/badges/fdyje/10/138x18.png?4886" src="https://ng.jumia.is/badges/fdyje/10/138x18.png?4886" class="_ni camp" alt="FDYJE"></div>

   <div class="info"><h3 class="name">Castillo Castillo De Liria RED 75cl x1</h3>
   <div class="prc">₦ 1,725</div><div class="s-prc-w"><div class="old">₦ 1,898</div>
   <div class="tag _dsct _sm">9%</div></div>
   <div class="rev"><div class="stars _s">4.6 out of 5<div class="in" style="width:91.99999999999999%">
   </div></div>(18)</div><svg aria-label="Express Shipping" viewBox="0 0 114 12" class="ic xprss" width="94" height="10">
   <use xlink:href="https://www.jumia.com.ng/assets_he/images/i-shop-jumia.c8de1c55.svg#express"></use>
   </svg><p class="shipp">Jumia Express items in your order will be delivered for free (Lagos &amp; Abuja only, excluding large items)</p></div></a>

The following attributes were use:

<class="info"> as the major node.
<class="name"> to get the name of the product
<class="prc"> to get the price of the product

Calling the URL with Reqwest

let mut page = 1;

    while page != 10 {
        let url = format!(
            "https://www.jumia.com.ng/catalog/?q=iphone&viewType=grid&page={:?}#catalog-listing",
            page
        );
        let res = reqwest::blocking::get(url).with_context(|| format!("opening url error"))?;
let document = Document::from_read(res).context("parsing response")?;

The first thing to note is that this is a multi-page scrapper, we are going to loop through different pages until we are satisfied. From the code above, I intend to go through the iPhone listing page 10 times and get the name of available iPhones and their price. To achieve looping through the page I used the format! macro, easy way for me to concatenate I guess. The macro will accept the page params as the program keeps looping through different pages.

I used reqwest to accept the URL and used the document method in select to parse it to a file that will be readable by the program.

let writer = OpenOptions::new()
            .read(true)
            .append(true)
            .create(true)
            .open("box.txt")
            .with_context(|| format!("opening file "))?;

        let mut writer = LineWriter::new(writer);

        let jumia = document.find(Class("info"));
        let link = document
            .find(Class("core"))
            .next()
            .context("writing to the output file")?;

        for node in jumia {
            let name = node
                .find(Class("name"))
                .next()
                .context("writing to the output file")?;
            let price = node
                .find(Class("prc"))
                .next()
                .context("writing to the output file")?;


            writeln!(
                writer,
                "{:?}---{:?}---{:?}",
                name.text(),
                price.text(),
                link.attr("href")
            )
            .context("writing to the output file")?;
        }
        page = page + 1;
    }

    Ok(())

The first part of the code specifies what the scrapped data will be saved as. It will be saved as box.txt.

The paged is being pulled down by the document variable and every node can easily be assessed and data can be scraped out of it.

Thanks for reading, you can also check nutrisoft, my nutrition web application.

Code available on github

Rust Project for Beginner: Natural Language Detector

Michael Olayemi — Sat, 19 Feb 2022 18:38:06 +0000

Rust is a lovely language, the more you learn it the more you love it. In this tutorial we are going to build a simple application that have the capacity to detect over 80 languages.
This tutorial showcase the following concept in rust:

Using external crate in our project
Creating functions in rust
Using of macros
Calling functions in rust

To get starter open your terminal and run the command:cargo init langdector If you don't have rust installed on your system, please head over to the official documentation.

Open the generated project in your vscode or your favourite IDE. Edit the cargo.toml file by adding this dependency:whatlang = "0.13.0.

After adding the external create, we can proceed to our main.rs and begin applying the logic.

We are going to create a function called what_language. With the following codes:

//calling the external crate
use whatlang::{detect, Lang, Script};

//creating a function
fn what_language(mut x: String) {
    let text = &x;
//using .unwrap() for exception handling
    let info = detect(text).unwrap();
//println! is one of the major macros used in rust
    println!("Language type {}", info.lang());
    println!("Language Script {}", info.script());
    println!("System confidence {}", info.confidence());
    println!("System reliability {}", info.is_reliable());

}

fn main() {
//creating string alice
    let alice = String::from(
        "Ĉu vi ne volas eklerni Esperanton? Bonvolu! Estas unu de la plej bonaj aferoj!",
    );
//calling the function
    what_language(alice);
}

The output :

Language type Esperanto
Language Script Latin
System confidence 1
System reliability true

The comments explained each logic._ &x_ obeys the concept of ownership and borrowing.

You can try to run the code :)

Head over to github for the code.
You can follow me of twitter for collaboration.

Creating a Web Page with Actix-Web (Rust )

Michael Olayemi — Wed, 01 Dec 2021 13:10:15 +0000

Overview
In this tutorial, we are going to build a simple website using actix, a stable framework based on rust programming language.
The simple website is called bookstore: it is a simple website for book index.

Rust is a young language, coming from Java background was so awesome, when I pick up rust, I fell in love with the language and felt like building everything in rust.

There are cool frameworks in rust to build a website but I choose actix because to me it doesn't have any con yet.

Let dive in, create an empty project with cargo (cargo new bookstore). Open the cargo.toml and add the following dependencies :
actix-web="3" actix-files="0.4.0" serde_json = "1.0.53" handlebars = { version = "4.1.4", features = ["dir_source"] }

The actix-web will help us to access the actix web framework.
actix-files will help us to add static files to our project.
Handlebar is a template engine that will help us to create template for our web page.

Using Handlebars
In the main.rs, paste the code below



use actix_files::{Files, NamedFile};
use actix_web::{web, App, HttpResponse, HttpServer, Result};
use handlebars::Handlebars;
use serde_json::json;

async fn index(hb: web::Data<Handlebars<'_>>) -> HttpResponse {
    let data = json!({
        "project_name": "Book Store",
        "books":[
            {
                "name":"Harry Potter",
                "author":"J K Rowlings",
                "image_path":"/static/image/download.jpeg"
            },
            {
                "name":"Lord of the ring",
                "author":"Tolken",
                "image_path": "/static/image/lord_of.jpeg"
            },
            {
                "name":"Americanah",
                "author":"Chimamada Adichie",
                "image_path":"/static/image/americanah.jpeg"
            },
            {
                "name":"Elon Musk",
                "author":"#####",
                "image_path":"/static/image/elon.jpeg"
            },
        ]


    });

    let body = hb.render("index", &data).unwrap();
    HttpResponse::Ok().body(body)


}

Handlebar helps load and compile the json before before using them. The data created with json will be used in our project.

In the main function, we have to initialize the handlebar and register the templates_directory.



#[actix_web::main]
async fn main() -> std::io::Result<()> {
    let mut handlebars = Handlebars::new();
    handlebars
        .register_templates_directory(".html", "./static/")
        .unwrap();
    let handlebars_ref = web::Data::new(handlebars);

    println!("listening on port 8080");
    HttpServer::new(move || {
        App::new()
            .app_data(handlebars_ref.clone())
            .service(Files::new("/static", "static").show_files_listing())
            .route("/", web::get().to(index))
    })
    .bind("127.0.0.1:9090")?
    .run()
    .await
}

Create a directory in the root folder called static and inside the static create two folders for images and css file. Inside the directory create index.html and paste the code below:



<!DOCTYPE html>
<html>
  <head>
    <meta charset="UTF-8" />
    <title>{{project_name}}</title>
    <link rel="stylesheet" href="static/css/index.css" type="text/css">
  </head>
  <body>
    <h1>{{project_name}}</h1>
    <section class="books">
      {{#each books}}
        <article class="book">
          <h3>{{this.name}}</h3>
          <img src="{{this.image_path}}" />
            <h4>Author: {{this.author}}</h4>
        </article>
      {{/each}}
    </section>
  </body>
</html>

Handlebar help us to render the data created in json into the html template, {{project_name}} will be the project name. {{#each books}} helps loop through the json to create the book index without writing a separate tag for each index.

When you are done, run cargo run in your terminal and access the page with 127.0.0.1:9090

Head over to github to download the full code.

If you find this article useful and will like to tip me with $1 worth of cryptocurrency, you can do it here and I will really appreciate.

Thanks