Forem: Olakusibe Aremu-Oluwole

Solving AttributeError: 'float' object has no attribute 'rint'

Olakusibe Aremu-Oluwole — Tue, 26 Jul 2022 09:13:00 +0000

Problem

Lately, I have been working on some ETL projects and during the transform stage I come across the error "AttributeError: 'float' object has no attribute 'rint'", combined with "TypeError: loop of ufunc does not support argument 0 of type float which has no callable rint method".

Explanation

It took a while but it became obvious that as I get/extract data from many sources (CSV, JSON, or XML), similar columns might have a mixture of the datatype (i.e a column may have a string and int data type mixed), and before I can proceed to the transform stage, I would need that column to have a designated datatype, especially if I want to perform other tasks on it (which was what I was trying to do and kept encountering the errors).

Solution

To convert the column with a mixed data type, you will need to cast it to the designated data type you intend to work with.

This can be achieved by using the pandas.DataFrame.astype function, as illustrated below

import pandas as pd

# an example of extracted data
extracted_data = pd.DataFrame({
    'Brand': ['Gucci', 'Nike', 'Adidas', 'Hermes', 'Zara'],
    'year_of_manufacture': [2010, 1999, '2012', 2011, '2008'], # mixed data type
    'price': ['4034.203', 5000.00, '7450.17567', 3023.004, '4901.32345'] # mixed data type
})

# cast specific columns to a desired data type
cast_to_type = {
    'year_of_manufacture': int,
    'price': float
    }
extracted_data = extracted_data.astype(cast_to_type)

# do the transfrom
extracted_data['price'] = extracted_data['price'].round(2)

print(extracted_data)

For Further Reading

pandas.DataFrame.astype function
Other ways to cast mixed data type in a list to a specific data type

Cover Photo by Vardan Papikyan on Unsplash

Free Microsoft Certification Exam - May 2022

Olakusibe Aremu-Oluwole — Wed, 11 May 2022 10:28:16 +0000

It's here, pick any (one) Microsoft Build Cloud Skills Challenge that would earn you a free Microsoft Certification exam upon completion.

The eligible exams include:

AI-102: Designing and Implementing a Microsoft Azure AI Solution
AZ-204: Developing Solutions for Microsoft Azure
AZ-220: Microsoft Azure IoT Developer
AZ-400: Designing and Implementing Microsoft DevOps Solutions
DP-420: Designing and Implementing Cloud - Native Applications Using Microsoft Azure Cosmos DB
MS-600: Building Applications and Solutions
PL-100: Microsoft Power Platform App Maker
PL-200: Microsoft Power Platform Functional Consultant
PL-300: Microsoft Power of BI Data Analyst
SC-200: Microsoft Security Operations Analyst
SC-300: Microsoft Identity and Access Administrator

It begins on May 24, 2022 and ends June 21 2022.

Check here for the offical rules and additional FAQs

microsoft #iot #ai #data #developer #security #building #power #challenges #free

Azure Functions: Requeue Items

Olakusibe Aremu-Oluwole — Wed, 13 Oct 2021 08:00:50 +0000

Introduction

As an azure functions dev, I am sure you've come across issues where a QueueTrigger azure function fails and after several retry attempt (dequeue count) the items are sent into a poison queue, and after resolving the errors, we seek ways of moving all the items on the poison queue back onto the main queue for reprocessing.

Most times to move the items from the poison queue back to the main queue, I use the Azure Storage Explorer, but it is a manual task, and I needed a way to automate it, and you can't do so with the queue binders (input/output), you will need to write a separate set of code and then automate it with either a TimeTrigger or HttpTrigger.

Show Me The Code

Requirement

Azure Storage connection string (for accessing the queues)
Nuget Library: Azure.Storage.Queues (v12.5.0 or latest)

public static async Task RequeueItems(QueueClient sourceQueue, QueueClient destinationQueue)
{
    int messageCount = 32;

    var sourceQueueProps = await GetQueuePropertiesAsync(sourceQueue);
    int queuelength = sourceQueueProps.ApproximateMessagesCount;

    if (queuelength > 0)
    {
        var iterate = Math.Ceiling((double)queuelength / messageCount);
        do 
        {                    
            QueueMessage[] _messages = await sourceQueue.ReceiveMessagesAsync(messageCount);
            if (_messages.Any())
            {
                foreach (var item in _messages)
                {
                    await destinationQueue.SendMessageAsync(item.Body);
                    await sourceQueue.DeleteMessageAsync(item.MessageId, item.PopReceipt);
                }
            }
            iterate--;
        } while (iterate > 0);
    }
}

The RequeueItems function accept two QueueClient parameters, one is the source queue (where the queue items are coming from) and the other is a destination queue (where the queue items should go), the function job is a simple dequeue-enqueue-delete operation that happens in a loop (the length of the source queue).

To automate it, a TimerTrigger will run RequeueItems function periodically, after getting the source and destination QueueClient objects to be used.

public static class RequeueUsingStatic
{
    [FunctionName(nameof(RequeueUsingStatic))]
    public static async Task Run([TimerTrigger("0 0 10-20/2 * * Mon-Fri")] TimerInfo myTimer, ILogger log)
    {
        log.LogInformation($"C# Timer trigger function {nameof(RequeueUsingStatic)} executed at: {DateTime.Now:dd-MMM-yyyy}");

        try
        {
            var azure_storage_Uri = Environment.GetEnvironmentVariable("AzureWebJobsStorage");

            QueueClient sourceQueue = await RequeueExtensions.GetQueueClientAsync(azure_storage_Uri, RequeueSampleData.sourceQueueName);
            QueueClient poisonQueue = await RequeueExtensions.GetQueueClientAsync(azure_storage_Uri, RequeueSampleData.poisonQueueName);

            await RequeueExtensions.RequeueItems(poisonQueue, sourceQueue);
        }
        catch (Exception ex)
        {                
            log.LogCritical($"C# Timer trigger function {nameof(RequeueUsingStatic)} failed at: {DateTime.Now:dd-MMM-yyyy}, Error Message: {ex.Message}");
        }
    }
}

So every two hours between 10 am to 8 pm, from Monday through Friday, the TimeTrigger would requeue all the failed items (on the poison queue) back to the main queue for reprocessing, hence depending on your scenario or use case, any trigger should work, such as HttpTrigger.

Conclusion

Besides the use case of moving queue items from the poison queue back to the main queue, the RequeueItems function, when needed can be used for moving all items between any two queues in bulk.

Here is a link to the complete code repo

CREDIT: Cover Photo by Peter Olexa on Unsplash