Forem: Ronald R

DATA ANALYSIS - Highest Rated Movie in Imdb - Data set

Ronald R — Sat, 13 Jan 2024 01:25:22 +0000

I obtain a dataset from kaggle and I wanted to do a data analysis with it to figure out what was the highest rated movie of all time in Imdb. Funny enough I never realised that it would take quite some work to actually find it. It was never simple as a this is rated higher than this movie. No it needs quite a bit of filtering to get it.

Let's take a look at what I did. There were over 45,000 movies listed and there were some mixing data in some of them so we had to clean it all up.

Test

Test 1 - Highest rated movie of all time

Result - The Land

Issue - It only has 4 votes and has low confidence level since in a sample size of 14,000 it only takes about 0.0003% of the size.

Test 2 - Highest rated movie of all time with atleast 7,000 votes

Result - Shawshank Redemption with 14,075 votes

Issue - It may have the highest numberof votes and a significant size but we still can not exclude the fact that it only has about 60% of the total vote count

Test 3 - Highest rated movie of all time with atleast 10,000 votes

Result - The Dark Knight with 12,269 votes which is about 87% of the vote , we also see that Inception has the highest vote of all time

Test 4 - Highest rated movie of all time with atleast 10,000 votes and atleast 1 billion in revenue.

Result - The Dark Knight with 12,269 votes and also $1,004,558,444 in revenue

Conculsion

In conclusion, the tests conducted to determine the highest rated movie of all time revealed several results. The initial highest-rated movie, "The Land," was found to lack statistical significance due to its low number of votes and limited representation in the overall sample. Moving forward, tests with higher thresholds for the number of votes led to more robust conclusions.

"The Shawshank Redemption" emerged as the highest-rated movie with over 7,000 votes, but the issue of it representing only 60% of the total vote count raised concerns. A more refined criterion of at least 10,000 votes highlighted "The Dark Knight" as the top contender, comprising 87% of the votes and surpassing even the highest voted movie "Inception."

Further refinement, incorporating both high votes and substantial revenue, solidified "The Dark Knight" as the conclusive result, boasting 12,269 votes and an impressive $1,004,558,444 in revenue. This comprehensive approach ensures a more reliable determination of the highest rated movie, considering both audience appreciation and commercial success.

DATA ANALYSIS - Student Mental Health - A Reading

Ronald R — Tue, 26 Dec 2023 19:04:11 +0000

For this Blog I decided to make something different, something special and simple.

I decided to analyze a Data set for Students Mental Health that I got from Kaggle, I didnt want to over complicate things so I just went on and started analyzing the file.

For this Analysis I decided to use Jupyter Notebook and a mixture of SQL instead of straight up analysing it with Notebook. I wanted to challenge myself.

After quite a few research I was able to analyze the data.

The Goal

I am to figure out which year of students is more likely to have exprience trouble with mental health and somehow correlate it with stress.

This was a very interesting Analysis just because I have always wondered why are students on their final years are more relax as to oppose to students in their first and with this analysis I was to able to see it in a bigger picture.

The Strategy

Total Number of Students resulted in atleast a 100

After Cleaning the Data making sure that there was no NaN or null numbers.
First off I queried the Male students getting their

Gender
Ager
Course
Year
CGPA

To access the size of the Data set

query = "SELECT Gender, Age, Course, Year, CGPA FROM student_mental_health_data_cleaned WHERE Gender = 'Male'"

I realize that there where only about 20 something student that were 'Male'

After which, I made the decision to just use both genders at the same time. Since the sample size ratio between 'Male' to 'Female' is fairly out balanced.

Next, i queried the data of students that has Depression and Anxiety

query = "SELECT Gender, Age, Year, Depression, Anxiety, Panic FROM student_mental_health_data WHERE (Depression = 'Yes' OR Anxiety = 'Yes')"

The result showed that out of a 100 students half experience some sorth of Mental Health issue.

"Please note that the number of responses can still vary depending on whether or not they are diagnosed or undiagnosed, for this analysis we will take the responses as face value to determine an outcome"

There are three major Factors that can affect the result

Uneven sample size
Professional Diagnosis
Emotional affection

After which we tallied the results of the dataset that we queried

The result showed that out of 50 students

21 of students are Year 1
15 of students are Year 2
13 of students are Year 3
2 of students are year 4

If we dissect the result it showed that Year 1 and Year 2 students are the most to exprience mental health.

A study done by Gretchen Geng and Richard Midford in 2015 titled "Investigating First and Second Year Education Students’ Stress
Level" provides a more insight on why this is because it was somehow related to completion of assessments because of their busy schedule at schools.
This finding corelates with the research that first and second year students’ high stress and eventualy lead to mental health issues at a stagerring high levels comes from a lack of time to study because of multiple factors as well as the ability to cope to a new and unfamilliar environment."

Year 1 can be factored in because they have the most number of students that participated in the survey. Though Year 2 in terms of ratio can be considered aas the most affected students and the chances of mental health exprience can occur.

Stress can be factored in the situation and the data set since accoring to Geng and Midford their coping mechanism is still developing as oppose to Year 3/4 students.

Summary

We can say that out of all the students Year 2 is more likely to expirence mental health issue. Though we still have to factor in the situation of each students, but by taking it in face value the result shows us which students in terms of ratio is the most affected.

And with that I move on to my next Data Analysis maybe this time instead of using SQL we'll just use python analysis.

DATA ANALYSIS - Student Mental Health - A Reading

Ronald R — Tue, 26 Dec 2023 19:04:11 +0000

For this Blog I decided to make something different, something special and simple.

I decided to analyze a Data set for Students Mental Health that I got from Kaggle, I didnt want to over complicate things so I just went on and started analyzing the file.

For this Analysis I decided to use Jupyter Notebook and a mixture of SQL instead of straight up analysing it with Notebook. I wanted to challenge myself.

After quite a few research I was able to analyze the data.

The Goal

I am to figure out which year of students is more likely to have exprience trouble with mental health and somehow correlate it with stress.

The Strategy

Total Number of Students resulted in atleast a 100

After Cleaning the Data making sure that there was no NaN or null numbers.
First off I queried the Male students getting their

Gender
Ager
Course
Year
CGPA

To access the size of the Data set

query = "SELECT Gender, Age, Course, Year, CGPA FROM student_mental_health_data_cleaned WHERE Gender = 'Male'"

I realize that there where only about 20 something student that were 'Male'

After which, I made the decision to just use both genders at the same time. Since the sample size ratio between 'Male' to 'Female' is fairly out balanced.

Next, i queried the data of students that has Depression and Anxiety

query = "SELECT Gender, Age, Year, Depression, Anxiety, Panic FROM student_mental_health_data WHERE (Depression = 'Yes' OR Anxiety = 'Yes')"

The result showed that out of a 100 students half experience some sorth of Mental Health issue.

There are three major Factors that can affect the result

Uneven sample size
Professional Diagnosis
Emotional affection

After which we tallied the results of the dataset that we queried

The result showed that out of 50 students

21 of students are Year 1
15 of students are Year 2
13 of students are Year 3
2 of students are year 4

If we dissect the result it showed that Year 1 and Year 2 students are the most to exprience mental health.

Stress can be factored in the situation and the data set since accoring to Geng and Midford their coping mechanism is still developing as oppose to Year 3/4 students.

Summary

And with that I move on to my next Data Analysis maybe this time instead of using SQL we'll just use python analysis.

The PR END

Ronald R — Fri, 08 Dec 2023 23:21:53 +0000

I had to make this entry because there was in no shape or form I would be able to merge my PR request because of Lack of resource. Not time but resource in this case I didn't have the right hardware to do it.

The Issue

As mentioned before the Issue was for a function in pytorch to accept kwargs** in its arguments. So as an issue it sounded straight forward but as I delve into it, it became harder and harder. So I had to navigate around pytorch and how it worked, I had to install countless of dependencies, libraries and software to make it work properly.

Anaconda
WSL (Windows Subsytem for Linux)
CMAKE
NCCL
CUDA

First of I needed conda to install pytorch properly and all of its dependencies, installing different version from Pytorch installation

I tried using pip but inorder for me to work with the function I needed a few dependencies that I can only find in Anaconda

I installed Anaconda thinking I can now use pytorch, just to realize to run setup.py I need to install CMAKE but then when I installed it. it said software incompatible, so i did little digging and tried another way around, I tried to by-pass it and use CUDA which is a parallel computing platform and programming model developed by NVIDIA.

I read the function and said "Gloo back end would not support this API" So My only choice was MPI and NCCL. Which can be find hereDistributed packge for torch

So the thing that got me with NCCL or NVIDIA Collective Communications Library because it only works with linux system, so with that I installed WSL. I sat on it a for a few days trying to figure out as a whole who the distributed package with pytorch worked. I went back to the problem and tried to work on it as a separate entity.

The code I had to change was this

def all_gather_into_tensor(output_tensor, input_tensor, group=None, async_op=False)

into this

def all_gather_into_tensor(output_tensor, input_tensor, group=None, async_op=False, **kwargs):

Now I just had to test it out
Using a test

    def setUp(self):
        # Set up distributed environment for testing
        os.environ['MASTER_ADDR'] = 'localhost'
        os.environ['MASTER_PORT'] = '12355'

        dist.init_process_group("nccl", rank=0, world_size=1)

    def tearDown(self):
        # Clean the environment
        dist.destroy_process_group()

    def test_all_gather_into_tensor_without_kwargs(self):
        # testing to see if kwargs work
        output_tensor = torch.tensor([1, 2, 3])
        input_tensor = torch.tensor([4, 5, 6])
        group = None
        async_op = False

        result = all_gather_into_tensor(output_tensor, input_tensor, group, async_op)

        self.assertIsNone(result) 

        with self.assertRaises(AttributeError):
            _ = output_tensor.kwargs

        with self.assertRaises(AttributeError):
            _ = input_tensor.kwargs

This would work on its own ofcourse with a little bit of changing in the code.
After that I spend tim back in the installation process. After spending literally a week trying to install and make it work. I couldn't anymore.

I had to run back to the planning stage and also see for all_gather_into_tensor_inplace would work with this but it all boils down to the main functionality of all_gather_into_tensor.

I learned about what BLOO, MPI and NCCL is and that it will only work with linux systems, I was thinking that using WSL would help me out but to no avail.

It feels kind of a letdown but I learned a lot from this release, even if it wasn't a success and I wasn't able to merge it.

I would happily accept the learning process and the opportunity I receive from this..

Progress report 0.4

Ronald R — Sun, 03 Dec 2023 00:26:38 +0000

This week I started wroking on the issue, I tried understanding the layers upon layers of code I need to look into.

I ended up a total of 4 functions I need to fully understand and how to make kwargs work into it.

all_gather_into_tensor
all_gather_tensor_inplace
legacy_allgather
all_gather_tensor
traceable_collective_remaps

From my understanding this is how it works.

We have all_gather_into_tensor as the main function we import it from torch.distributed.distributed_c10d and then we assign it as legacy_allgather from there it is called by traceable_collective_remaps to be rewritten in dynamo and then assign it to all_gather_tensor_inplace where all the variables and attributes would be placed finally into all_gather_tensor calling again all_gather_into_tensor to be placed in all_gather_tensor_inplace

Confusing? i know but thats the very very straight forward way of saying it.

Now I realize that we could just add the keyword kwargs** and be done with it but after analyzing is I saw that there was a total mismatch in arguments that might be causing the bugs, so I went on and ask ifit is happening with the another function. if my theory is correct and it is happening on another function then it means we have to go into all_gather_into_tensor and fix the problem from there if not then we just need to replace it straight forward.

I am very thankful that I learned testing because I know it is going to come in very useful for this one.

The first test I did was searching up wha the arguments looks like and why it wasnt accepting kwargs

and the result I got was this

First extra element 4:
'tag'

- ('args', 'kwargs', 'error', 'error_msg_dict')
+ ('output', 'input', 'group', 'async_op', 'tag', 'gather_dim')

meaning there are mismatches on the elements being parse so that might be one of the issue.

I still have to keep checking, i did hope to submit a PR this week but I need a few more things to understand, that is why I might hop on zoom call tomorrow with the dev team and try to understand what is wrong. Let's see.

For now we continue!!

The Plan 0.4

Ronald R — Wed, 29 Nov 2023 17:54:51 +0000

Release 0.4 is taking on the biggest problem, the biggest task, and the largest project we can for now. It's crazy to think that we are in the final legs of our term.

We have three things.

What I'll do
How I'll do it
The goals.

Let's start with the What

What I need to do is find a bug that I want to work with. I have no clue where to find it. so I placed in a few labels on github

archived:false is:open language:Python label:bug

pretty simple but straight forward I am always looking for python projects to work on, I had a few chances to take a look on some of the repositories I worked but it was not enough, well for me though. I sent a few comments on a few issues what I wanted to work on but there is really one issue I want to work on. Which was funccol collectives rewrite in dynamo does not work w/ kwargs #114179 which is an issue from pytorch.

Just a quick rundown on the bug we need to fix a bug. So this code is working it's fine

dist.all_gather_into_tensor(
                output_tensor=flat_tensor, input_tensor=input_flat_tensor_shard, group=group
            )

BUT

When we start rewritting it with dynamoDB an start using kwargs or keyword arguments it's not working.

dist.all_gather_into_tensor(
                flat_tensor, input_flat_tensor_shard, group=group
            )

Now the How

Let's discuss the how, I need to learn how dynamo rewriting works.they mentioned dynamically rewriting it with dynamo, which in all honestly I have no clue how dynamically re writing works so ill start there. I have nothing much to say here yet, just because the how is something I need to work on first.

they asked me to check out

def _traceable_collective_remaps():
    # We can't rely on importing from distributed, since it's not always built
    if torch.distributed.is_available():
        from torch.distributed._functional_collectives import (
            traceable_collective_remaps,
        )

        return traceable_collective_remaps
    return {}

So that's what im doing here first.

The goal

Well the goal is find a way to fix it and hopyfully merge it. It's not as easy as it sounds to be honest. I have to back track a bit and work my way up. I'm guessing I'm starting at -200 right now and not even at 0 just because I need to learn dynamo and being able to dynamically re write with it.

The goal hopefully I can submit a PR by next week but we will see.
For now let's get going and learn dynamo!

Hopefully I can give good update by next week.

okay See you!

Packaged and Released

Ronald R — Thu, 23 Nov 2023 04:21:47 +0000

This is the final lab for the semester and It has been such an amazing exprience, I really learned a lot from this course, while it may be te last lab our journey for OSD is not done yet we still have a big release coming up but for now, let's talk about Package and Release.

Packaging

Packaging It was quire interesting to know how this works because it always baffles me when i see a tar.gz extension and always wonder what it was, now i know that it was a matter of packaged.

For this Lab I used PypI to packaged my file, I had to edit my pyproject.toml file to properly packaged my project, I was so happy to find out that the guide the made was pretty easy and straight forward just because sometimes and THERE ARE times when they have a guide and it was all over the place.

had ensure pip was installed
created a toml file
registere in PyPI to get a package token
Packed using twine
Uploaded release

These are basically the steps they had me follow which was very straightforward

Release

I had a few Releases here with me. Though the latest one was the one where I got a feedback.

Feedback

I asked my partner to try and use Waypoint. I asked her to open up and README.md and follow the installation process and use its features and usage.

She told me that it was pretty straightforward and the directions are clear, no bugs were found during the testing phase. HOWEVER it was noted that readme.md could have been clearer with its features and tool description. so I had to clean it and ensure that it was clear and understandable. I didnt alter the code that much since it was somewhat bugfree and I ensure it was just because there are some that I worked on that are full of bugs and really unuseable that is why I decided early on if I am making a project I will ensure that it will have as minimal bugs as possible.

Installation

Link this is the link to my latest relaese. yes I have multiple, the easiest way to install this is basically download a package and extract them from there they can easily follow readme.MD

Ending

In conclusion this course was very interesting to me and was very awesome to learn because it provided me with a greater insight on software development. specially usng github since it is truly one of those things that is a must learn when it comes to software development.

I Tried..

Ronald R — Sun, 19 Nov 2023 19:18:15 +0000

With this lab I wanted to try to fix a table columns that is bleeding to another column. sounds easy enough. What I didn't anticipate that it was harder than it looked and was quite complicated that even the main collaborators found it to be harder than expected.Issue My goal was to dynamically get the longest name and set that as the size/width of the column

   max_width: dict[str, int | str] | str | None = None

was my initial plan but it did not quite work as expected because it was stll changing as the API data is being red and somewhat make it more cluttered. I was having trouble with this issue for a few days but ultimately I decided to just let another person work on it because it was really quite difficult for me to fix specially testing the data.

WHat I would have done differently

It would have been great to learn how to do unittest properly before I attemtped to do this, just because I was having trouble connecting to their servers since and load the data properly.

Someone took Over

When it was taken over by someone else the same ideaolgy that I wanted to implement was still there but I felt like it was bit of a hassle to work on this without the proper knowledge. all in all it was still very interesting even if I didn't get to merge thi on my own. I still learned a lot.

Thouht It would have been quite a work if I still proceeded and probably took a longer time if I decided to finish it, but it's fine for me atleast I tried even if I wasnt able to finish it properly.

I had to close the PR that I requested even though I didnt want to. But it did take about almost 2 weeks before this was finished all in all. So it was truly quite Hard and was not at all a "Good Issue" this word is become harder and harder to justify.

Intel CVE-bin-tool

Ronald R — Sun, 19 Nov 2023 18:42:31 +0000

I wanted to add this as part of my release 0.3 because this was the largest project I had to work on, and I dealt with multiple formatting issue before I was able to properly configure the module to includes docstring in it's proper format and style required.

It may not seemed a lot but let me explain

The issue docs: add missing docstrings to version_scanner.py was requested to properly maintain and understand what the module and its function doess. The reason why I wanted to include this because while it did not needed me to enhance or add a code but it required me to fully understand the module and function it self which took me a while. I know it might not seem much but to try to learn somethign you have now knowledge about is fairly challenging by it self, on top of that I had to traverse and figure out how other module is connected specially import is a necessity for this module.

I mean just take a look at the imports, I had to go through each function and try to understand what it does.

The Module version_scanner.py "Scans files for CVEs using CVE checkers" if you are not familliar CVE as The Common Vulnerabilities and Exposures (CVE) which is a system the identifies all vulnerabilities and threats related to the security of information of the systems. This module basically check all the threats that could potentially arise through the program and I have to try to learn and understand each function missing with docstrings.

I was lucky enough to fully understand and grasp what each function does and was able to add the required doc string for each function.

Before lab 7 came to place where we learned python black and lint I had to implement it and format this module which luckily wasn't that hard of a task.

A sample docstring I had to include was

"""process a Set of checker objects, run them on file lines, and find the information about the detected products and versions. It uses logging to provide debug and error information along the way. """

Wordy I know but that was needed to ensure that I detail it completely and correctly

I had to close my initital PR Updated doc strings to version_scanner.py #3417 and became troublesome when it was failing the tests they set up so I had to re-do the entire module again because of this and submitted another PR updating docstring with formatted with black. requesting new PR for version_scanner.py #3429. while it may seemed an easy task trying to contribute to a large scale project and knowing that somehow you need to give a little bit extra to your best was a bit nerver racking. it was because cve-bin-tool by intel is very widly used by a lot of people as of today which is Nov 12,2023 there has been over 2,000 commits and the last commit was done 3 days ago. I thought that maybe the reason it wasnt properly passing was because there was multiple merge commits that they did prior to me pushing my commits and I never properly synced my file, but I am not sure if that was even the case.

This work made me reflect on possible real life exprience, where everything you do will not go always as planned specially when working with projects and with others you have to make sure that the file you have is basically and always uptodate. while it doesn't need to be a coding enhancement or new feature addition to contribute working on projects and adding as simple as a docstring can be a challenge in of it self, where you have to try to learn and understand the module you are working on and ensure that you follow the requirements they asked for to be able to complete the requested task.

Are you sure? Feature Enhancement.

Ronald R — Sun, 19 Nov 2023 18:42:08 +0000

For this first part of release I decided to work on with DagsHub github in accordance to DagsHub

A platform where machine learning and data science teams can build, manage, and collaborate on their projects.

The reason why I choose this was I was truly decided to get my hands on anything that has machine learning and anything that includes working on datas.
The reason why I wanted to include this for this release 0.3 was because it was by sheer luck and a little bit of preserverance finding the issue that I was able to figure this out.

The Issue

Data Engine: Allow downloading all binary fields when qr.get_blob_fields is called with no args

When I referenced this in It was not "Good First Issue" I was not exagerating when I said it was not easy. This time let me break it down why that is. Aside from installing and re-installing over and over again and looking over thousands of files that has already a thousand commit was very daunting. The request was somewhat vague but understandable enough.

The way we were trained and thought these past few semester made me ready for this kind of task. where 1 liner was the way to go ever since I started my first semester, not much detail. it was similiar to "Create a function which does this" and that was pretty much it.

To ensure I got the right Idea I had to verify the functionality of the code. Luckily they were very responsive that I was able to determine that what needed was basically keeping the same functionality of the code but also being able to download the files without specyfying any arguments.

For me to be able to find the correct code to work on this was the path.

dagshub/data_engine/model/query_result.py

That is llike 3 folders to navigate just to get to the module. on top of that terms like self.datasource.fields, MetadataFieldType , and get_blob_field() where if you breakdown states that "In a binary large objects of data I need to get its class module datasource and its fields". I know that sounded daunting to me as well. Which in turn was the case that I needed to do.

In order to achieve this and finish the task I had to

Import from dagshub.data_engine.dtypes import MetadataFieldType

Remember this was my first ever contribution on github for opensource what a way to start.

After commiting the changes I was then asked to format using python Black which was a godsend when it was time to work on it since I was somewhat familliar with it the lab I had to work with python Black was something. After which Added an enhancement - issue 366 was finally merged.

Let's do a quick run down shall we.

Open source Project
Over 100 branches
Over 1000 Commits
Large Scale Project
Terminology I am unfamilliar
First ever Github Contribution to Open Source

That was truly a very interesting issue to work on luckily I was able to submit a PR and was able to merge it successfully. I learned a lot from this project and truly spent quite a good amount of time working on it luckily I didnt have much to do those days where I was able to spend time working on it for the entire day or two. This is truly something an experience I will never forget.

Automated Testng

Ronald R — Thu, 16 Nov 2023 20:07:40 +0000

For this lab we had to create a CI that would basically run the tests when we push into the repository after we requested a Pull Request. Seemed straight forward enough right? WRONG, for someone who is unfamilliar with CI, I had no clue where to start luckily I had our professor guide us how to do it.

So I started with implementation of the tests, I'll be honest It was quite trouble some because when I created the yaml file I did it manually and when I tried pushing it the first time it was giving me error Closed PR for CI testing as you can see right here. it was because the yaml file I had was different and was failing for some reason. so I had to re do the entire thing all over again.

I had to close the file, and requested another PR jsut to ensure it was done crrectly. I tried the github actions congifuration python for the workflow and it included all the things I needed. which also included flake8 which is my linter program which was convient enough since we had to implement an optional linter test test.

This is the main PR that I did, I was a bit sad because I was not able to implement my automatic failure test code because it was failling, maybe i should have merged it still but I did not want to make such a hassle for anyone else working on my repo. luckily it worked well with my lab because it was required for us to purposely fail a program which luckily enough it did and I was able to finish the first part of the testing.

For the second part this is where it got interesting. I tried working on like 3 repository and had their own issue. 1 was not properly set up and I tried eveyrthing I could to run it and it was still not working, I used 2 different machines and was not able to make it work, so I had to just move on. Next I worked on program but the CI wasnt set up yet, thinking to myself, I could make one for this repo but that means me doing their work for them, so I moved on again.

The third repository I worked for was a godsend, it had its own problem but was really manageable, I spoke with the owner of the repo and was appreciative enough to communicate with me to run the program properly, It was running all the tests without actually calling anything specific and, even if we did call in a specifi unittest to run it will still run everything. I reviewed te file and saw that there was a missng module in the tests folder which was a innit.py file tobe able to import the package or take import from otherm odule properly.

After implementing this module we were able to run the unittest individually, I created a simple unittest to test the argument passer of -v or -version. It was a simple code but it showed the functionality of the program and by implementin that init.py we were able to run the program properly.

All in all this lab was very helpful as it was interesting to learn CI and unittest as those are the things that I truly wanted to learn from the get go. Maybe to make things easier i should just work on pycharm but let's see. ON TO THE NEXT AND FINAL LAB!

Testing. Testing. 123..

Ronald R — Fri, 10 Nov 2023 01:01:06 +0000

For this lab I wanted to make a heavier techincal blog than just my regular blog just so I can get a feel how this blogging works.
3a68733 this was the commit I did for this week. I updated the contribution.MD file to show how to properly work on Waypoint. In this Lab we worked on tesing.

1st and Main commit

Added a testing framework for initial unit tests creation - during this commit I needed to somehow figure out a framework to use with my project. Since I was working with python I had a couple of options, I decided to ues nose2 and unittest for this.

Breaking down this code I needed to set up the framework where I would be able to use unnittest and also import the functions I needed to create the testing module.

import unittest
from waypoint import process_text, process_file, set_config
from testing import argtest

I had a total of 9 testing that I needed to ensure that all the function works. In those 9 I tested all the main core of the project which is convertion

String to Html
txt File to Html
Md file to Html
toml file processing
folder processing and conversion

the next test i needed was to ensure the system arguments condition are met properly

-h for help
-v for version
-c for config file
test to fail.

I added a test to fail to ensure that the program and testing are working properly. This was mostly for my personal agenda, just because testing is an area I am very unfamilliar that I needed to ensure that all if not most testing are done properly.

Now the testing depndicies i used was nose2 and unittest

python -m unittest discover -k "test_txt" # To test a string
python -m unittest discover -k "test_txtfile" # To test a txt file
python -m unittest discover -k "test_tomlfile" # To test a toml file
python -m unittest discover -k "test_folder" # To test a folder file

these codes where use to test the unit individually as oppose to all. I installed nose2 to actually run everything, yes I know unittest has the capability of it being able to run but i found it easier to use nose2 when running the test plus it provided me an option to add a watcher which sadly unittest did not.

Now for coverage It was simple enough to use, which was coverage.py to run it all. I realized I now have multiple way to test my program!

This was a very challenging lab as testing was something I really wanted to get my hands on but did not know where to start, looking back at my work during Hacktoberfest it took me days to actually figure out how to test the thing I did, which this came in sooner but oh well, at least now I know. On to the next one!!

Forem: Ronald R

DATA ANALYSIS - Highest Rated Movie in Imdb - Data set

Test

Conculsion

DATA ANALYSIS - Student Mental Health - A Reading

The Goal

The Strategy

Summary

DATA ANALYSIS - Student Mental Health - A Reading

The Goal

The Strategy

Summary

The PR END

The Issue

Progress report 0.4

The Plan 0.4

Let's start with the What

Now the How

The goal

Packaged and Released

Packaging

Tags

Release

Feedback

Installation

Ending

I Tried..

WHat I would have done differently

Someone took Over

Intel CVE-bin-tool

Are you sure? Feature Enhancement.

The Issue

Automated Testng

Testing. Testing. 123..