Forem: Marine

How we saved our partners 💵$460,000 and 2,5 months⏰ of work

Marine — Fri, 24 May 2024 09:16:35 +0000

In 2022, I worked on a data science project for a retailer. The project was to predict cashflows through better demand forecasting and inventory management. This project followed the common pitfalls of all data science and AI projects and made me rethink our strategy and tools.

I’ll tell you all about the mistakes I made ( and will never do again) and what tool I used to save money and time.

Common pitfalls in AI & Data projects

Over the years, I’ve worked on many AI and dastascience projects, delivering substantial ROI through algorithms and AI models. Despite the AI hype, many non-software companies struggle with successful AI strategies, often limited to standard data projects and few impactful AI deployments, resulting in uneven AI adoption.

Here are the main causes:

Siloed Teams: There’s often a big disconnect between the data scientists and the end-users. There are valid reasons for having different roles and the need for specialization. However, it's important to recognize that in real projects, this leads to a significant gap between data scientists and end-users. Each group tends to use different technology stacks; for instance, data scientists usually work with Python, while IT developers might use JavaScript, Java, Scala, and other languages. This influences teamwork in taking more time and making teamwork tricky.

These are the various groups involved in a typical AI/ DS project:

Getting acceptance from the end-users / business users: If end-users aren't part of the development process, they might not use the software once it’s up and running.

What I needed:

1- Go All-In on Python: It's easy to learn and works well with other tech. It is at the heart of the AI stack and ideal for integrating with other environments. We considered libraries like Streamlit, which is excellent for prototype quick applications, but we quickly felt the limits of this library for performant multi-user applications.

2- Better Interaction with end-users: It is critical to ensure the software works well for users and track how happy they are with it.

The solution:

Taipy came from the need for a easy tool, like Python, and strong enough for big projects. It handles loads of data fast, can be tailored to specific business needs, and connects data scientists with business users. It also makes decision-making smarter with features that let users play with different scenarios.
Now let's go into detail:

- Answer to the siloed teams

The obvious answers would go towards these points:

Standardize on a single programming language.
Provide an easy-to-learn and use programming experience for all skill levels.
Python is ideal for AI, with many user-friendly libraries, though they often face performance and customization issues.

For instance, libraries like Plotly Dash offer full-code solutions, while Streamlit or Gradio are easier but lack performance and flexibility. Python developers shouldn't have to choose between productivity and performance.

We created Taipy to combine ease of development with high performance and customization.

- Answer to bringing back the end-user to the center of the project

Addressing the two key points is crucial:

Smooth end-user interaction with backend algorithms.
For smooth interaction, end-users need control over algorithm variables through the GUI, the ability to run algorithms with different parameters, and the option to compare results and track KPI performance over time. Taipy addresses this with the 'scenario' concept, storing all data elements and enabling users to track runs, revisit past scenarios, and analyze results.
Easy tracking of business-user satisfaction.
For tracking satisfaction, Taipy's scenario function bridges the gap between end-users and data scientists by providing access to all runs and allowing end-users to tag and share scenarios with data scientists. This feature enhances software acceptance beyond basic testing and drift detection.

Taipy - Build Python Data & AI web applications

Support open-source and give Taipy a star: https://github.com/Avaiga/taipy

What about the results?

Implementing Taipy transformed our approach to managing cash flow and demand forecasting. The tool improved our processing capabilities by leaps and bounds and gained quick acceptance from end-users thanks to its intuitive design and relevance to their daily tasks.
We went from using 4 full-time developers to 1.5. The initial team was eclectic, with specialties ranging from Javascript and Java and Python.
The use of Taipy enabled the data scientist using Python to create a full-blown application ready for use by the end-users. This facilitated communication and reduced the siloed team process common to all AI projects. A gain of time and money is crucial to the success of any project.

For the concrete results, Taipy reduced the overall costs by a factor 10!

Project Phase	Budget	IT Staff	Duration
Initial Setup	$600K	4	8 months
With Taipy	$60K	1.5	2 months

Achievements:

4x Faster Projects: We sped up everything from start to finish and spent less on keeping things running.
10x Cheaper: Most of the tech work was done by Python developers, reducing the need for help from other departments and making project management a breeze.

Check out these applications made with Taipy: https://docs.taipy.io/en/latest/gallery/

Conclusion

Hope you enjoyed a little context on the creation of Taipy and what we to achieve with it.

Building A ChatGPT Wizard with MistralAI Using Taipy

Marine — Tue, 21 May 2024 13:58:23 +0000

TL; DR

Let's learn how to build a simple chatbot using the Taipy GUI library and the Mistral-7B-Instruct-v0.1-GGUF language model from the ctransformers library.

The walkthrough

loading the language model
generating responses to user prompts
updating & clearing conversation history
application styling

By the end of this article, we will have a basic understanding of how to build a chatbot using these tools.

Loading the Mistral-7B-Instruct-v0.1-GGUF model

Mistral 7B is a super-smart language model with 7 billion parameters!
It beats the best 13B model, Llama 2, in all tests and even outperforms the powerful 34B model, Llama 1, in reasoning, math, and code generation.
How?
Mistral 7B uses smart tricks like grouped-query attention (GQA) for quick thinking and sliding window attention (SWA) to handle all sorts of text lengths without slowing down.

Source: Mistral.AI Docs

And there's more! Mistral AI Team fine-tuned Mistral 7B for specific tasks with Mistral 7B – Instruct.
It outshines Llama 2 13B in chat and rocks both human and automated tests.
The best part? Mistral 7B – was released under the Apache 2.0 license.

Download GGUF files using ctransformers

Step 1: Install ctransformers

With no GPU acceleration



pip install ctransformers

or install with ctransformers with CUDA GPU acceleration



pip install ctransformers[cuda]

or install with ctransformers with AMD ROCm GPU acceleration (Linux only)



CT_HIPBLAS=1 pip install ctransformers --no-binary ctransformers

or install with ctransformers with Metal GPU acceleration for macOS systems only



CT_METAL=1 pip install ctransformers --no-binary ctransformers

Load the model

All set? Let's run the code below to download and send a prompt to the model. Make sure to free up space on your computer and connect to a good internet connection.



# import the AutoModelForCausalLM class from the ctransformers library
from ctransformers import AutoModelForCausalLM

# load Mistral-7B-Instruct-v0.1-GGUF, Set gpu_layers to the number of layers to offload to GPU. The value is set to 0 because no GPU acceleration is available on my current system.
llm = AutoModelForCausalLM.from_pretrained("TheBloke/Mistral-7B-Instruct-v0.1-GGUF", model_file="mistral-7b-instruct-v0.1.Q4_K_M.gguf", model_type="mistral", gpu_layers=0)

# call the model to generate text.
ask = 1000 
turn = 0
while turn < ask:
    user = input("Enter your message: ")
    print(llm(user))

The model will continue the statement as follows,

Say Hello Taipy!

Taipy is an open-source Python library that makes it simple to create data-driven web applications.
It takes care of the visible part (Frontend) and the behind-the-scenes (Backend) operations.
Its goal is to speed up the process of developing applications, from the early design stages to having a fully functional product ready for use.

Source: Taipy Docs

Requirement: Python 3.8 or later on Linux, Windows, and Mac.

Star ⭐ the Taipy repository

Your support means a lot🌱, and helps us in many ways, like writing articles! 🙏

Installing Taipy:

Open up a terminal and run the following command, which will install Taipy with all its dependencies.



pip install taipy

We're set!
Let's say hello to Taipy!



# import the library
from taipy import Gui

hello = "# Hello Taipy!" 

# run the gui
Gui(hello).run()

Save the code as a Python file: e.g., hi_taipy.py.
Run the code and wait for the client link http://127.0.0.1:5000 to display and pop up in your browser.
You can change the port if you want to run multiple servers at the same time with Gui(...).run(port=xxxx).

Create a chat interface with Taipy

Now we are familiar with Taipy, let's get our hands dirty and build our chat interface.

Step 1: Import the AutoModelForCausalLM class from the ctransformers library

In this step, we import the AutoModelForCausalLM class from the ctransformers library, which is used to generate text using pre-trained language models.



from ctransformers import AutoModelForCausalLM

Step 2: Import the Taipy library
In this step, we import the Taipy GUI library, which is used to build the user interface for our chatbot.



from taipy.gui import Gui, notify

Step 3: Load the Mistral-7B-Instruct-v0.1-GGUF model



llm = AutoModelForCausalLM.from_pretrained("TheBloke/Mistral-7B-Instruct-v0.1-GGUF", model_file="mistral-7b-instruct-v0.1.Q4_K_M.gguf", model_type="mistral", gpu_layers=0)

Step 4: Initialize the prompt and response variables

In this step, we initialize the prompt and response variables as empty strings.



prompt = ""
response = ""

Step 5: Define the chat function

In this step, we define the chat function, which is called when the user clicks the "Chat" button in the user interface. This function takes the current state of the GUI as an input, generates text using the pre-trained language model based on the user's prompt, and updates the response variable in the state.



def chat(state):
    notify(state, 'info', 'Thinking...')
    state.response = llm(state.prompt)

Step 6: Define the user interface

Time to define the user interface for our chatbot using the Taipy GUI library. The user interface consists of an input field where the user can enter a prompt, a "Chat" button that triggers the chat function, and a display area where the chatbot's response is shown.



page = """
# Chatbot Wizard! {: .color-primary}
Enter Prompt: <|{prompt}|input|> <br />
<|Chat|button|class_name=plain mt1|on_action=chat|> <br />
MistralAI: <br /> <|{response}|>
"""

Step 7: Run the Taipy GUI application

Now let's run the Taipy GUI application using the run method.



Gui(page).run(debug=True)

Full Code



# import the AutoModelForCausalLM class from the ctransformers library
from ctransformers import AutoModelForCausalLM

# import taipy library
from taipy.gui import Gui, notify

# load Mistral-7B-Instruct-v0.1-GGUF, Set gpu_layers to the number of layers to offload to GPU. The value is set to 0 because no GPU acceleration is available on my current system.
llm = AutoModelForCausalLM.from_pretrained("TheBloke/Mistral-7B-Instruct-v0.1-GGUF", model_file="mistral-7b-instruct-v0.1.Q4_K_M.gguf", model_type="mistral", gpu_layers=0)

# initialize the `prompt` and `response` variables as empty strings.
prompt = ""
response = ""

def chat(state):
    notify(state, 'info', 'Thinking...')
    state.response = llm(state.prompt)

page = """
# Chatbot Wizard! {: .color-primary}
Enter Prompt: <|{prompt}|input|>
<|Send Prompt|button|class_name=plain mt1|on_action=chat|> <br />
MistralAI: <br /> <|{response}|>
""" 

Gui(page).run(debug=True)

Here it is, a simple chat interface!

Let's level up our application to become a chatbot, as we imagine.

Mistral AI Chatbot

Step 1: In this step, we initialize the prompt and response and the conversation



prompt = ""
response = ""
conversation = {
    "Conversation": ["Hello", "Hi there!   What would you like to talk about today?"]
}

Step 2: Update the chat function



def chat(state):
    # Notify the user that the chatbot is thinking
    notify(state, 'info', 'Thinking...')

    # Generate a response using the loaded language model
    response = llm(state.prompt)

    # Add the user's prompt and the bot's response to the conversation history
    state.conversation["Conversation"].append(state.prompt)
    state.conversation["Conversation"].append(response)

    # Update the conversation object to contain the entire conversation history
    state.conversation = {"Conversation": state.conversation["Conversation"]}

    # Clear the user's input prompt
    state.prompt = ""

    # Notify the user that the bot has generated a response
    notify(state, 'info', 'Response received!')

Step 3: Add clear_conversation function to clear the conversation history

The function sets the state.conversation object to a new dictionary with a single key-value pair, where the key is Conversation and the value is an empty list.
This effectively clears the conversation history, as the state.conversation object is now an empty dictionary with a single key-value pair containing an empty list.
The updated state.conversation object will be reflected in the chatbot UI, showing an empty conversation history.



def clear_conversation(state):
    state.conversation = {"Conversation": []}

Step 4: ZHUZH it up
Let's define the layout of the user interface for the Chatbot.
Let's add a logo by downloading and saving it in the same directory as the script.
Then attach clear_conversation to the New chat button.

Styling with CSS

Now, let's style our chat ui by floating the response the left and the prompt to the right hand side.

Step 1: Create a CSS file with the same title as the python file and save it in the same directory.



.mistral_mssg td {
    position: relative;
    display: inline-block;
    margin: 10px 10px;
    padding: 15px;
    background-color: #ff8c00;
    border-radius: 20px;
    max-width: 80%;
    box-shadow: 0 4px 8px 0 rgba(0, 0, 0, 0.2), 0 6px 20px 0 rgba(0, 0, 0, 0.19);
    font-size: medium;
  }

  .user_mssg td {
    position: relative;
    display: inline-block;
    float: right;
    margin: 10px 10px;
    padding: 15px;
    background-color: #9400D3;
    border-radius: 20px;
    max-width: 80%;
    box-shadow: 0 4px 8px 0 rgba(0, 0, 0, 0.2), 0 6px 20px 0 rgba(0, 0, 0, 0.19);
    font-size: medium;
  }

.flexy{
    display: flex;
    justify-content: center;
    max-width: 50vw;
    margin: 4em auto;
    align-items: center;
}

Step 2: Create the style_conv function
The style_conv function is a callback function that is used to apply styles to the conversation history table in the Taipy GUI. It takes three arguments: state, idx, and row.

The state argument is a dictionary containing the GUI's current state, including the conversation history.
The idx argument is the index of the current row in the table, and the row argument is the index of the current column in the table.

The function checks the value of the idx argument to determine which style to apply to the current row. If idx is None, the function returns None, indicating no style should be applied.

If idx is an even number, the function returns the string user_mssg, corresponding to the CSS class for the user's prompts. If idx is an odd number, the function returns the string mistral_mssg, corresponding to the CSS class for the chatbot's responses.

Here is the code for the style_conv function:



def style_conv(state, idx: int, row: int) -> str:
    if idx is None:
        return None
    elif idx % 2 == 0:
        return "user_mssg"  # return user_mssg style
    else:
        return "mistral_mssg"  # return mistral_mssg style

To use the style_conv function in the Taipy GUI, we need to pass it as the value of the style attribute in the table element. For example:



<|{conversation}|table|style=style_conv|show_all|width=100%|rebuild|>

Step 3: Add a sidebar

Redefine the page to add the sidebar.



page = """
<|layout|columns=300px 1|
<|part|render=True|class_name=sidebar bg_black|
# Chat **Wizard**{: .color-primary} # {: .logo-text}
<|New Chat|button|class_name=fullwidth plain|on_action=clear_conversation|>

### History
<|{history}|table|show_all|>
|>

<|part|render=True|class_name=p2 align-item-bottom table|
<|{conversation}|table|style=style_conv|show_all|width=100%|rebuild|>

<|part|class_name=card mt1|
<|{prompt}|input|label=Ask anything...|class_name=fullwidth|on_action=chat|>
<|Send Prompt|button|class_name=plain mt1 fullwidth|on_action=chat|>
|>
|>
|>
"""

Final thoughts

In conclusion, this article demonstrated how to build a simple chatbot using the Taipy GUI library and the Mistral-7B-Instruct-v0.1-GGUF language model from the ctransformers library. The code provided shows how to load the language model, generate responses to user prompts, update the conversation history, and clear the conversation history. The chatbot's UI, built using the Taipy GUI library, provides a user-friendly interface for interacting with the chatbot. Overall, this article provides a useful starting point for building more sophisticated chatbots using these Taipy.

Resources:

HuggingFace: Mistral-7B-Instruct-v0.1-GGUF

Taipy: Taipy Docs

Python for AI : Cheatlist

Marine — Wed, 20 Mar 2024 16:15:40 +0000

TL;DR

Getting into AI and ML without using Python is very difficult and, might I say, even impossible.
So here's a list of prominent Python libraries for your AI and ML models.
These libraries have been, and continue to be, institutions in the AI landscape.

I. Building an application for your AI

What is ML & AI if you can't share it with non-data scientists/programmers?
Let's start with two major frameworks that help showcase your results through GUIs.

1- Taipy

Taipy was created to give data scientists the skills to develop production-ready applications.

This open-source Python library is designed for easy development for front-end (GUI) and ML/Data pipelines. Don’t compromise on customization, performance, and scaling.

Star ⭐ the Taipy repository

Your support means a lot🌱, and helps us in many ways, like writing articles! 🙏

2- Gradio

This Python library facilitates the quick sharing of AI/ML models through easy-to-create basic applications. It’s a great way to showcase your model quickly.

Star ⭐ the Gradio repository

II. AI and ML frameworks

Now let's enter the main part of the article, the major, most important Python libraries that'll help getting into ML/AI.

3- Scikit learn

This might be Python’s top 3 most famous libraries, and rightfully so.
Sklearn is a reference in Machine Learning. It includes different models such as K-means clustering, regression, and classification algorithms.
It also excels in dimension reduction techniques.
Sklearn also provides data selection and validation functions. It's easy to learn/use and should be your go-to ML library during your data science journey.

Star ⭐ the Sklearn repository

4- Tensorflow

This library is a must-know for Neural Network modeling. Perfect when dealing with unstructured data such as image classification or NLP (Natural Language Processing). TensorFlow is widely used in research and industries as it provides a complete API for designing and manipulating Neural Networks.

Star ⭐ the TensorFlow repository

5- Pytorch

Pytorch is known for its more significant focus on natural language processing and a more Pythonic feel, reducing the steep learning curve for TensorFlow.

Star ⭐ the PyTorch repository

6- Keras

Keras is a high-level API that runs on top of frameworks such as TensorFlow. If starting with Neural Networks, start with Keras. It is ideal for quick implementations as it simplifies the implementation process, making it the best beginner-friendly option for Neural Network implementation.

Star ⭐ the Keras repository

7- FastAI

This library simplifies training fast and accurate neural nets; it’s built on top of PyTorch.

Star ⭐ the FastAI repository

8- FLAML

This library is a game changer. It finds and tests the optimal hyperparameters and machine learning models for your data and use cases.

Star ⭐ the FLAML repository

9- Catboost

This library, standing for Categorical Boosting, is the way to go if your dataset predominantly consists of categorical data. This library will remove the preprocessing headache of complex one-hot encoding, eliminating the need to preprocess categorical data. It can provide better accuracy than XGBoost when running with default parameters.

Star ⭐ the Catboost repository

10- PyCaret

This is an excellent Machine Learning automation tool. It automates all your ML workflows easily as it is low code.

Star ⭐ the PyCaret repository

11- H20

H20 is a user-friendly machine-learning platform for more innovative applications. It is fast and scalable. It includes algorithms for Deep Learning, GLM, PCA, RulkeFIt, etc…

Star ⭐ the H20 repository

12- XGBoost

XGBoost is one of the most popular libraries regarding Machine Learning algorithms.
This gradient-boosting library is widely used in real-life use cases, particularly for tabular data.
It is a favorite among Kaggle competition winners.
This library includes regression and classification algorithms but also provides feature selection tools.

Star ⭐ the XGBoost repository

13- TPOT

This library for AutoML will optimize your Machine Learning pipelines using genetic programming.

Star ⭐ the TPOT repository

14- ChatterBot

Quickly build your chatbot with this library. You’ll be able to enhance your user engagement and interactions and services.

Star ⭐ the ChatterBot repository

III. Natural Language Processing (NLP)

15- NTLK

NLTK is an essential toolkit for Natural Language Processing.
NLTKs' key features include processing and manipulating text( tokenization, stemming, etc.…) and classifications with NLP tasks for sentiment analysis, for example.

Star ⭐ the NLTK repository

16- SpaCy

It is the newer kid on the block, focusing on making NLP more accessible and user-friendly.
The library optimized the process to guarantee incredible speed and efficiency.

Star ⭐ the SpaCy repository

17- Gensim

This library specializes in topic modeling and document similarity—a good fit for your unsupervised text use cases and tasks.

Star ⭐ the Gensim repository

18- HF transformers

This is your tool for advanced NLP tasks. This library has state-of-the-art natural language processing models and algorithms.

Star ⭐ the HF transformers repository

IV. Model Visualization & Evaluation

19- Matplotlib

Matplotlib is the main widget library in Python, and for a good reason.
Matplotlib allows the plotting of 2D graphs with a wide range of chart types and also allows for significant customization.
The fine-grain control of the elements is a real advantage of this library.

Star ⭐ the Matplotlib repository

20- imbalanced-learn

This library gives you tools for dealing with imbalanced data—a life-saver when your dataset is far from balanced.

Star ⭐ the imbalanced-learn repository

21- SHAP

This library helps generate an explanation of your model’s output. It’s a great way to bring in some interpretability in black-grey box models.

Star ⭐ the SHAP repository

22- Missingno

It is a great solution to identify missing values in your data. Missingo helps you visualize them quickly, making this process simpler and more efficient.

Star ⭐ the Missingno repository

23- Lazy predict

It helps build baseline models to compare and evaluate without extensive code. A great tool for newbies in ML. It is low-code and takes care of the parameter-tuning hassle for you.

Star ⭐ the Lazy Predict repository

24- Category Encoders

This library will help you deal with categorical data. It helps build baseline models to compare and evaluate without extensive code. This library is Sklearn compatible.

Star ⭐ the Category Encoders repository

V. Computer Vision

25- OpenCV

OpenCV provides various algorithms around real-time computer vision.
You can process multiple formats, including objects, humans, and handwriting.

Star ⭐ the OpenCV repository

Have fun!

Exploring these libraries will allow you to handle most AI and ML use cases. Python libraries go beyond tools, they participate in the continuous innovation in the AI landscape, so make sure to support them!

Python libraries you need to know in 2025

Marine — Tue, 12 Mar 2024 13:46:51 +0000

TL;DR

Are you getting into Python? It’s a jungle!
You have libraries just about anything you can think about - from creating games to building web applications.

With this list, get a quick idea of 50 standard Python libraries and what they do whether you’re just getting started or looking to deepen your Python game.

1. Taipy

Taipy has been designed to expedite application development, from initial prototypes to production-ready applications.
This open-source Python library is designed for easy development for both front-end (GUI) and ML/Data pipelines.
It is low code and designed for any pythonista.

Star ⭐ the Taipy repository

Your support means a lot🌱, and helps us in many ways, like writing articles! 🙏

2. NumPy

Essential for numerical computations, supporting large, multi-dimensional arrays and matrices. This library is part of Python royalty.

Star ⭐ the Numpy repository

3. Pandas

A cornerstone for data manipulation and analysis, offering intuitive data structures and operations for manipulating numerical tables and time series. Another Python indispensable library, a must-know library.

Star ⭐ the Pandas repository

4. Matplotlib

A versatile tool for creating a wide range of static, minimal, and interactive visualizations. A lot of parameters to play with, this library is very useful when plotting ML and AI graphs.

Star ⭐ the Matplotlib repository

5. SciPy

Specialized in technical and scientific computing, with Scipy you can do optimization, integration, interpolation, and more.

Star ⭐ the SciPy repository

6. Scikit-learn

A go-to library for machine learning, providing a wide range of supervised and unsupervised learning algorithms. The only library you should know when starting with Machine Learning.

Star ⭐ the Scikit-learn repository

7. TensorFlow

A comprehensive framework for machine learning offers various tools, libraries, and community resources. THe learning curve might be a little steep, but TF is important to know in the Python and ML landscape.

Star ⭐ the TensorFlow repository

8. PyTorch

Favored for both academic research and production due to its flexibility, offering dynamic neural network creation and manipulation.

Star ⭐ the PyTorch repository

9. Keras

A high-level API for building and training deep learning models, designed to facilitate building and working with neural networks.

Star ⭐ the Keras repository

10. Requests

Simplifies the process of making HTTP requests, making web scraping and API consumption more accessible.

Star ⭐ the Requests repository

11. Beautiful Soup

A tool for web scraping that facilitates data extraction from HTML and XML files.

Star ⭐ the Beautiful Soup repository

12. Flask

A lightweight and extensible web framework, making it ideal for building small to medium web applications.

Star ⭐ the Flask repository

13. Django

Designed for rapid development and clean, pragmatic design, this high-level framework.

Star ⭐ the Django repository

14. Selenium

This library automates web browsers, enabling the simulation of actual user actions for testing web applications.

Star ⭐ the Selenium repository

15. Pygame

Provides Python modules for writing video games, including graphical and sound libraries.

Star ⭐ the Pygame repository

16. Pillow (PIL Fork)

Extends Python Imaging Library capabilities, supporting various image file formats.

Star ⭐ the Pillow repository

17. SQLAlchemy

This library offers a full suite of tools for working with databases through Python, providing a robust ORM layer and SQL expression language.

Star ⭐ the SQLAlchemy repository

18. PySpark

As you can tell from the name, this library brings the power of Apache Spark to Python, facilitating big data processing and analysis with a Pythonic approach.

Star ⭐ the PySpark repository

19. Dash

Enables the creation of analytical web applications directly in Python without requiring deep knowledge of web development.

Star ⭐ the Dash repository

20. Plotly

Specializes in creating interactive and visually appealing graphs and charts suitable for web and mobile applications.

Star ⭐ the Plotly repository

21. Nltk

This library makes natural language processing accessible and easy to use.

Star ⭐ the Nltk repository

22. SpaCy

Offers industrial-strength natural language processing capabilities with pre-trained models for many languages.

Star ⭐ the SpaCy repository

23. Gensim

Focused on unsupervised topic modeling and natural language processing, you can use this library to analyze document similarity.

Star ⭐ the Gensim repository

24. PyTest

A robust framework for writing small to complex functional tests, enhancing test readability and maintainability.

Star ⭐ the PyTest repository

25. unittest

The built-in framework for constructing and running tests mirroring the xUnit architecture found in other languages.

Unitest is built-in.

26. Fabric

Simplifies SSH for application deployment or system administration tasks, automating remote shell commands.

Star ⭐ the Fabric repository

27. Vizzu

Aimed at animated data visualizations and storytelling, Vizzu is the go-to library to create dynamic and interactive charts.

Star ⭐ the Vizzu repository

28. Polars

A DataFrame library optimized for performance and efficiency, capable of easily handling large datasets.

Star ⭐ the Polars repository

29. Docker-Py

Provides Pythonic access to the Docker Remote API, enabling automation of Docker container management.

Star ⭐ the Docker-Py repository

30. OpenCV

A staple in computer vision and image processing, offering a comprehensive suite of algorithms and tools.

Star ⭐ the OpenCV repository

31. Scikit-image

Dedicated to image processing, it extends the capabilities of SciPy and NumPy to the visual landscape.

Star ⭐ the Scikit-image repository

32. SymPy

This library is made for your symbolic computation, offering features ranging from algebraic solving to calculus.

Star ⭐ the SymPy repository

33. Virtualenv

Essential for creating isolated Python environments and managing project dependencies cleanly.

Star ⭐ the Virtualenv repository

34. Click

Streamlines the creation of command-line interfaces, promoting code that is both composable and easy to extend.

Star ⭐ the Click repository

35. Argparse

Facilitates the parsing of command-line arguments, which is essential for CLI application development.

Argparse is built-in.

36. Logging

Offers a flexible logging system, from simple logging to complex per-module configurations.

Logging is built-in.

37. PyYAML

Handles YAML files, supporting serialization and deserialization of Python objects to and from YAML.

Star ⭐ the PyYAML repository

38. xlrd/xlwt

Ideal for reading and writing Excel files, bridging the gap between Python and Excel documents.

Star ⭐ the xlrd repository
Star ⭐ the xlwt repository

39. Pandas-Profiling

Generates comprehensive profile reports from pandas

Star ⭐ the Pandas-Profiling repository

40. TQDM

Tqdm is a wrapper to any loop that will track the advancement with a progress bar.

Star ⭐ the TQDM repository

41. Faker

Need fake data that looks real? Faker's got your back.

Star ⭐ the Faker repository

42. Flake8

A must-need library to keep your code cleaner with easily to implement style checks.

Star ⭐ the Flake8 repository

43. Black

Bring your code formatting to the next level.

Star ⭐ the Black repository

44. Mypy

It's like having a grammar teacher for your code but for types.

Star ⭐ the Mypy repository

45. Pydantic

The library you need to validate your Python scripts.

Star ⭐ the Pydantic repository

46. FastAPI

FastAPI is a web framework for building RESTful APIs.

Star ⭐ the FastAPI repository

47. Catboost

Gives your machine-learning models ways to handle categorical data.

Star ⭐ the Catboost repository

48. Seaborn

Making elevated data visualization.

Star ⭐ the Seaborn repository

49. Turtle

Brings programming to life with cool graphics and animations. A great way to learn and get started with Python.

Turtle is built-in.

50. Asciimatics

Asciimatics is a library that allows you to create full-screen text UIs.

Star ⭐ the Asciimatics repository

Final thoughts

And there you have it!
A sneak peek in the Python jungle! Knowing most of these libraries sets you up to tackle any project! So get at it, Pythonista!

+10 Resources to Empower Women in Technology

Marine — Wed, 06 Mar 2024 09:46:35 +0000

TL;DR

With March 8th coming up, it’s always a time for me to get a little reflective on my experience as a woman in tech.

I’ve been working in tech for more than five years.
I started as a Data Scientist, and now I’m exploring and loving the DevRel 🥑 role for Taipy.
Needless to say, evolving in the tech scene has been a ride full of ups, downs, and everything in between.

International Women’s Day isn’t just a date on the calendar; this day is essential for us to question how things can improve. A day for celebration, reflection, and change!

This day makes me question my own evolution.
Progressing in tech has been rewarding and exhilarating, though I’ve had to overcome impostor syndrome and continuously find a sense of belonging.
Yet, these challenges have shaped the woman I am today and honed my greatest strengths.

Throughout my evolution, I’ve discovered an incredible support system within various communities and resources that I’m excited to share with you:

Communities

Pyladies

As a Python enthusiast, Pyladies holds a special place in my heart. I have attended their meetups in Paris and London, and they host events in 50+ cities worldwide. As the name reveals, this group is an active part of the open-source Python community. A great place to be up-to-date on the newest trends in Python and even better your coding skills with their workshops.

Check out PyLadies

Women in Machine Learning and Data Science

Women in Machine Learning and Data Science supports and promotes women interested in advancing in the Machine Learning and data science landscape.
Attending their events in France and the UK has been an excellent opportunity to learn the latest trends and connect with fellow data science enthusiasts.

Check out WiMLDS

Women in Tech

This global organization works to help women embrace technology. They have a global vision and push for empowerment and closing the gender gap.

Check out Women in Tech

Women Who Code

Created to empower women in technical careers. They have coding events and resources focusing on helping women in their career paths. If you feel you need some guidance carrer-wise, don't hesistate to check their website out.

Check out Women Who Code

Conferences

Grace Hopper Celebration

The most significant event gathering women in technology. This event is named after an inspirational computer-scientist and showcases women's careers and research in the computing landscape.

Check out Grace Hopper Celebration

Women In Tech Festival

A dynamic event that showcases women in tech through inspiring talks, networking, and workshops.

Check out Grace Hopper Celebration

Women+ in Data and AI

This was one of my favorite conferences last year. The energy was

inspiring— a modern take on conferences showcasing underrepresented genders' contributions in the field.

Check out the Women in Tech Festival

Podcasts

Women in Tech Podcast

A podcast that will inspire you with great stories from women in tech, from their journeys to their challenges, and some well- constructed advice.

Check out the Women in Tech Podcast

Tech Girls Cast

Another inspirational podcast, it will take you through the stories of women and inspire you in your daily life.

Check out the Tech Girls Cast

She Talks Tech

This podcast brings powerful conversations with key women in the sector. They cover subjects around leadership and personal growth.

Check out the She Talks Tech podcast

Subreddits

/girlsgonewired

A community for women and non-binary people in STEM. You can share your story and ask for advice. A subreddit to use as a support system.

Check out the Girls Gone Wired subreddit

/womenintech

A great subreddit to discuss your experiences, challenges and achievements in the tech industry. Shine and support women in the subreddits.

Check out the Women in Tech subreddit

/LadiesOfScience

It's a support subreddit for women in tech. You'll find testimonies, advice, and showcases.

Check out the Ladies if Science subreddit

Final thoughts

My tech journey has been a mix of sometimes feeling like an outsider and learning to embrace my strengths and uniqueness. These communities have played a part in this evolution.

Let’s continue to break barriers and make sure the tech world is inclusive and welcoming.

Don't hesitate to share your favorite resources in the comments!

How To Create an AI Photo App with Python

Marine — Tue, 27 Feb 2024 16:47:06 +0000

Let's learn how to build an image recognition application using Python and Taipy.
We'll start by developing the model, and then we will use Taipy to build a Graphical User Interface (GUI) to use it.
The application will allow us to upload an image, and it will identify the content using our trained model.

What is a Neural Network Builder

Let’s start!
The first phase is to create a Neural Network for image classification.
We will use a Neural Network Builder from TensorFlow and the CIFAR-10 dataset.
Tensorflow is an essential for Artificial Intelligence library to develop and train our neural network.

Star ⭐ the TensorFlow repository

The dataset comprises +50,000 images and is crucial for training image recognition models.
We will be able to put our model to the test with Taipy, and build an application.

Star ⭐ the Taipy repository

Your support means a lot🌱, and really helps us in so many ways, like writing articles! 🙏

Our model will be trained on the ten categories present in the CIFAR dataset:

airplane ✈️
automobile 🚗
bird 🦜
cat 🐈
deer 🦌
dog 🐶
frog 🐸
horse 🐴
ship ⚓

Our model will be able to classify images into these ten categories.

Creating a Neural Network Builder

Prerequisites

Python- The Python programming language should be available on your computer
virtualenv - A tool for creating isolated virtual Python environments I will be using virtualenv for this project; however, you can use your preference, like venv, or Conda, and adapt your commands.

IMPORTANT : Depending on your setup, you might need to use the command python or python3 when running commands in the terminal

Setup

Ok, time to build!
Run these commands to set your project up:

mkdir ml-photo-app
cd ml-photo-app
mkdir neural-network-builder
cd neural-network-builder
virtualenv venv
source venv/bin/activate
cd venv

Now let's install the real deal, two Python libraries: TensorFlow and numpy- a library for mathematical operations on arrays.

Use the following command:

pip install tensorflow numpy

Download CIFAR dataset

But first, data!
Now, let’s download the CIFAR-10 dataset from here.

The one we need is the CIFAR-10 Python version.
When you download and unzip the files, you should see a folder named cifar-10-batches-py.
Copy that folder and all the files inside the project we just created: neural-network-builder/venv.

Python script: generate-model.py

Create a file called generate-model.py.
This will be our Python script for the model training and exporting.

Add the code below to the generate-model.py.

import os
import shutil
import pickle
import numpy as np
import tensorflow as tf
from tensorflow.keras import datasets, layers, models

def load_cifar10_data(data_dir):
    train_images = []
    train_labels = []

    for i in range(1, 6):
        with open(os.path.join(data_dir, f'data_batch_{i}'), 'rb') as f:
            data_dict = pickle.load(f, encoding='bytes')
            images = data_dict[b'data']
            labels = data_dict[b'labels']

            train_images.extend(images)
            train_labels.extend(labels)

    train_images = np.array(train_images).reshape(-1, 3,
                                                  32, 32).transpose(0, 2, 3, 1)
    train_labels = np.array(train_labels)

    with open(os.path.join(data_dir, 'test_batch'), 'rb') as f:
        data_dict = pickle.load(f, encoding='bytes')
        test_images = data_dict[b'data'].reshape(
            -1, 3, 32, 32).transpose(0, 2, 3, 1)
        test_labels = np.array(data_dict[b'labels'])

    return (train_images, train_labels), (test_images, test_labels)


def build_model():
    # Define the model architecture
    model = models.Sequential()
    model.add(layers.Conv2D(
        32, (3, 3), activation='relu', input_shape=(32, 32, 3)))
    model.add(layers.MaxPooling2D((2, 2)))
    model.add(layers.Conv2D(64, (3, 3), activation='relu'))
    model.add(layers.MaxPooling2D((2, 2)))
    model.add(layers.Conv2D(64, (3, 3), activation='relu'))

    # Add dense layers on top
    model.add(layers.Flatten())
    model.add(layers.Dense(64, activation='relu'))
    model.add(layers.Dense(10))

    return model

def train_model(model, train_images, train_labels, test_images, test_labels):
    # Compile and train the model
    model.compile(optimizer='adam',
                  loss=tf.keras.losses.SparseCategoricalCrossentropy(
                      from_logits=True),
                  metrics=['accuracy'])

    history = model.fit(train_images, train_labels, epochs=50,
                        validation_data=(test_images, test_labels))

    # Check if the model directory exists
    if os.path.exists('model'):
        # If it does, delete it
        shutil.rmtree('model')
    # Recreate the model directory
    os.makedirs('model')

    # Save the model
    model.save('model/cifar-10-batches-py-model.keras')

    return history

# Load and preprocess the CIFAR10 dataset
data_dir = 'C:/Users/marin/Documents/GitHub/AB/ml-photo-app/neural-network-builder/venv/cifar-10-batches-py'
(train_images, train_labels), (test_images,
                               test_labels) = load_cifar10_data(data_dir)


# Normalize pixel values to be between 0 and 1
train_images, test_images = train_images / 255.0, test_images / 255.0

# Build and train the model
model = build_model()
history = train_model(model, train_images, train_labels,
                      test_images, test_labels)


# Print the history dictionary
print(history.history)

Run the generate-model.py script

Moment of truth! Run your file.
This file will build, train, and save our model.

Depending on your computer, this process can take more or less time.

Let’s focus on the parameter called epochs in our script, which is set to epochs=50.

An epoch is an important hyperparameter, representing one complete cycle through all training datasets.
Each sample will update the model’s parameters.
This cycle is not about the time it takes but the number of times it runs through the data.
This key parameter influences the training process. Indeed, having more epochs affects the model’s learning rate.

The higher the number, the longer it will take to train the model.

Underfitting could happen if there are insufficient epochs for the model to identify the underlying patterns in the data. However, an excessive number of epochs might cause the model to overfit the training set, resulting in subpar generalization on fresh, untried data.

To run the script, use this command here (you might need to use python or python3):

python3 generate-model.py

Now, a little patience! Wait for your model to be trained and saved in the model folder.
When creating our GUI front end, we will copy that model folder.
Now, let’s add a GUI to play with our model!

Build our GUI using Taipy

Setup

Navigate back to the main project folder ml-photo-app and then run these scripts to set up our interface:

mkdir frontend
cd frontend
virtualenv venv
source venv/bin/activate
cd venv

Now, let’s install the Python libraries we will be using:

Taipy
TensorFlow
Numpy
Pillow ( PIL) - Python imaging library

Run this command:

pip install taipy tensorflow pillow numpy

Frontend folder

Let’s get our fresh new model and copy it from our neural-network-builder project into our frontend folder.
Create two files in our root folder inside of frontend:

index.py
index.css.

These are the files we will run for our GUI.

Set your CSS preferences: index.css

Add this code to the index.css file we just made:

@import url('https://fonts.googleapis.com/css2?family=Alegreya+Sans:ital,wght@0,100;0,300;0,400;0,500;0,700;0,800;0,900;1,100;1,300;1,400;1,500;1,700;1,800;1,900&display=swap');

body {
  background: rgb(36, 57, 86);

  font-family: 'Alegreya Sans', sans-serif;

  font-weight: 400;

  font-style: normal;

  font-size: 18px;
}

.container {
  margin: 0 auto;
}

.attachment,
.prediction {
  display: flex;

  flex-flow: row nowrap;

  align-items: center;
}

.attachment div {
  margin: 1rem;
}

.prediction p {
  margin-right: 1rem;
}

Create the index.py Python script

To create an application with Taipy, you can use Markdown, the Python API, or HTML.
In this tutorial, we will use the HTML method, but feel free to use whichever option!

And lastly, add this code to the index.py script:

from taipy.gui import Gui
from taipy.gui import Html
from tensorflow.keras import models
from PIL import Image
import tensorflow
import numpy as np


class_names = {
    0: 'airplane',
    1: 'automobile',
    2: 'bird',
    3: 'cat',
    4: 'deer',
    5: 'dog',
    6: 'frog',
    7: 'horse',
    8: 'ship',
    9: 'truck',
}


model = models.load_model("C:/Users/marin/Documents/GitHub/AB/ml-photo-app/model/cifar-10-batches-py-model.keras")


def predict_image(model, path_to_img):
    img = Image.open(path_to_img)
    img = img.convert("RGB")
    img = img.resize((32, 32))
    data = np.asarray(img)
    data = data / 255
    logits = model.predict(np.array([data])[:1])
    probs = tensorflow.nn.softmax(logits).numpy()

    top_prob = probs.max()
    top_pred = class_names[np.argmax(probs)]

    return top_prob, top_pred


opt = tensorflow.keras.optimizers.legacy.Adam(learning_rate=0.1)


content = ""
img_path = "https://placehold.co/600?text=No+Image+Available&font=roboto"
prob = 0
pred = ""


html_page = Html("""
<div class="container">
<h1>Machine Learning Photo App</h1>
<p>There is a prediction indication that ranges from 0 to 100. The greater the value, the more certain the machine model is that the prediction is true. This all depends on the Neural Network Builder model that we generated. The longer you train the model, the smarter it will get. If it does not have enough training time, it will make incorrect predictions.</p>
<div class="attachment">
<div><taipy:file_selector extensions=".png">{content}</taipy:file_selector></div>
<div><p>Choose an image from your computer to upload</p></div>
</div>
<div>
<taipy:image>{img_path}</taipy:image>
<taipy:indicator min="0" max="100" width="25vw" height="25vh" orientation="vertical" value="{prob}">{prob}</taipy:indicator>
</div>
<div class="prediction">
<p>Prediction:</p><div><taipy:text>{pred}</taipy:text></div>
</div>
</div>
""")

def on_change(state, var_name, var_val):
    if var_name == "content":
        top_prob, top_pred = predict_image(model, var_val)
        state.prob = round(top_prob * 100)
        state.pred = "Its a " + top_pred
        state.img_path = var_val


app = Gui(page=html_page)

if __name__ == "__main__":
    app.run(use_reloader=True, port=8000)

Our application will run on the 8000 port here, but feel free to change it.

Run the application

app.run(use_reloader=True, port=8000)

With the use_realoder set to *True”, if you make any changes to your GUI code, you don’t have to rerun everything; just refresh your application page.

To run our app, use this command here:

taipy run index.py

How to use the application?

Upload any .png, preferably with an image part of these ten categories!

airplane
automobile
bird
cat
deer
dog
frog
horse
ship

Have fun seeing how your application classifies images!

Final Thoughts

Our project is completed!

We used Python to create an image classifier model that we could use directly through a GUI with Taipy. Don't hesitate to check out Tensflow's and Taipy's documentation if you want to make a more comprehensive application.

Feedback welcomed!

Python libraries for your DataScience CV in 2024

Marine — Tue, 13 Feb 2024 12:07:03 +0000

TL;DR

In 2024, Python is still the primary language for data science thanks to its simplicity but also with the various libraries for data cleaning, feature engineering, visualization, and machine learning.
If you want to start or pivot your career to be more data science-oriented, this list will give you the libraries you need to know.

1- Taipy

Field: Full application

Key features:

Towards Data science: Notebook compatible & easy integration with Machine learning platforms (Dataiku, Databricks, etc.…)
Taipy scales as more users on the application
Taipy works with large datasets
Asynchronous mode: ideal for handling high-load applications

Star ⭐ the Taipy repository

Your support means a lot🌱, and really helps us in so many ways, like writing articles! 🙏

2- Matplotlib

Field: Data Visualization

Matplotlib is the most famous visualization widget library.
With this library, you can plot any 2D graph easily with its extensive range of charts and customization capabilities.
A great library to check your model’s performance with simple and quick charts.

Star ⭐ the repository

3- Pandas

Field: Data Manipulation and Analysis

How to code in Python without knowing Pandas? Pandas are Python royalty!
The two data structures of this library are:

dataframes
series This library allows data loading, cleaning, and preparation quickly and efficiently.

Key functions include:

Loading data
Reshaping data frames
Basic statistics

Star ⭐ the repository

4- Numpy

Field: Numerical Computing

Numpy is less generalist than Pandas, but this is an essential tool for scientific computing and data preprocessing.
When using Numpy, you will become familiar with arrays and know how to efficiently make data manipulations and mathematical functions.
This library is definitely essential to your data science projects.

Star ⭐ the repository

5- Scikit-Learn

Field: Machine Learning

Another Python library, and this time, your top choice for machine learning in Python.
This library has various algorithms:

K-means clustering
Regression
Classification

But it also sets up your machine learning project through data splitting and dimension reduction techniques, for example.

Star ⭐ the repository

6- Seaborn

Field: Statistical Data Visualization

Seaborn will bring some added features to Matplotlib.
This library brings in complex and attractive visualizations when Matplotlib emphasizes preciseness and simplicity.

Star ⭐ the repository

7- TensorFlow or Pytorch

Field: Deep Learning

Pytorch or TensorFlow that is the question.
These two libraries offer an interface for neural networks.
They are flexible and give you efficient APIs to build and create neural network models.

The choice is up to you, but here are some differences:

PyTorch has a more Natural Language Processing angle
Pytorch has a more pythonic feel

Star ⭐ the TensorFlow repository

Star ⭐ the PyTorch repository

8- Keras

Field: Deep Learning

Keras is a great way to start with Deep Learning as it runs on top of TensorFlow but with a simplified implementation process.

Star ⭐ the repository

9- Statsmodel

Field: Statistical Modeling

This library has an array of statistical models.
It is an excellent tool for the Exploratory Data Analysis phase of your Machine Learning project.

The array of capabilities ranges from descriptive analysis to statistical tests; it is also a suitable library for handling time series data, univariate and multivariate statistics, etc.

Star ⭐ the repository

10- Polars

Field: Fast Data Manipulation

Polars is a DataFrame library created to handle and process large datasets.
It was inspired by Python’s top library- Pandas, but with a (fast) twist, it’s 10 to 100 times faster. A must-know tool when handling large datasets.

Star ⭐ the repository

Conclusion

These ten libraries are essential for any ML project, and mastering them will enhance your Datascience CV.

Don't hesitate to comment your favorite ML/AI libraries!

Getting Started in your Contributing Journey

Marine — Mon, 05 Feb 2024 18:03:39 +0000

TL;DR

In this article, you will find the key steps to starting your open-source contributing journey and some essential resources on where to start.

Some Open-Source Repos to Get Started

Before getting started, here are some projects that have some first good issues.
These are great tags to look up for in projects. They offer opportunities for beginners to tackle their first set of manageable issues.

1- Get to know the Open-Source Culture

First things first, get deep into understanding the open-source culture. Get to know the practice of sharing and collaborating on resources in a way that allows others to use and modify them.
Resources: Check out this website. They have insightful articles about how to get started, best practices, and valuable hands-on advice.

2- Selecting your First Project

For a first contribution, ensuring your projects align with your interests is essential.
Look up libraries in the language you like or in thematics that might bring added value to your portfolio.
Don’t forget to coordinate your first contribution with your skillset; I recommend looking for “First Good Issues” tags.
Ensure the project you choose has a good and active community to guarantee they are responsive in helping you.
Resources: Check out this website that curates a list of the first good issues available making your search easier.

3- Mastering GitHub

Contributing to your first project goes hand in hand with mastering GitHub.
Resources: I recommend using this GitHub app, GitHub Learning Lab, to get started with GitHub and make sure you master it before starting contributing.
Also, check out this repository that gives a hands-on guide on how to make your first contribution.

4- Understanding the licenses

When contributing, you will always see a license on the repository.
Make sure you get familiar with the different open-source licenses to understand how your contribution is protected and shared.
Resources: Thanks to this website, licensing will hold no mysteries.

5- The "CONTRIBUTING.md" file

A vital and recurring part of the contribution process is examining and understanding the CONTRIBUTING.md file or contribution guidelines of the repository before making your PR.
Example: Check out this example of contributing guidelines.

6- Monetize your Work

Once you have contributed a couple of times and feel it’s something you enjoy and are comfortable with, you can look into projects that help you monetize your contributions.
Resources: Check out Quine; this website showcases open-source projects and their issues, a great way to combine business with pleasure.

Contributing is not just about coding.
You can contribute through documentation, design, and community support, so don’t feel intimidated and start contributing and make the community stronger!
If you have some projects to share that are good for newbie contributors, don't hesitate to share them!

Specialized Python libraries for Unique Tasks

Marine — Wed, 31 Jan 2024 15:06:59 +0000

TL;DR

Python is the language of Data, ML, and AI, but Python also has various libraries for other specific needs. These lesser-known libraries serve interesting and particular needs.

Taipy

What it does: Taipy is an open-source Python library for building production-ready applications, front-end & back-end, in no time.

Why it’s unique: With Taipy, Python programmers can bring any ML/AI application to an automated and production-ready project. There is no need to know other languages; with Taipy, do front-end and back-end in Python.

Star ⭐ the Taipy repository

Your support means a lot🌱, and really helps us in so many ways, like writing articles! 🙏

PyPDF2

What it does: This Python library focuses on PDF files. With it, you can do any action, from merging, cropping, and transforming your PDF files.

Why it’s unique: It makes dealing with PDFs easier (no need for sketchy websites) and directly accessible to Pythonistas.

Arrow

What it does: Arrow makes dealing with dates, times, and timestamps easy with a beginner-friendly interface. You can create, manipulate, format, and convert dates and times.

Why it’s unique: Arrow fills the Python datetime module's gap with added functionalities with the timezone conversions, for example.

TQDM

What it does: TQDM adds progress bars for your iterative processes.

Why it’s unique: This accessible library enhances user experience with appealing and informative progress bars.

Geopy

What it does: Geopy focuses on coordinates. With it, you can locate countries, cities, or addresses through coordinates using third-party geocoders.

Why it’s unique: It groups APIs of multiple geolocation services, making it a unified framework for dealing with geographical data.

PyAutoGUI

What it does: This Python library will replace your hands on your mouse and keyboard to automate processes with other applications.

Why it’s unique: A real productivity win, this library can automate all your repetitive tasks on your computer.

Colorama

What it does: Add some color to your terminal with Colorama. A real game-changer for your eyes!

Why it’s unique: Colorama will increase the readability and user experience of all your command line and script outputs and of course beautify them.

Whoosh

What it does: It is your Python search engine library. With Whoosh, you can index all your texts to facilitate searching.

Why it’s unique: This library is a great productivity feature for any of your applications; it adds a search functionality!

MoviePy

What it does: This Python library simplifies movie editing using Python. With MoviePy, you can cut, concatenate, and add titles (and more!) to your videos.

Why it’s unique: This library brings the video editing process to Python.

Pyjokes.es

What it does: This library generates one-liner jokes on command!

Why it’s unique: This elementary and straight-to-the-point library is great to try for a little added humor in the day.

Conclusion:

All these libraries show Python’s versatility and how the language is growing to cater to specific needs.

Feel free to reach out if you have any questions or feedback!

Web frameworks📊: Your 🐍Python Picks

Marine — Mon, 22 Jan 2024 13:28:10 +0000

TL;DR

Python’s web framework landscape is diverse and keeps on expanding.
Let’s focus on 10 libraries and understand their strengths and weaknesses.

1- Taipy

🌟 Advantages:

User-friendliness: Taipy is a Python library that creates production-grade applications with minimal coding
Full-stack capabilities: Taipy has both front-end and back-end functionalities, simplifying complex interface development
Geared towards Data science: Notebook compatible & easy integration with Machine learning platforms (Dataiku, Databricks, etc.…)
Asynchronous mode: ideal for handling high-load applications

⚠️ Disadvantage:

Emerging Community: still growing, with fewer resources and a smaller community compared to established libraries

Star ⭐ the Taipy repository

Your support means a lot🌱, and really helps us in so many ways, like writing articles! 🙏

2- Dash

🌟Advantages:

Integration with Plotly: creates interactive and data-driven dashboards with a plethora of graph options and customization
Python-centric workflow

⚠️ Disadvantage:

Learning curve: understanding the callback and layout concepts can be challenging for beginners

3- Django

🌟 Advantages:

Scalable and versatile: good for large-scale projects thanks to its robustness and flexibility
Rich ecosystem: various integrations with third-party apps and plugins
Strong Community: active and prosperous user base

⚠️ Disadvantages:

Suitability for more minor projects: complex for more minor use cases
Learning Curve: completeness of the library makes the time to master high

4- Fast API

🌟 Advantages:

High Performance: Fast API focuses on speed and efficiency
Asynchronous mode: suitable for high-load applications

⚠️ Disadvantages:

Less established: fewer resources relative to more stable competitors (i.e., Flask or Django)
Complexity: some concepts (asynchronous mode) can be challenging for beginners to apprehend

5- Streamlit

🌟 Advantages:

User-friendly interface: simplifies the design and creation of web applications
Fast prototyping: suitable for quick development

⚠️ Disadvantages:

Limited use cases: not suitable for complex, production-ready applications
Customization limitations: offers less customization due to low-code structure

6- Flask

🌟 Advantages:

Use Case flexibility: suitable for designing small to medium applications
Rich extensions: a wide variety of plug-ins
Beginner-friendly: easy learning curve for moderate coders

⚠️ Disadvantage:

Scalability limits: not the most scalable option for large and complex applications

7- Bottle

🌟 Advantages:

Minimalist Approach: Bottle is a lightweight framework, suitable for beginners
Fast development: simple and quick to create prototypes
Lightweight: suitable for straightforward web application use cases

⚠️ Disadvantages:

Limited features: not suitable for complex applications
Smaller community: the community is smaller, with fewer resources

8- Pyramid

🌟 Advantages:

High Customization: customization through templates and database integration
Rich Extension Ecosystem

⚠️ Disadvantages:

Steep learning curve
Smaller user base: a smaller library means a smaller community and fewer resources

9- Falcon

🌟 Advantages:

Performance-oriented
Control and customization: minimalist design makes flexible user control

⚠️ Disadvantages:

Not beginner friendly: can be challenging for new Pythonistas.
Less suitable for full-fledged applications

10- CherryPy

🌟 Advantages:

Simplicity: good option for smaller objects through simple API
Extensive plug-ins: various extensions and plug-ins

⚠️ Disadvantages:

Lower popularity: less popular
Smaller community support

Conclusion:

Choosing a framework to build your web applications depends on various requirements like the complexity of the projects or the developers' coding level.
By understanding the strengths and weaknesses of these 10 libraries, developers can facilitate their decision-making on which framework to go for.

Feel free to reach out if you have any questions or feedback!

🐍 Python Playground: 16 ways 📚 to get started

Marine — Mon, 15 Jan 2024 15:29:04 +0000

TL;DR

This article is a guide designed to assist those new to Python programming with easy-to-use and engaging resources. From beginner-friendly libraries to interactive coding platforms, you should find your perfect fit!

LIBRARIES

The following Python libraries are easy to use and great for getting started with Python.

1- Taipy

A new library in the Python landscape, Taipy is a great low-code Python web app for creating powerful web applications with minimal code. Get started with web applications with Taipy.
🔑 Features:

Extensive interactivity
There are more customization capabilities for your layout, styling, etc. (no CSS needed)
Multipage & multi-user applications

Star ⭐ the Taipy repository

Your support means a lot🌱, and really helps us in so many ways, like writing articles! 🙏

2- Turtle Graphics

The place to start for absolute beginners. This pre-installed Python library teaches you the fundamentals of Python programming in a fun and visual way.

3- EasyGUI

Get started with graphical user interfaces (GUIs) with this library.
Its simplicity makes it a top choice for beginners to create your basic GUI.

4- Matplotlib

This widget library is a go-to in the Python landscape for a good reason. With the extensive range of chart types, you can plot any 2D graph.
This library allows for significant customization through fine-grain of the graphical elements.

5- Pandas

Even if all Pythonistas use Pandas, this library is straightforward to comprehend and allows you to do many things.
You can learn about dataframes and series and how to handle data efficiently.

🔑 features:

Loading data from various sources
Reshaping dataframes
Basic data analytics through basic statistics

6- Thonny

Thonny is a great IDE for beginners.
It has excellent features to help understand the programming process and straightforward writing and debugging of Python code. You can try out the previous libraries in this effortless environment.

CODING PLATFORMS

Coding platforms offer an academic-like setting, ideal for quick and structured learning.

7- Datacamp

This was the platform I used to get started with Python, so how could I not include it? Datacamp is an interactive learning platform focused on programming around data science and analytics. They have courses of different levels, making it a great place to start learning about Python and data analytics concepts.

8- Code Combat

Code Combat makes learning fun as you learn Python while playing a game.
This platform has a world where you advance your character in the game while learning how to code.
This website caters to the younger generation, but I recommend trying it out for beginners of any age!

9- Sololearn

This platform revolves and evolves through their community. Sololearn focuses on short Python lessons, making a great tool for learning Python on the go.

TUTORIAL & CHALLENGES

Tutorials and challenges are an excellent way to enhance and test your Python skills with small and manageable projects.

10- Codechef

This website features programming competitions.
For those who learn better with competition, Codechef is for you. They offer various challenges to cater to all programming levels.

11- Codewars

This community website also offers coding challenges.
It's a great way to learn and challenge yourself with Python.

12- Machine Learning Mastery

Machine Learning Mastery offers tutorials and books around Machine Learning and Python.
These tutorials were beneficial for some of my Machine Learning projects, and I still go back to them from time to time!

FULL PROJECTS

Building an entire project in Python is a great way to put your skills to the test in real-life scenarios. In general, these projects are a great addition to your portfolio.

13- Kaggle

This platform was my go-to for testing my Python and machine learning skills. This platform showcases data science projects.
Kaggle is also a great resource to add to your CV.

14- freeCodeCamp

Even though freeCodeCamp focuses on web development, this platform also uses Python projects. A great way to learn by doing.

HACKATHONS

Hackathons are a great way to learn in a dynamic and collaborative environment. Participation in hackathons is academically and professionally valued.

15- Major League Hacking

MLH organizes hackathons for students from all around the world. Their hackathons can be on-premise but also online.
Hackathons are great for applying your Python skills to real-world projects.
Also, it is a great way to have projects to showcase in interviews or your CVs.

16- Devpost

Devpost is a website for online hackathons. They provide events < with a lot of choices regarding subjects and prizes!

Conclusion

These resources offer various ways to kickstart your Python programming journey through these easy-to-use libraries, interactive learning platforms, and practical challenges.

Feel free to reach out if you have any questions or feedback!

✨2024 Resolution: Be more Open-Source centric

Marine — Mon, 08 Jan 2024 13:14:27 +0000

TL;DR

Why rely on proprietary software and services when there is (almost) always an open-source alternative that can do the job just as well, if not better?
Here are 10 open-source alternatives I’ve been using, covering everything from project management and communication to data analytics.

1- Taipy instead of Tableau

Tableau might be one of the top players in data visualization, but Taipy offers a robust alternative.
Taipy is an open-source, low-code Python web app that allows you to create comprehensive web applications that showcase your data visualizations.
Taipy is low code, super customizable, and offers more flexibility when building your dashboards.

Star ⭐ the Taipy repository

Your support means a lot🌱, and really helps us in so many ways, like writing articles! 🙏

2- Cal.com instead of Calendly

Calendly was a game-changer for simplifying scheduling, but Cal.com managed to bring it to the next level. This open-source gem has features like:

Team scheduling
Integrated video conferencing
Automatic time zone detection

3- Plausible instead of Google Analytics

Sure, Google Analytics is a big name, but sometimes smaller tools offer just as much, and a great example is Plausible.
This open-source tool provides website analytics features just like Google, and no, they don’t compromise on data privacy.

4- AppFlowy instead of Notion

Notion is an excellent workspace for anything about note-taking and project management but if you want an even more straightforward option, try AppFlowy.
This tool offers a minimalist alternative, focusing on simply creating and organizing lists, notes, and tasks.
The interface is very user-friendly; you’ll be a pro in no time.

5- Penpot instead of Figma

Figma is a design powerhouse, but its open-source cousin, Penpot, has been gaining momentum over the last year.

Here are Penpot’s key features:

Collaborative design capabilities
Vector editing
Interactive prototypes
Cost-effectiveness

6- Fonoster instead of Twilio

Twilio is a communication platform that provides APIs for SMS, voice, video, and authentication and offers a seamless customer experience.
Let me introduce you to Fonoster, the cost-effective alternative. Fonoster provides a similar offer with voice and messaging services. Fonoster focuses on scalability while giving you a seamless customer experience.

7- NextCloud instead of Dropbox

NextCloud is the open-source rival to Dropbox.
It offers file hosting, collaboration, and synchronization features, all while keeping your data private and under your control.

8- Jitsi instead of Google Meets

Jitsi is the alternative to Google Meets, offering similar video conferencing capabilities.

Their key features:

End-to-end encryption
Screen sharing
And no registration is required!

9- Padloc vs 1Password

1Password is well-established in the password management landscape, but Padloc, an open-source tool, focuses on privacy and security just as much.
You can securely store and manage your sensitive and private information securely with Padloc, just like 1Password.

10- Crowd.dev instead of Common Room

Common Room has been gaining momentum in the community building landscape, but don’t overlook their open-source alternative, "crowd.dev".
Whether it’s project management, funding, or collaboration, "crowd. dev" can’t be overlooked to build and develop online communities.

Conclusion

When choosing a tool, remember to check out the open-source option.
Open source brings transparency, customizability, and cost-effectiveness to the equation, making a good choice in most cases.

Congratulations, you made it to the end! Don't hesitate if you have any questions.