Forem: MokhtarEbrahim

Advanced Text Matching and Fuzzy Comparison for Data Professionals

MokhtarEbrahim — Sat, 22 Mar 2025 21:35:51 +0000

In the world of data analysis, the biggest challenges often come from the messiest data. Even with the abundance of data tools available today, one persistent problem remains frustratingly difficult: comparing text data with inconsistencies, typos, and variations.

That's why I built a Text Compare tool - a specialized tool that combines multiple sophisticated text matching approaches in one powerful package.

The Problem TextCompare Solves

Have you ever faced any of these challenges?

Customer data with inconsistent name spellings across different systems
Product catalogs with slight variations in item descriptions
Financial records with similarly named entities that should be matched
Lists that need to be reconciled despite formatting differences

These problems are notoriously difficult to solve with conventional tools. Excel formulas fall short, database queries require exact matches, and many solutions only handle one type of comparison.

TextCompare's Unique Approach

What makes TextCompare different is its comprehensive, multi-layered approach to finding relationships between text strings:

1. Multiple Matching Algorithms Working Together

Unlike tools that rely on a single matching method, TextCompare employs several simultaneously:

Exact Matching: Identifies perfect matches
Approximate Matching: Uses Levenshtein distance to find text with minor differences
Phonetic Matching: Connects words that sound similar but are spelled differently
Numeric Tolerance Matching: Identifies numbers that are within specified ranges

This multi-algorithm approach means you're not limited to a single definition of "similarity."

2. Highly Configurable Options

Every data set has unique characteristics. TextCompare lets you fine-tune the process:

Case sensitivity: Toggle whether capitalization matters
Whitespace handling: Choose how to treat spaces, tabs, etc.
Symbol filtering: Ignore punctuation that shouldn't affect matching
Substitution mapping: Define character replacements (e.g., "ö" → "o")
Exclusion lists: Specify words to ignore
Levenshtein threshold: Control how strict or lenient fuzzy matching should be

3. Comprehensive Analysis, Not Just Matching

TextCompare doesn't just tell you what matches - it provides deep analysis of your data:

Duplicate detection: Identifies repeated entries within each dataset
Frequency analysis: Shows the distribution and occurrence patterns
Match explanations: Provides details on why records matched
Statistics: Delivers performance metrics and processing information

4. Built for Scale

Unlike desktop tools that struggle with large data sets, TextCompare was built for scale:

Efficient processing: Optimized algorithms handle large files without bogging down
Asynchronous operation: Process in the background while you continue working
Progress tracking: Real-time updates on comparison status
Resource optimization: Automatically utilizes available system resources

Real-World Examples

Customer Data Reconciliation

A financial institution had customer records spread across 12 systems with no consistent ID. TextCompare identified matching records with 97% accuracy despite name variations, allowing them to build a unified customer view without manual matching.

Product Catalog Consolidation

An e-commerce company merged with a competitor and needed to combine product listings. TextCompare matched similar product descriptions despite different formatting conventions, reducing weeks of manual work to a few hours.

Compliance Verification

A global organization needed to check employee names against sanction lists where transliteration caused variations in spelling. TextCompare's phonetic matching identified potential matches that exact comparison would have missed.

Technical Implementation

TextCompare processes files by:

Preprocessing data with your specified options
Building optimized data structures for rapid comparison
Running parallel matching processes for different algorithms
Consolidating results into a comprehensive report

Processing is as simple as:

Upload or connect your data sources
Configure your comparison options
Start the comparison process
Review the detailed matching report

Why TextCompare Stands Out

Unlike other solutions that:

Only handle one type of matching
Require exact formatting
Struggle with large datasets
Provide limited configuration
Offer minimal analysis

TextCompare delivers a complete text comparison toolkit in one package, with the power and flexibility to handle real-world data challenges at scale.

Would like to hear all your thoughts!

The tool: https://likegeeks.com/compare-two-lists/

Python pytest tutorial

MokhtarEbrahim — Sat, 06 Aug 2022 19:44:52 +0000

What is Python pytest?
Install pytest for Python
How to use pytest?
Run pytest with specific Python version
Pytest fixtures
Pytest Conftest
Pass pytest arguments when invoked via Command line
Skip a method using pytest marker
Pytest failures
Generate HTML report
Common pytest Plugins
Pytest vs. Unittest
Conclusion

https://likegeeks.com/python-pytest/

Sort NumPy arrays in Python

MokhtarEbrahim — Sun, 02 Jan 2022 14:04:41 +0000

NumPy Sort Algorithms
Sort in Ascending Order
Sort in-place
Sort by making a copy of the array
Sort in Descending Order
Sort in-place
Sort by making a copy of the array
Sort 2D Array
NumPy Axes
Sort 3D Array
Sort by Column
Sort Using Order
Sort by Multiple Columns
Sort by Row
Sort by Column till Specified Row or from Specific Row
Sort by Datetime
Sort with Lambda
Sort with NaN Values
Sort NumPy Array Containing Floats
Conclusion

https://likegeeks.com/sort-numpy-array/

NumPy Meshgrid Tutorial

MokhtarEbrahim — Tue, 09 Mar 2021 08:41:02 +0000

What is a meshgrid?
What are the benefits of meshgrid?
Creating a 2D NumPy meshgrid from two 1D arrays
Creating a NumPy meshgrid with sparse=True
Creating a NumPy meshgrid of polar coordinates
NumPy meshgrid with ‘matrix indexing’
Flip a NumPy meshgrid
Creating Meshgrid with NumPy matrices
Creating a 3-dimensional meshgrid
Creating a 3D surface plot using NumPy meshgrid
Avoiding NumPy meshgrid memory error
Conclusion

https://likegeeks.com/numpy-meshgrid/

Normalization using NumPy norm (Simple Examples)

MokhtarEbrahim — Fri, 19 Feb 2021 13:44:53 +0000

Introduction
NumPy norm
norm of an array
Norms of columns and rows of a matrix
Norm of an n-dimensional array
Why do we need norms?
L1 norm of a vector
NumPy norm of arrays with nan values
Euclidean distance using NumPy norm
Performance comparison: NumPy norm vs sqrt
Conclusion

https://likegeeks.com/numpy-norm/

NumPy random seed (Generate Predictable random Numbers)

MokhtarEbrahim — Fri, 12 Feb 2021 11:14:15 +0000

What is NumPy Random Seed?
What is the Pseudo-Random Number?
Random Seed Importance
NumPy.random has no Seed Number
NumPy.random.seed(0)
NumPy.random.seed(101)
random seed scope
Seed to the Time
Random Seed Multiprocessing
Seed the same across computers
Random seed after 1000 time
Random seed 2d array
How to change random seed?
NumPy random seed shuffle
NumPy random seed vs Python random seed
Conclusion

https://likegeeks.com/numpy-random-seed/

Dijkstra's algorithm in Python (Find Shortest & Longest Path)

MokhtarEbrahim — Wed, 10 Feb 2021 16:22:39 +0000

Pathfinding Problem
Adjacency List Representation
Adjacency Matrix Representation
Computation Time and Memory Comparisons
Difficulties of Pathfinding
Dijkstra’s Shortest Path: Python Setup
Dijkstra’s Shortest Path: Step by Step
Putting it all Together
Longest Path and Maze Solving

https://likegeeks.com/python-dijkstras-algorithm/

Depth First Search algorithm in Python (Multiple Examples)

MokhtarEbrahim — Wed, 01 Jul 2020 14:26:17 +0000

1 Introduction
2 The Depth First Search Algorithm
3 Representing a graph
3.1 Adjacency Matrix
3.2 Adjacency List
4 Implementing Depth First Search(a non-recursive approach)
5 DFS using a recursive method
6 Depth First Search on a Binary Tree
6.1 What is a Binary Tree?
6.2 Representing Binary Trees using Python classes
6.3 Implementing DFS for a binary tree
7 Depth First Search using networkx
7.1 Constructing a graph in networkx
7.2 Visualizing the graph in DFS
7.3 Graph traversal in networkx – DFS
8 Topological sorting using Depth First Search
9 Finding connected components using DFS
10 Conclusion

https://likegeeks.com/depth-first-search-in-python/

Python Correlation Matrix Tutorial

MokhtarEbrahim — Wed, 17 Jun 2020 14:26:53 +0000

1 What is the correlation matrix?
1.1 What is the correlation coefficient?
2 Finding the correlation matrix of the given data
3 Plotting the correlation matrix
4 Interpreting the correlation matrix
5 Adding title and labels to the plot
6 Sorting the correlation matrix
7 Selecting negative correlation pairs
8 Selecting strong correlation pairs (magnitude greater than 0.5)
9 Converting a covariance matrix into the correlation matrix
10 Exporting the correlation matrix to an image
11 Conclusion

https://likegeeks.com/python-correlation-matrix/

Hope you like it.

Best Regards,

NumPy Where Tutorial (With Examples)

MokhtarEbrahim — Wed, 20 May 2020 14:23:11 +0000

1 A very simple usage of NumPy where
2 How does NumPy where work?
3 2D matrices
4 Multidimensional array
5 Using the result as an index
6 Parameters ‘x’ and ‘y’
7 Apply on Pandas DataFrames
8 Multiple conditions
9 Nested where (where within where)
10 Finding rows of zeros
11 Finding the last occurrence of a true condition
12 Using on DateTime data
13 Conclusion

https://likegeeks.com/numpy-where-tutorial/

Exiting/Terminating Python Scripts (Simple Examples)

MokhtarEbrahim — Wed, 06 May 2020 14:38:32 +0000

1 Why does Python automatically exit a script when it’s done?
2 Detect script exit
3 Graceful exit
4 Exit with error messages
5 Exiting without error
6 Exit and release your resources
7 Exit after a time
8 Exiting using a return statement
9 Exit in the middle of a function
10 Exit when conditions are met
11 Exit on keypress
12 Exit a multithreaded program
13 End without sys exit
14 Exit upon exception
15 Exit and restart

https://likegeeks.com/terminating-python-scripts/

Hope you like it.

20+ Examples For NumPy Matrix Multiplication

MokhtarEbrahim — Tue, 05 May 2020 14:34:14 +0000

1 Basic Terminologies
2 Matrix multiplication with a vector
3 Multiplication with another matrix
4 Multiplication between 3 matrices
5 NumPy 3D matrix multiplication
6 Alternatives to np.matmul()
6.1 The ‘np.dot()’ method
6.2 The ‘@’ operator
7 Multiplication with a scalar (Single value)
8 Element-wise matrix multiplication
9 Matrix raised to a power (Matrix exponentiation)
9.1 Element-wise exponentiation
9.2 Multiplication from a particular index
10 Matrix multiplication using GPU
10.1 Using CPU
10.2 Using GPU
11 Conclusion

https://likegeeks.com/numpy-matrix-multiplication/

Hope you find it useful.

Regards,