Forem: Frankkk

Python Tips- Maximum Value float('inf')

Frankkk — Fri, 02 Feb 2024 04:39:49 +0000

In Python, there are two types: 'int' and 'float'. It's important to note that in Python 3, integers (int) have no maximum limit.
Here is a question: what is the maximum value in Python?

For the float, we can use sys.float_info to check the details.

>>> import sys
>>> sys.float_info
sys.float_info(max=1.7976931348623157e+308, max_exp=1024, max_10_exp=308, min=2.2250738585072014e-308, min_exp=-1021, min_10_exp=-307, dig=15, mant_dig=53, epsilon=2.220446049250313e-16, radix=2, rounds=1)

>>> print(sys.float_info.max)
1.7976931348623157e+308

The float value range is

-1.7976931348623157e+308 <= f <= 1.7976931348623157e+308

Also, we can declare the max value in float

>>> a = float('inf')
>>> a
inf
>>> import math
>>> math.isinf(a)
True
>>> 
>>> b = math.inf
>>> math.isinf(b)
True

We can compare 'inf' and float_info.max

>>> a > sys.float_info.max
True

How about the max value of 'int'? It is a 64 bit value.

>>> sys.maxsize
9223372036854775807
>>> 
>>> 
>>> print(sys.maxsize == 2**63 - 1)
True

But sys.maxsize is not the max int value, you can declare a value

>>> b = 10**309
>>> type(b)
<class 'int'>
>>> b>sys.float_info.max
True
>>> b>sys.maxsize 
True
>>> b > float('inf')
False

So, if you want to know why the Integer (int) has no max limit in Python, you need read the source code, the int object is with variable size.

typedef struct _longobject PyLongObject;
struct _longobject {
  PyObject_VAR_HEAD
  digit ob_digit[1];
};
#define PyObject_VAR_HEAD      PyVarObject ob_base;
typedef struct {
    PyObject ob_base;
    Py_ssize_t ob_size; /* Number of items in variable part */
} PyVarObject;
typedef struct _object {
    _PyObject_HEAD_EXTRA
    Py_ssize_t ob_refcnt;
    struct _typeobject *ob_type;
} PyObject;

Image is from [5].

Please refer to [6] to read the source code in C~

In summary, we can use float('inf') as the max value in Python. For the float, we max value is sys.float_info.max. for the integer, we can use the value sys.maxsize. But Integer (int) has no max limit in Python 3, from the source code in C language, the int object is defined variable size, that is why it has no max limit. Actually, the maximum integer value depends on the memory size, you can try~

Reference

[1.Maximum and minimum float values in Python]
https://note.nkmk.me/en/python-sys-float-info-max-min/
[2.Integer (int) has no max limit in Python 3]
https://note.nkmk.me/en/python-int-max-value/
[3.numeric-types-int-float-complex]
https://docs.python.org/3/library/stdtypes.html#numeric-types-int-float-complex
[4.Int object source code ]
https://fasionchan.com/python-source/builting-object/int/
[5.Int object]
https://juejin.cn/post/7209612932367597623
[6.Python source code]
https://github.com/python/cpython/blob/v3.7.0/Objects/longobject.c#L3

Python Tips: range (Function or Class) in Details

Frankkk — Sun, 17 Dec 2023 23:50:17 +0000

In Python, when we want to generate sequences of numbers and write a for loop to check the numbers, we may use range(), for example:

>>> for i in range(5):
...     print(i)
... 
Output:
0
1
2
3
4

However, do you really know the function range()? What does range() function return? Do you know the class range?

Introduction

The range function is within Python Built-in Types.
The range type represents an immutable sequence of numbers and is commonly used for looping a specific number of times in for loops[1]. It includes three arguments:

range(start, stop[, step])
start
The first value you want to generate (or 0 if the parameter was not supplied)
stop
The last value value you want to generate, it not within the result
step
The value of the step parameter (or 1 if the parameter was not supplied), can be positive(Increase) or negative(Decrease)

Let’s show some examples below:

>>> for i in range(5):
...     print(i)
... 
0
1
2
3
4
>>> 
>>> for i in range(0,5):
...     print(i)
... 
0
1
2
3
4
>>> 
>>> for i in range(0, 5, 2):
...     print(i)
... 
0
2
4
>>> for i in range(5, 0, -1):
...     print(i)
... 
5
4
3
2
1
>>>

Let's get into more details about range.

What type of return value from range()?

We can use type to check the return value from range, it shows <class ‘range’>

>>> x = range(5)
>>> type(x)
<class 'range'>

In fact, class range in python is a class where derived from class xrange(), function range is more efficient than class range()because it is a built-in function and does not require creating a new object instance[6].

Using `help range` We can see the source code of range class

>>> help(range)
class range(object)
 |  range(stop) -> range object
 |  range(start, stop[, step]) -> range object
 |  
 |  Return an object that produces a sequence of integers from start (inclusive)
 |  to stop (exclusive) by step.  range(i, j) produces i, i+1, i+2, ..., j-1.
 |  start defaults to 0, and stop is omitted!  range(4) produces 0, 1, 2, 3.
 |  These are exactly the valid indices for a list of 4 elements.
 |  When step is given, it specifies the increment (or decrement).
 |  
 |  Methods defined here:
 |  
 |  __bool__(self, /)
 |      self != 0
 |  
 |  __contains__(self, key, /)
 |      Return key in self.
 |  
 |  __eq__(self, value, /)
 |      Return self==value.
 |  
 |  __ge__(self, value, /)
 |      Return self>=value.
 |  
 |  __getattribute__(self, name, /)
 |      Return getattr(self, name).
 |  
 |  __getitem__(self, key, /)
 |      Return self[key].
 |  
 |  __gt__(self, value, /)
 |      Return self>value.
 |  
 |  __hash__(self, /)
 |      Return hash(self).
 |  
 |  __iter__(self, /)
 |      Implement iter(self).
 |  
 |  __le__(self, value, /)
 |      Return self<=value.
 |  
 |  __len__(self, /)
 |      Return len(self).
 |  
 |  __lt__(self, value, /)
 |      Return self<value.
 |  
 |  __ne__(self, value, /)
 |      Return self!=value.
 |  
 |  __reduce__(...)
 |      Helper for pickle.
 |  
 |  __repr__(self, /)
 |      Return repr(self).
 |  
 |  __reversed__(...)
 |      Return a reverse iterator.
 |  
 |  count(...)
 |      rangeobject.count(value) -> integer -- return number of occurrences of value
 |  
 |  index(...)
 |      rangeobject.index(value) -> integer -- return index of value.
 |      Raise ValueError if the value is not present.
 |  
 |  ----------------------------------------------------------------------
 |  Static methods defined here:
 |  
 |  __new__(*args, **kwargs) from builtins.type
 |      Create and return a new object.  See help(type) for accurate signature.
 |  
 |  ----------------------------------------------------------------------
 |  Data descriptors defined here:
 |  
 |  start
 |  
 |  step
 |  
 |  stop
(END)

We can use `dir()` to check the attributes:

>>> dir(range)
['__bool__', '__class__', '__contains__', '__delattr__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__getitem__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__iter__', '__le__', '__len__', '__lt__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__reversed__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', 'count', 'index', 'start', 'step', 'stop']
>>>

Here, we can see it includes __iter__, which means is an instance of iterable[2].

Iterable is an object which can be looped over or iterated over with the help of a for loop. Objects like lists, tuples, sets, dictionaries, strings, etc. are called iterables[3].

Here, range is iterable, but it is not an Iterator, we can check it below:

# code
from collections.abc import Iterable, Iterator
x = range(5)

if isinstance(x, Iterable):
  print(f"{x} is iterable")
else:
  print(f"{x} is not iterable")

if isinstance(x, Iterator):
  print(f"{x} is Iterator")
else:
  print(f"{x} is not Iterator")

... 
... 
range(0, 5) is iterable
range(0, 5) is not Iterator
>>>

The definition of range in Python doc

Sequence Types

Rather than being a function, range is actually an immutable sequence type, as documented in Ranges and Sequence Types — list, tuple, range [4].

class range(stop)
class range(start, stop, step=1)
Rather than being a function, range is actually an immutable sequence type, as documented in Ranges and Sequence Types — list, tuple, range.

Arguments

range(start, stop[, step])
The arguments to the range constructor must be integers (either built-in int or any object that implements the index() special method) [5].

Operations

Common sequence operations(totally 12 items)

Common sequence operations can be found here:

https://docs.python.org/3/library/stdtypes.html

Range objects implement the collections.abc.Sequence ABC, and provide features such as containment tests, element index lookup, slicing and support for negative indices (see Sequence Types — list, tuple, range)

The collections.abc.Sequence ABC is provided to make it easier to correctly implement these operations on custom sequence types.

>>> r = range(0, 20, 2)
>>> r
range(0, 20, 2)
>>> 
>>> 11 in r
False
>>> 
>>> 10 in r
True
>>> 
>>> r.index(10)
5
>>> 
>>> r[5]
10
>>> 
>>> r[:5]
range(0, 10, 2)
>>> 
>>> r[-1]
18

Source code of range in C

We can check the range code in C from c code below:
https://github.com/python/cpython/blob/main/Objects/rangeobject.c

First, we can know the rangeobject.
We have the start, stop, step pointers, also we have length, which can be used to store the length of the range elements. PyObject_HEAD defines the initial segment of every PyObject.

typedef struct {
    PyObject_HEAD
    PyObject *start;
    PyObject *stop;
    PyObject *step;
    PyObject *length;
} rangeobject;

Workflow when create a range object

1) When reading the code, we can follow the 3 main functions:

range_new->range_from_array->make_range_object

Range_new

It mainly call the range_from_array function.

static PyObject *
range_new(PyTypeObject *type, PyObject *args, PyObject *kw)
{
    if (!_PyArg_NoKeywords("range", kw))
        return NULL;

    return range_from_array(type, _PyTuple_ITEMS(args), PyTuple_GET_SIZE(args));
}

range_from_array

Check the arguments(Py_ssize_t num_args), including start, stop, step,

If invalid arguments, return null, error
If start is none, use 0,
If step is none, use default value 1, >Here, we can see it will check step argument by step = validate_step(step); While, in validate_step, it will return 1 if step is null.

    /* No step specified, use a step of 1. */
    if (!step)
        return PyLong_FromLong(1);

Finally make a object make_range_object

static PyObject *
range_from_array(PyTypeObject *type, PyObject *const *args, Py_ssize_t num_args)
{
    rangeobject *obj;
    PyObject *start = NULL, *stop = NULL, *step = NULL;

    switch (num_args) {
        case 3:
            step = args[2];
            /* fallthrough */
        case 2:
            /* Convert borrowed refs to owned refs */
            start = PyNumber_Index(args[0]);
            if (!start) {
                return NULL;
            }
            stop = PyNumber_Index(args[1]);
            if (!stop) {
                Py_DECREF(start);
                return NULL;
            }
            step = validate_step(step);  /* Caution, this can clear exceptions */
            if (!step) {
                Py_DECREF(start);
                Py_DECREF(stop);
                return NULL;
            }
            break;
        case 1:
            stop = PyNumber_Index(args[0]);
            if (!stop) {
                return NULL;
            }
            start = _PyLong_GetZero();
            step = _PyLong_GetOne();
            break;
        case 0:
            PyErr_SetString(PyExc_TypeError,
                            "range expected at least 1 argument, got 0");
            return NULL;
        default:
            PyErr_Format(PyExc_TypeError,
                         "range expected at most 3 arguments, got %zd",
                         num_args);
            return NULL;
    }
    obj = make_range_object(type, start, stop, step);
    if (obj != NULL) {
        return (PyObject *) obj;
    }

    /* Failed to create object, release attributes */
    Py_DECREF(start);
    Py_DECREF(stop);
    Py_DECREF(step);
    return NULL;
}

make_range_object

Return the rangeobject by PyObject_New()

static rangeobject *
make_range_object(PyTypeObject *type, PyObject *start,
                  PyObject *stop, PyObject *step)
{
    rangeobject *obj = NULL;
    PyObject *length;
    length = compute_range_length(start, stop, step);
    if (length == NULL) {
        return NULL;
    }
    obj = PyObject_New(rangeobject, type);
    if (obj == NULL) {
        Py_DECREF(length);
        return NULL;
    }
    obj->start = start;
    obj->stop = stop;
    obj->step = step;
    obj->length = length;
    return obj;
}

Other operation functions in C

Index

We can check the index from a range:

>>> x = range(5)
>>> list(x)
[0, 1, 2, 3, 4]
>>> x.index(3)
3

In the C code, it first checks whether the target value in the range or not. If it contains the target value, it will call the PyNumber_Subtract() to get the distance from start to calculate the index.
PyObject *idx = PyNumber_Subtract(ob, r->start);

The whole function code is here:

static PyObject *
range_index(rangeobject *r, PyObject *ob)
{
    int contains;

    if (!PyLong_CheckExact(ob) && !PyBool_Check(ob)) {
        Py_ssize_t index;
        index = _PySequence_IterSearch((PyObject*)r, ob, PY_ITERSEARCH_INDEX);
        if (index == -1)
            return NULL;
        return PyLong_FromSsize_t(index);
    }

    contains = range_contains_long(r, ob);
    if (contains == -1)
        return NULL;

    if (contains) {
        PyObject *idx = PyNumber_Subtract(ob, r->start);
        if (idx == NULL) {
            return NULL;
        }

        if (r->step == _PyLong_GetOne()) {
            return idx;
        }

        /* idx = (ob - r.start) // r.step */
        PyObject *sidx = PyNumber_FloorDivide(idx, r->step);
        Py_DECREF(idx);
        return sidx;
    }

    /* object is not in the range */
    PyErr_Format(PyExc_ValueError, "%R is not in range", ob);
    return NULL;
}

Compare two range objects

We can also compare two range objects, actually, it compare the length of range objects, cmp_result = PyObject_RichCompareBool(r0->length, r1->length, Py_EQ);

>>> x = range(5)
>>> y = range(5)
>>> x==y
True
>>> id(x)
4551323104
>>> id(y)
4551266592

Conclusion:

We talked about how to use range and the source code behind it in Python and C! In summary:

range() is a built-in function, support 3 arguments: range(start, stop[, step])
In Python 3, range is a class, which means range() is the constructor function
It belongs to Sequence Types, it is iterable, not a iterator
It supports common operations including containment tests, element index lookup, slicing and support for negative indices, just as the operations in list
Range does not allow any of its parameters to be a float, should be integers

If you are interested, please check the Python doc and C code in Github!

Life is short, I use Python~

Reference:

[1] https://docs.python.org/3/library/stdtypes.html#range
[2] https://docs.python.org/3/glossary.html#term-iterable
[3] https://www.analyticsvidhya.com/blog/2021/07/everything-you-should-know-about-iterables-and-iterators-in-python-as-a-data-scientist/
[4] )https://docs.python.org/3/library/functions.html#func-range
[5] https://docs.python.org/3/library/stdtypes.html#typesseq-range
[6] https://www.copahost.com/blog/python-range/
[7] https://github.com/python/cpython/blob/main/Objects/rangeobject.c

MLflow Tutorial with Image Recognition Example (Mnist)

Frankkk — Tue, 31 Oct 2023 03:25:49 +0000

Introduction

What is MLOps?

MLOps(Machine Learning Operations) is a core function of Machine Learning engineering, aimed to simplify the deployment, maintenance, monitoring of machine learning models in production with reliability and efficiency. MLOps needs the collaboration from data scientists, devops engineers, and operation engineers[1].

Fig 1. Process of MLOps [2]

What is MLflow?

MLflow is an open source MLOps platform developed by Databricks. It helps machine learning engineers track and manage the models, code, dependencies, as well as deploy the models to the production environment. In other words, MLflow can simplify the process of building, training, deploying, and monitoring machine-learning models. For more details, please check https://mlflow.org/docs/latest/what-is-mlflow.html .

Fig 2. MLflow Component[3]

Why are MLOps and MLflow?

When training neural networks or machine learning models, we often face challenges listed as below, we can manually document model parameters in a text file or back up models for each experiment. This can be cumbersome and hinder collaboration with others. However, with the assistance of MLOps and MLflow, we can effortlessly train, deploy, and track models.

Challenge 1: How to track your model's version when you need to retrain your models?
We need to train our models, and adjust the parameters, environments, dependencies, datasets. It is hard to maintain the models when we need to collaborate with others.
Challenge 2: How to deploy a model and provide services to the public?
After we train a model, we need to publish the models and provide services to others. We need to provide APIs, and maintain the service reliable 7x24.
Challenge 3: How to monitor the models?
Again, we need to track the service, maintain the service.

MLflow Demo:

In this example, we use an image recognition model based on Keras/TensorFlow, and MNIST dataset. If you are interested, you can follow[4] for the example of sklearn_logistic_regression.

Installation & Setup

Step 1. Install conda
https://docs.conda.io/projects/conda/en/latest/user-guide/install/linux.html
Step 2. Create a virtual environment for mflow
cd $your_folder conda create -n mlflow (you can custom your environment name)
Step 3: Activate the conda environment
conda activate mlflow
Step 4: Install mlfow
conda install mlflow (or pip install mlflow)
Step 5: Run an MLFlow server with a filestore backend, the default port is 5000, you can change it by ‘-p’ option (i.e., mlflow server -p 5111)
mlflow server -h 0.0.0.0 --backend-store-uri /home/xxx/mlruns ( you can custom your backend uri path)
Step 6: Test the web http://localhost:5000

Prepare and Run the MLflow Project

Step1: Clone the MLflow project[5][6]:
git clone https://github.com/RoboticsAndCloud/mlflow-examples.git
Step 2: Go to the mlflow-examples - Keras/TensorFlow - MNIST example:
cd mlflow-examples/python/keras_tf_mnist

Here, notice three files: MLproject, conda.yaml, train.py

MLproject: a YAML defines the MLflow project structure, including project name, dependencies(in conda.yaml), parameters and entry point. Check https://mlflow.org/docs/latest/projects.html for more details.

conda.yaml: a YAML file defines your project’s dependencies

train.py: A python script which shows how to train your models and log the parameters into MLflow

Step 3: Set the tracking URI
Here, we can export the environment variable to ensure our results are logged to our MLflow server
export MLfLOW_TRACKING_URI=http://localhost:5000
Step 4: Running the MLflow Experiment
mlflow run . --experiment-name=keras_mnist --run-name runname_first_keras_keras_mnist
Step 5: Check the MLflow Dashboard http://localhost:5000

Model Serving

Model Serving exposes the models, allowing us to access the service through REST API endpoints.

Step 1: Get the run ID

Step 2: Server the model
mlflow models serve -m runs:/5c6a476d67d84239b01f874241f4009f/keras-model --port 5001
Step 3: Send recognition request and test the service

Target File Image '0':(You can find some from Mnist dataset)

Set the tracking URI

export MLfLOW_TRACKING_URI=http://localhost:5000

Send request

python3 keras_predict.py --model-uri runs:/5c6a476d67d84239b01f874241f4009f/keras-model  --data-path /home/ascc/LF_Workspace/mnist_png/testing/0/9993.png

Results

We can see ‘0’ is the highest probability.

Send HTTP request and Results
Remember, the port should be the same with the model server port, here is 5001

python convert_png_to_mlflow_json.py /home/ascc/LF_Workspace/mnist_png/testing/0/9993.png | curl -X POST -H "Content-Type:application/json"   -d @-   http://localhost:5001/invocations

Model version control:

Once the model has been verified, you can proceed to register it, verify its version, and deploy it in another production environment.

Version control is crucial when deploying models in a production environment. It enables you to monitor changes and revert to a specific version if needed.

The development process can be segmented into multiple stages for model verification. MLflow defines three key stages: staging, production, and archived.

Errors and solution you may meet:

Error 1 ModuleNotFoundError: No module named 'pip._vendor.six'

Solution :You need update your pipenv
pip install pip -U
pip install pipenv -U

Error 2 AttributeError: module 'virtualenv.create.via_global_ref.builtin.cpython.mac_os' has no attribute 'CPython2macOsFramework'

Solution: Virtualenv installed twice ( apt, pip, delete the python3-virtualenv by apt, sudo apt purge python3-virtualenv)

Error 3 mlflow.exceptions.MlflowException: Run '37011fce0ac847dbaa31efb5fabf842d' not found

Solution: You miss your tracking URI, 
export MLfLOW_TRACKING_URI=http://localhost:5000

Summary:

This article demonstrates how to train, track, and manage a model using the MLflow platform. MLflow enables seamless design, deployment, and monitoring of machine learning models. Furthermore, we posit that MLOps can significantly enhance contributions to the field of AI.

Reference:

[1]MLOps https://www.databricks.com/glossary/mlops
[2]What Is MLOps https://ml-ops.org/content/mlops-principles
[3]MLflow component https://www.datacamp.com/tutorial/mlflow-streamline-machine-learning-workflow
[4]Getting Started With MLflow https://saturncloud.io/blog/getting-started-with-mlflow/
[5]MLflow examples https://github.com/amesar/mlflow-examples
[6]MLflow examples https://github.com/RoboticsAndCloud/mlflow-examples

Forem: Frankkk

Python Tips- Maximum Value float('inf')

Reference

Python Tips: range (Function or Class) in Details

Introduction

What type of return value from range()?

Using help range We can see the source code of range class

We can use dir() to check the attributes:

The definition of range in Python doc

Sequence Types

Arguments

Operations

Common sequence operations(totally 12 items)

Source code of range in C

Workflow when create a range object

Range_new

range_from_array

make_range_object

Other operation functions in C

Index

Compare two range objects

Conclusion:

Reference:

MLflow Tutorial with Image Recognition Example (Mnist)

Introduction

What is MLOps?

What is MLflow?

Why are MLOps and MLflow?

MLflow Demo:

Installation & Setup

Prepare and Run the MLflow Project

Model Serving

Model version control:

Errors and solution you may meet:

Summary:

Reference:

Using `help range` We can see the source code of range class

We can use `dir()` to check the attributes: