Forem: Srijeyanthan

Angular production build error for safari-15.2-15.3

Srijeyanthan — Mon, 04 Apr 2022 11:22:40 +0000

building angular production causing "error: Invalid version: "15.2-15.3", this error is mainly coming from new safari browser in-compatibility. If the error comes, we can check first what are the browsers added in to our browsers list, this is the list controls angular application supported.

check the supported list.
npx browserslist

and_chr 99
and_ff 96
and_qq 10.4
and_uc 12.12
android 99
baidu 7.12
chrome 99
chrome 98
chrome 97
chrome 96
edge 99
edge 98
edge 97
firefox 98
firefox 97
firefox 96
firefox 91
ie 11
ios_saf 15.4
ios_saf 15.2-15.3
ios_saf 15.0-15.1
ios_saf 14.5-14.8
ios_saf 14.0-14.4
ios_saf 12.2-12.5
kaios 2.5
op_mini all
op_mob 64
opera 83
opera 82
safari 15.4
safari 15.2-15.3
safari 14.1
samsung 16.0
samsung 15.0

the highlighted line is causing issue, we can either remove or adding the following file in to your project would solve the problem.

Add .browserslistrc in to your project directory, and the following content.

last 1 Chrome version last 1 Firefox version last 2 Edge major versions last 2 Safari major versions last 2 iOS major versions Firefox ESR not IE 9-10 # Angular support for IE 9-10 has been deprecated and will be removed as of Angular v11. To opt-in, remove the 'not' prefix on this line. not IE 11 # Angular supports IE 11 only as an opt-in. To opt-in, remove the 'not' prefix on this line. not ios_saf 15.2-15.3 not safari 15.2-15.3

it will now work.

Handle HTTP OPTIONS

Srijeyanthan — Fri, 01 May 2020 02:26:34 +0000

The HTTP OPTIONS (RFC 7231, section 4.3.7: OPTIONS) method is used to describe the communication options for the target resource. The client can specify a URL for the OPTIONS method, or an asterisk (*) to refer to the entire server.

It is a request from the client to know what HTTP methods the server will allow, like GET, POST, etc.

The request might look like this when asking about the options for a particular resource:

OPTIONS /index.html HTTP/1.1

or like this when asking about the server in general:

OPTIONS * HTTP/1.1

The possible response from the server would be like this

HTTP/1.1 200 OK
Server: nginx/1.14.0 (Ubuntu)
Date: Fri, 01 May 2020 00:39:41 GMT
Content-Type: application/octet-stream
Content-Length: 0
Connection: keep-alive
Content-Length: 0
Content-Type: text/plain
Access-Control-Allow-Origin: *
Access-Control-Allow-Headers: origin,authorization, x-requested-with, content-type, accept
Access-Control-Allow-Methods: GET, POST

Why is the server receiving an HTTP OPTIONS request?
a. Most of the REST API applications or systems are needing it.
b. Browsers will send it to servers as "preflighted" requests to see if the server understands CORS

This is one of the best methods to control external access and safeguard your API endpoints. For example, let's assume, we defined API endpoints where the authorization header is needed. If we don't have that OPTIONS request, the browser will just send it to the server, where the server needs to do the validation, and then proceed accordingly.

So, what if the browser knows beforehand that the request can not be sent due to the server not authorized to accept any request without authorization header, so the browser can avoid un-necessary sending out the network.

Normally, it is best to practice to implement this OPTION header handling at API gateway level or front-webservers like Nginx. The API gateway should able to configure your end-points related OPTION headers so that the application behind your API gateway can avoid the processing and focus only business logic without too much worry about HTTP Header administration.

For example, if you need to configure all your OPTION request returned like this from Nginx

location / {
               if ($request_method = OPTIONS ) {
                        add_header Content-Length 0;
                        add_header Content-Type text/plain;
                        add_header Access-Control-Allow-Origin *;
                        add_header Access-Control-Allow-Headers 'origin,authorization, x-requested-with, content-type, accept';
                        add_header Access-Control-Allow-Methods 'GET, POST';
                        return 200;
                }
                client_max_body_size 200M;
                proxy_pass  http://xx.xx.xx.xx:8080;  // This is your micro service end point

        }

Further reading: https://stackoverflow.com/questions/15381105/cors-what-is-the-motivation-behind-introducing-preflight-requests
: https://security.stackexchange.com/questions/21413/how-to-exploit-http-methods

Decode pointer arithmetic

Srijeyanthan — Thu, 12 Mar 2020 13:39:50 +0000

This is a very short article about quick pointer arithmetic decoding. The basic pointer arithmetic is very easy to handle, and most of us are familiar with them. Array arithmetic is a bit tricky, but using following formula in your mind, can be easily interpreted.

in C/C++ handle pointer with an array as follows

int a[5] = {1,2,3,4,5}

int *p = a

print a // address of a, in this case, &a[0]
print a+1 // address of a[1] -> &a[1]

print *a // dereference array a[0]
print *(a+1) // dereference array a[1]

print p // address of &a[0]
print p+1 // address of &a[1]
print *(p+1) // dereference array a[1]

Look at the following example

int(*a)[5]; 

int b[5] = { 1, 2, 3, 4, 5 }; 
int i = 0; 
// Points to the whole array b 
a = &b;

If we print the value of *(*a+1)? Can you guess?

Now the question of, do we have any fundamental formula to apply all the cases, yes !!, we do have.

*(A+1) = A[i]
A+1 = &A[i]

Now, let's decode the above expression, *(*a+1) -> *( *(a+0) +1) -> *(A[0] +1) -> *(A[1])

int B[2][3]

print B = &B[0] Using above formula, B+0 -> &B[0]
print *B = B[0] or &B[0][0] *B -> *(B+0) -> B[0]
print B+1 = &B1 -> &B[1]
print *(B+1) = B[1] & &B[1][0] *(B+1) -> B[1]

print (B+1)+2 B[1]+2 &B[1][2] *(B+1)+2 ->B[1]+2 -> &B[1][2]
print *(*B+1) = = *(&B[0][1]) *((B+0)+1) -> (B[0] +1) ->(&B[0][1]) -> B[0][1]

Likewise, all the complex pointer arithmetics are easily solvable, There are many use cases for this variation, developers should remember this variation as functions are accepting all this variation to access. It will help to read complex source codes.

C/C++ hacks, part 1

Srijeyanthan — Tue, 10 Mar 2020 11:25:00 +0000

Even though C++ has gone too far these days by having several conventions like c++ 11/13 etc, programmers do not tend to use the new in-built library function, instead, they like to have their own logic or simple implementation. I would like to highlight some of the handy logic that all C/C++ programmers can use.

Find the given number is odd or even, you can simply use the AND operator to see output bit 0 or not

if it is 0 even
if it is 1 odd

#include <iostream>

using namespace std;


int main(int argc, char **argv)
{
     // find the number of even or odd
    int iNumber=101;
    if(iNumber & 1)
        std::cout<<"odd"<<std::endl;
    else
        std::cout<<"even"<<std::endl;
}

Let's assume, you are working with some protocol and need to build a raw buffer composed with tag and data (even your data is int/float/double, it will be written as serious by bytes)

Now, as per the protocol definition, the tag must be maximum 2 bytes, so we can hold maximum tag data like 65535

As per our protocol convention, we are going to assemble 2 bytes for a tag, and then 4 bytes for data length, finally actual data buffer.

The application which receives this data can easily extract the data by do the following

First, read the two bytes, then convert to tag
Then read 4 bytes to read, how much data is resided in the frame
Using the above length parameter, it can extract data.

So, how do we compose this frame

char *_zBuf ; // assume, it is already allocated
unsigned short usTag = (unsigned short) _iTag;

// get the first byte
((unsigned char*) _zBuf)[1] = (unsigned char) usTag & 0x00FF;
usTag >>= 8;
// get the second byte
((unsigned char*) _zBuf)[0] = (unsigned char) usTag & 0x00FF;

// now we need to write the length
unsigned int usDataLength = (unsigned int) _zData.length();

uint32_t uiLenConverted = htonl(usDataLength);
memcpy(_zBuf + 2, &uiLenConverted, sizeof(uint32_t));
// now write the data
memcpy(_zBuf + 6, _zData.length(), _zData.c_str());

XOR operator has many use cases in C++ due to the following properties

The XOR of a number with itself is zero.
A ^ A = 0

The XOR of a number with zero is the number itself.
A ^ 0 = A

The order in which you do XOR does not matter.
A ^ B = B ^ A

Assume the following use case
The set is given with a series of integers, where all the numbers are placed evenly except one. Find that number?

for example array[5] = {10,10,80,14,14} , answer should 80

#include <iostream>

int FindOddOccurance(int arr[], int n)
{
    int res = 0, i;
    for (i = 0; i < n; i++)
    {
// using above mentioned XOR properties.
        res ^= arr[i];
    }
        return res;
}

int main(void)
{
    int arr[] = { 10,10,80,14,14 };
    int n = sizeof(arr) / sizeof(arr[0]);
    std::cout<<"The odd occurring element is :"<<FindOddOccurance(arr, n)<<std::endl;
    return 0;
}

Writing part 2...

Compile-time polymorphism !!

Srijeyanthan — Sat, 22 Feb 2020 07:11:13 +0000

Curiously recurring template pattern is one of the rarely used design patterns in general programming, normally this is referred to as CRTP. We all are well familiar with dynamic polymorphism implementation in C++, and how it compiler handles dynamic features using virtual functions.

The above diagram show, how B and C have their own implementation of the write method, it is very straightforward. Once your program compiled, compiler basically creates VPtr table(virtual pointer table) and stores the addresses of virtual methods in that table. Once the Virtual Table is created, it is shared across all the instances of the class.

The compiler only creates one single instance of VTable to be shared across all the objects of a class. Every instance of the class B or C has its own version of VPtr.

Normally, compiler adding 4 bytes for VPtr and If we print the size of a class object containing at least one virtual method, the output will be sizeof(class data) + sizeof(VPtr).

So, what is the issue here? well, if you are working on a high throughput system where every single function call is matters, and collectively delaying the overall process. If your program heavily implementing a virtual function, then frequently calling this writing method in the above example ends up in frequent lookup of VTable.

Now let's jump into compile-time polymorphism where bindings happen during compile time.

template <class T>
class Writer
{
  public:
    Writer()  { }
    ~Writer()  { }

    // Look at this declaration
    void write(const char* str) const
    {
      static_cast<const T*>(this)->writeImpl(str); 
    }
};

class FileWriter : public Writer<FileWriter>
{
  public:
    FileWriter(FILE* aFile) { mFile = aFile; }
    ~FileWriter() { fclose(mFile); }

    //here comes the implementation of the write method on the subclass
    void writeImpl(const char* str) const
    {
       fprintf(mFile, "%s\n", str);
    }

  private:
    FILE* mFile;
};


class ConsoleWriter : public Writer<ConsoleWriter>
{
  public:
    ConsoleWriter() { }
    ~ConsoleWriter() { }

    void writeImpl(const char* str) const
    {
      printf("%s\n", str);
    }
};


// Driver code 
int main() 
{ 
    // An Writer type pointer pointing to FileWriter 
    Writer<FileWriter>* pWriter = new FileWriter; 


    pWriter->write("Hi I am faster"); 



    return 0; 
}

Now, the compiler won't allocate that 4-byte extra memory to hold VPtr, and it will execute really faster during the high-frequency function calls. Please note that CRTP won't really fit in all the use cases, the developer should decide the places where it can fit to reduce the overhead of VTable lookup.

Sending M-JPEG streaming over HTTP

Srijeyanthan — Sat, 22 Feb 2020 00:56:54 +0000

The M-JPEG called motion JPEG is a video compression method in which each video frame is compressed separately as a JPEG image. M-JPEG is now widely used by most of the video-capture devices such as digital cameras, IP cameras, and webcams. On the other side, clients such as browsers, VLC, QuickTime players are supporting to play video.

Sending M-JPEG over the network has slightly higher overhead than other advanced compression technologies like H.264 due to individual frame encoding. For example, H.264 is mainly focusing consecutive frames to compress, but then, the client needs to wait until it receives all the necessary frames to reconstruct the video. That is why M-JPEG is faster sending it if you have really great bandwidth network.

This article explains, how developers can send M-JPEG over HTTP after they processed ( do some image processing). HTTP doesn't have any methods to send streaming data, we will have to use the multi-part sending procedure to stream video.

Let's say, we need to stream webcam video through your own web server,

Note: Actual web server needs nonblocking I/O implementation to handle multiple clients.

#include <opencv2/core.hpp>
#include <opencv2/videoio.hpp>
#include <opencv2/highgui.hpp>
#include <opencv2/tracking.hpp>

#include <iostream>
#include <stdio.h>
using namespace cv;
using namespace std;


#define PROT_CRLF   "\x0D\x0A"

int main(int, char**)
{
    Mat frame;

    VideoCapture cap;

    int deviceID = 0;             
    int apiID = cv::CAP_ANY;     

    cap.open(deviceID + apiID);
    // check if we succeeded
    if (!cap.isOpened()) {
        cerr << "ERROR! Unable to open camera\n";
        return -1;
    }

    cout << "Start grabbing" << endl
        << "Press any key to terminate" << endl;
    int iFrameCount=0;
    for (;;)
    {
        ++iFrameCount;
        cap.read(frame);

        if (frame.empty()) {
            cerr << "ERROR! blank frame grabbed\n";
            break;
        }

        imshow("Live", frame);

        //Update the tracking result with new frame
        multiTracker->update(frame);

      // Draw tracked objects
       for(unsigned i=0; i<multiTracker->getObjects().size(); i++)
      {
        rectangle(frame, multiTracker->getObjects()[i], colors[i], 2, 1);
      }


      std::stringstream ssFrameText;
      ssFrameText<<iFrame;
    cv::putText(img_resized, //target image
            ssFrameText.str().c_str(), //text
            cv::Point(20, img_resized.rows - 20), //top-left position
            cv::FONT_HERSHEY_COMPLEX_SMALL, 0.8, CV_RGB(118, 185, 0), //font color
            2);

       // Now we are ready to send the processed frame. 


    }

    return 0;
}

Now, we will send the processed frame over HTTP. The process is the following, first, we need to encode the OpenCV image using cv::imencode, this will result in the encoded raw buffer.

 void SendOverHTTP(Mat &frame)
 {

   std::vector<int> encode_params;
   encode_params.push_back(CV_IMWRITE_JPEG_QUALITY);
   encode_params.push_back(m_iImageQuality);

   std::vector<uchar> encoded_buffer;
   cv::imencode(".jpeg", frame, encoded_buffer, encode_params);

   if(bIsFirstFrame)
   {
       int iWriteLen = snprintf(zNetworkBuffer, 40, "%s%s%s", "Content-Type: ", "image/jpeg", PROT_CRLF);
       zNetworkBuffer += iWriteLen;
       // other headers also needed
   }
   else
   {
      iWriteLen = snprintf(zNetworkBuffer, 40, "%s%d%s%s", "Content-Length: ", encoded_buffer.size(), PROT_CRLF,PROT_CRLF);
      zNetworkBuffer += iWriteLen;
   }
 }

If you want to access the video through chrome, then sending raw M-JPEG won't work, as chrome stopped supporting direct streaming. Instead, we can embed the URL and send it to the client like the following.

<html>
<body>
<img src="http://127.0.0.1:9600?image_width=640&image_height=480"
</body>
</html>

Ready to stream :)

An efficient way to send larger messages over the network

Srijeyanthan — Tue, 11 Feb 2020 10:41:29 +0000

In this post, we will see how we can send large messages over the socket to reduce the bandwidth and increase the message throughput. We are always writing server applications to interact with clients back and worth, but most of the time, we just send the messages using open API application protocol like JSON.

If you are not worried about bandwidth and performance, we can directly send the message as it is, but that is not the case for every application and environment. Think about the following use-case.

Producer A generating JSON message in the following schema, where data is a big chunk of the binary message. Total message size is 665417 bytes.

{
  "op": "publish",
  "topic": "/map",
  "msg": {
    "info": {
      "origin": {
        "position": {
          "w": -21.2,
          "x": -10,
          "y": 0
        },
        "orientation": {
          "w": 0,
          "x": 0,
          "y": 0,
          "z": 1
        }
      },
      "width": 608,
      "height": 384,
      "resolution": 0.05,
      "map_load_time": {
        "secs": 0,
        "nsecs": 1000000
      }
    },
    "header": {
      "header_stamp": {
        "secs": 0,
        "nsecs": 1000000
      },
      "frame_id": "map",
      "seq": 0
    },
    "data": [
      -1,
      -1,
      0,
      0
    ]
  }
}

How can we send this message over the network to the client efficiently? The most successful method is to use the PNG compression, using that, we can selectively agree on the fields to compress or compress the entire message and re-packaged in a simplified JSON format.

PNG compression basically compresses any message into PNG format by finding the minimum width and height of the input data. We are going to use the 3 channels (BGR) to fill our data. For example, 3 channel arrays are like the following.

So, let's write our own PNGDataCompression function in C++.

      std::string ConvertToPNGEncoding(std::string _zDataMsg)
     {
        int iBufferLen = _zDataMsg.length();
        int iWidth = floor(sqrt(iBufferLen / 3.0));
        int iHeight = ceil((iBufferLen / 3.0) / iWidth);
        // we need to calculate how much space is left to fill to complete
        // entire data structure.
        int iBytesNeeded = int(iWidth * iHeight * 3);

        // let's fill remaining space with new lines.
        while (iBufferLen < iBytesNeeded)
        {
                _zDataMsg.append("\n");
                iBufferLen += 1;
        }

        // now lets encode the entire buffer using opencv png encoding.
        std::vector<uint8_t> vecRAWBuffer;
        vecRAWBuffer.assign(_zDataMsg.begin(), _zDataMsg.end());

        cv::Mat imgOriginalData(iWidth, iHeight, CV_8UC3);
        imgOriginalData.data = vecRAWBuffer.data();

        cv::Mat imgGreyImage;
        cv::cvtColor(imgOriginalData, imgGreyImage, CV_BGR2RGB);

        std::vector<uint8_t> vecOfEncodedBuffer;
        std::vector<int> encode_params;
        encode_params.push_back(CV_IMWRITE_PNG_COMPRESSION);
        cv::imencode(".png", imgGreyImage, vecOfEncodedBuffer, encode_params);

         // now we have the binary data, in order to send through the standard JSON, we will apply base64 encoding. 


          string zPNGEncoded(vecOfEncodedBuffer.begin(), vecOfEncodedBuffer.end());

        std::string encodedData = StandardBase64Encode(zPNGEncoded.c_str(),zPNGEncoded.length());

         return encodedData;
     }


// standared base64 encoding. 
     std::string StandardBase64Encode(const char * _zEncodingBuffer, unsigned int _iBufferLen) {
  std::string ret;
  int i = 0;
  int j = 0;
  unsigned char char_array_3[3];
  unsigned char char_array_4[4];

  while (_iBufferLen--) {
    char_array_3[i++] = *(_zEncodingBuffer++);
    if (i == 3) {
      char_array_4[0] = (char_array_3[0] & 0xfc) >> 2;
      char_array_4[1] = ((char_array_3[0] & 0x03) << 4) + ((char_array_3[1] & 0xf0) >> 4);
      char_array_4[2] = ((char_array_3[1] & 0x0f) << 2) + ((char_array_3[2] & 0xc0) >> 6);
      char_array_4[3] = char_array_3[2] & 0x3f;

      for(i = 0; (i <4) ; i++)
        ret += zBase64Chars[char_array_4[i]];
      i = 0;
    }
  }

  if (i)
  {
    for(j = i; j < 3; j++)
      char_array_3[j] = '\0';

    char_array_4[0] = (char_array_3[0] & 0xfc) >> 2;
    char_array_4[1] = ((char_array_3[0] & 0x03) << 4) + ((char_array_3[1] & 0xf0) >> 4);
    char_array_4[2] = ((char_array_3[1] & 0x0f) << 2) + ((char_array_3[2] & 0xc0) >> 6);
    char_array_4[3] = char_array_3[2] & 0x3f;

    for (j = 0; (j < i + 1); j++)
      ret += zBase64Chars[char_array_4[j]];

    while((i++ < 3))
      ret += '=';
  }

  return ret;
}

As a result, the original message size is 665417, it will be reduced to just 10500. So, we can construct our new message like following

   {
     "op":"png"  /*what is the compression we used, so that client can use is accordingly */
     "data":"iVBORw0KGgoAAAANSUhEUgAA...."
   }

So, servers heavily sending large messages to the client can use the above technique to publish messages.

Python has all the necessary libraries to implement the above technique by using a couple of imports.

from PIL import Image
from base64 import standard_b64encode, standard_b64decode
from math import floor, ceil, sqrt


def pngencode(string):

    length = len(string)
    width = floor(sqrt(length/3.0))
    height = ceil((length/3.0) / width)
    bytes_needed = int(width * height * 3)
    while length < bytes_needed:
        string += '\n'
        length += 1
    i = Image.frombytes('RGB', (int(width), int(height)), string)
    buff = StringIO()
    i.save(buff, "png")
    encoded = standard_b64encode(buff.getvalue())
    return encoded

I hope this will help serverside programmers to implement png compression over larger messages.