Forem: darryncampbell

Long Polling vs WebSockets

darryncampbell — Tue, 28 Nov 2023 11:07:18 +0000

Web applications were initially designed as a client-server architecture. Clients will create an HTTP/HTTPS request to the designated server requesting or modifying a piece of information. For example, a basic web application will follow a similar flow.

The client requests data from the server
The load balancer routes the requests to the appropriate server
The server queries the applicable database for some data
The database returns the queried data to the server
The server processes the data and sends the data back to the client

A simple HTTP request is the most common way to receive server information. However, what if you wanted to get data back as soon as it was added to the database or sent to the server? With a basic web application designed as a client-server architecture, you would have to repeat this process over and over to check if there is new information that has been added to the database. This process is known as polling or sometimes referred to as short polling. The downside to this approach is that data will not be returned the majority of the time as the server hasn’t received any new information. Let's solve this problem and discuss the advantages and disadvantages of Long Polling and Websockets.

Overview: Long polling vs WebSockets

Long Polling

Long polling is an approach where the client will send an API request to the server, but instead of receiving an instant response from the server, it entails maintaining the HTTP connection. Maintaining the HTTP connection enables the server to reply later when data becomes available or the timeout threshold has been reached. After receiving the response, the client will immediately send the subsequent request.

Instead of sending numerous requests repeatedly until the server receives new information, as in polling, the client only has to send one request to the server to get the latest information. After receiving the data, the client can initiate a new request, repeating this process as often as necessary.

A flow for Long polling will look as follows:

The client-side makes an HTTP request to the server requesting some data
The server does not respond immediately with the requested information but waits until new information is available
When new data becomes available, the server responds with new information
The client receives that data and immediately sends another request to the server, re-starting the process

WebSockets

WebSockets are a modern technology built on top of a device’s TCP/IP stack. The only relationship to HTTP protocol is that HTTP servers interpret its handshake to establish a connection. It is a bidirectional, full-duplex protocol that is stateful, meaning the connection between the client and server will persist until either party decides to terminate it.

Unlike long polling, which is only a half-duplex solution, the process does not need to repeat after receiving the latest information from the server. WebSocket technology allows us to keep the connection alive after the new information has been returned and perform bidirectional updates. The client can send information back to the server and listen for further information in the same request.

A WebSocket connection flow will look something like this:

The client-side initiates a WebSocket by sending a request which contains an upgrade header to switch the communication protocol to a WebSocket protocol
If the server can establish a connection and agrees with the client's terms, then it sends a response to the client acknowledging the WebSocket handshake request
Once the client receives a successful WebSocket connection, the client and the server can now start sending data in both directions allowing real-time communication
The server or the client decides to terminate the connection

When to choose Long Polling vs WebSockets

There is a debate on when to use long polling or WebSocket protocol. Both have their benefits and limitations and are often used for different purposes. In this section, we will discuss the key benefits of both long polling and WebSockets.

Pros of Long Polling vs WebSockets

Compatibility: Long polling is an older technology used more as a technique, making it a more compatible option than WebSockets. It is built on top of XMLHttpRequest, consistent with a broader range of web browsers and network configurations.

Network: With today’s technology, people constantly switch networks from 3G to LTE to WiFi. WebSockets must be configured to adapt to a change in the network connection. This configuration is because the connection has to be re-established with the server and cannot be revived after the client has opted to close the connection. With long polling, this is not an issue as it is set up where after the predetermined time (usually 20 seconds), the client will try to send another request re-establishing a connection with the server automatically and does not have to be handled in an error state as with WebSockets.

Use cases to choose long polling over WebSockets

Long polling and WebSockets are generally used in cases where real-time updates are required. Some examples include in-app chat, real-time pricing, geo-tracking, and IoT.

Long polling provides benefits over WebSockets in use cases with low-frequency real-time updates. These benefits are because long polling is a half-real-time solution where the connection needs to be re-established. Additionally, as mentioned above, if users are in an environment with low bandwidth or an unstable network provider, long polling is architected to re-establish the connection with no additional complications. However, long polling is an older technology/technique to perform real-time updates. As a result, it is less advanced and less flexible than WebSockets but has more support for legacy systems.

When to choose WebSockets vs long polling

Pros of WebSockets vs long polling

Reduced Resource Utilization: WebSockets maintain a persistent connection between the client and the server, reducing the overhead of establishing a new connection for each real-time update. The constant connection reduces resource utilization on the client and server side regarding network bandwidth, memory, and CPU to achieve real-time communication.

Improved Scalability: Due to the nature of WebSockets and its bidirectional communication between the client and the server, the server can push updates to the client in real time, reducing the number of requests sent. Long polling must re-establish a connection every time the client needs new information. As the user base scales, this can put a lot of strain on an individual server.

Advanced Functionality: WebSockets provide full-duplex communication channels that achieve real-time data transfer and low latency. Long polling is sometimes considered only a half-real-time solution and not ideal for high-traffic scenarios or use cases that require real-time updates. Advanced functionality brings a smoother end-user experience as they will receive more seamless updates to their application.

Use cases to choose WebSockets over long polling

WebSockets are better suited for applications that require high-frequency updates. Examples include chat applications or real-time data feeds. The persistent connection allows for efficient transmission making it a more seamless experience for the end user. Multiplayer games and collaboration tools generally use WebSockets as well. The bi-directional two-way communication allows the server to signal the client, which can be beneficial for receiving real-time updates from other clients. For example, the server can signal the client, telling it to update another player's position depending on their actions.

In terms of scaling, or when a user base starts scaling, it is ideal to switch to the WebSocket protocol. The strain on an individual server will become too large if the client base uses long polling technology. Sending a request every 20 seconds has poor utilization and will cause the server to slow down over time, increasing latency per request.

How PubNub fits into the conversation of Long polling vs WebSockets

WebSockets and long polling do offer valuable solutions to create real-time applications. However, many other considerations come into play when building upon these use cases. With today’s technology, nothing is as simple as sending a message or data from one client to another. There is almost always required functionality on top of the real-time system that a developer is trying to create. For example, in an in-app chat, you can look at presence updates (signalling) when users are online, profanity filtering, or even read/deliver messages.

On top of adding specific functionality, there are still problems with underlying infrastructure, such as complexities with handling scalability. When using particular technologies such as Socket.io to create a WebSocket, developers will still have to take care of dynamically spawning servers around the world behind a load balancer to manage the utilization of each server. These problems become more and more complex the more you scale.

PubNub’s real-time data APIs allow users to develop powerful, event-driven applications to facilitate real-time communication across all devices, regardless of the specific use case. PubNub offers a variety of SDKs, such as a JavaScript SDK for web applications and a C-Core SDK for IoT applications, to ensure seamless integration with the chosen device. With PubNub, you don’t have to worry about selecting a suitable alternative or the underlying complexities of implementing a real-time solution.

Now that you understand when to use long polling vs WebSockets sign up for a free trial or schedule a demo to explore what you can build with PubNub.

How can PubNub help you?

This article was originally published on PubNub.com

Our platform helps developers build, deliver, and manage real-time interactivity for web apps, mobile apps, and IoT devices.

The foundation of our platform is the industry's largest and most scalable real-time edge messaging network. With over 15 points-of-presence worldwide supporting 800 million monthly active users, and 99.999% reliability, you'll never have to worry about outages, concurrency limits, or any latency issues caused by traffic spikes.

Experience PubNub

Check out Live Tour to understand the essential concepts behind every PubNub-powered app in less than 5 minutes

Get Setup

Get Started

The PubNub docs will get you up and running, regardless of your use case or SDK

WebSocket Alternatives

darryncampbell — Tue, 28 Nov 2023 11:02:02 +0000

The rapid growth of the internet has led to an increase in demand for real-time technology, enhancing communication between clients and servers. WebSockets have grown in popularity and are now one of the most popular methods for enabling real-time communication. However, WebSockets may not always be the best option, as the choice of technology depends on the specific use case. There are many alternative options available that might be better suited for particular situations.

What is a WebSocket?

A WebSocket is a protocol that provides full-duplex communication and bi-directional communication channels between clients and servers over the web. It allows real-time data transmission between the client side and the client-server without reconnecting to the server. Unlike a traditional HTTP protocol, WebSockets enable the client and server to initiate communication and send data anytime, making them ideal for realtime data transmission.

Why choose a WebSocket Alternative?

Almost every time developers think about real-time communication, we use WebSockets. Why is this the case? Are they the only/best choice to achieve real-time transmission, no matter the use case?

To put it simply, WebSockets provide advanced functionalities but, depending on the use case, can have many drawbacks that can quickly increase a project's complexity. To solve this, let's consider why developers would pick a WebSocket alternative.

Performance: WebSockets are most known for their advanced functionality but sometimes can pose an overkill solution and lead to poor performance. It is essential to consider the use case when deciding to use WebSockets. For example, there are better solutions than WebSockets if you need a high concurrency of connections. WebSocket connections have to remain alive between the client and the server leading to increased load on the server and underlying infrastructure as you scale.

Interoperability: WebSockets are supported by all modern web browsers/networks. By chance, if the software is being used in a legacy browser or network, it is essential to consider if WebSockets has support in that legacy system. Additionally, networks and proxies sometimes block WebSocket connections altogether, so where you use the software is another factor to consider.

Security: WebSockets are not secure as some alternatives and have vulnerabilities that malicious actors can exploit. Some reasons WebSockets are considered less secure than other protocols include cross-site WebSocket hijacking and origin spoofing.

WebSocket Alternatives

Long Polling

Long polling is an approach where the client sends an HTTP/HTTPS request to the server, but instead of receiving an instant response from the server, it entails maintaining the HTTP connection. Maintaining the HTTP connection enables the server to reply later when data becomes available, or the timeout threshold has been reached. After receiving the response, the client will immediately send the subsequent request.

Long polling achieves real-time communication without the bi-directional communication channels that WebSockets provide. The core difference is instead of maintaining a persistent connection as with WebSockets, the client will send multiple requests to re-establish the connection with the server.

MQTT

MQTT is a lightweight, publish-subscribe, machine-to-machine network protocol designed for remote locations with devices that have limited network bandwidth. Smart sensors, wearables, and other IoT devices typically need to use such a protocol due to their resource constraints. While WebSockets can be used for various applications, MQTT is explicitly designed for machine-to-machine communication, which is why it’s considered an alternative under these use cases. MQTT provides features like low overhead, efficient message delivery, and support for offline operation.

WebRTC

WebRTC is a free, open-source project written in C++ and JavaScript that provides web browsers and mobile devices with real-time communication. WebRTC is built to stream audio and video directly from peer-to-peer without needing server-side connections or additional plugins. For use cases such as live streaming, video conferences, or group calls, these functionalities can be engineered rapidly with HTML5 and JavaScript using the WebRTC API. In summary, WebRTC’s peer-to-peer nature can help reduce the server load and provide a more efficient and high-quality communication experience compared to WebSockets in these use cases.

WebTransport vs WebSocket

WebTransport is a new web communication protocol that provides a low-latency and high-throughput connection between a client and a server. It was created to address some of the limitations of traditional protocols, such as WebSockets, and provide a modern alternative that can better handle the demands of today’s web applications.

While WebSockets typically start as an HTTP/1.1 Upgrade request, WebTransport supports multiple protocols, including HTTP/2, HTTP/3, and QUIC, and is a UDP-based transport. Additionally, WebTransport supports sending data reliably via its streams API and unreliably via its datagrams API. With this new technology, how does it compare to WebSockets and what are the advantages of switching to WebTransport protocol?

Advanced Functionality: A challenge with WebSockets is that it can be difficult to send different types of data, such as an image and text, over a single connection. The issue arises because the data is divided into smaller units, known as packets, for transmission over a TCP connection. When dividing the data into packets, there is no way to distinguish between the plain text and the encoded image contained within the data. To resolve this, WebSockets often need to make two separate connections between the server and client, one for pictures and one for text. However, WebTransport provides a solution by supporting multiple streams, known as multiplexing. By supporting multiple streams, developers can send different types of data over one connection, reducing the overhead of establishing and maintaining numerous connections. This becomes essential when the server needs to support many clients. Multiple connections per client, as in WebSockets, can lead to complexities when the server starts to scale.

Improved Performance: WebTransport is a UDP-based transport protocol more performant than TCP-based protocols like WebSockets. Due to it being UDP based, there is no error correction or retransmission. Error correction refers to detecting and correcting an error that may occur in the transmitted data. Retransmission refers to the processing of resending lost or corrupted data packets over the network. Due to not having the overhead of correction and retransmission, this lowers the latency of WebTransport. WebTransport also offers a WebTransportCongestionControl configuration which can optimize the throughput and latency of requests. Secondly, Establishing a new connection with WebTransport over QUIC handshake is typically faster than TCP over TLS.

Reliability: The lack of error correction and retransmission might make a developer think WebTransport is less reliable than WebSockets. That is not the case; WebTransport, by design, provides more reliability than WebSockets, especially in scenarios with high-bandwidth communication and challenging network conditions. WebTransport provides a streams API for reliable unidirectional (WebTransportReceiveStreams) or bidirectional two-way (WebTransportBidirectionalStream) data transfer. This API endpoint offers configurations such as WebTransportReliabilityMode, which can allow the connection to have a TCP fallback in scenarios where reliable data transfer is essential.

Security: WebTransport offers a more secure protocol than WebSockets. It provides end-to-end encryption by default, which ensures that the data between the client and server cannot be intercepted or modified in transit. One of these security features is the use of an “Origin” header, which provides a way for the server to verify that the requests are coming from a trusted source. This helps prevent cross-site request forgery (CSRF) attacks.

Other Alternatives

Sever-Sent Events

Server-Sent Events (SSE) is a server push technology that allows a server to push data to a client over a single HTTP connection. SSE is a simple, efficient protocol for sending real-time updates to a web page or application, making it well-suited for specific use cases. Unlike traditional HTTP requests, which close the connection after a single response, SSE opens a long-lived connection between the server and the client, allowing the server to send multiple updates over the same connection. Compared to WebSockets’s bidirectional connections, Server-Sent Events are only unidirectional. The client only has to initiate the connection and close it. However, the limited feature set of Server-Sent Events makes it easier to implement and debug. Server-Sent Events was the start of the EventSource API, which makes it easy to implement in web browsers.

How PubNub can help with WebSocket alternatives

WebSocket protocol and its alternatives, such as WebTransport, are all valuable solutions to handling real-time communications. However, no matter what solution is selected, time will be wasted in implementing the underlying infrastructure and complexities when scaling. For example, when using particular technologies such as Socket.io to create a WebSocket server, developers will still have to take care of dynamically spawning servers around the world behind a load balancer to manage the utilization of each web server.

Now that you understand when to use WebSockets or an alternative, sign up for a free trial or schedule a demo to explore what you can build with PubNub.

How can PubNub help you?

This article was originally published on PubNub.com

Our platform helps developers build, deliver, and manage real-time interactivity for web apps, mobile apps, and IoT devices.

Experience PubNub

Check out Live Tour to understand the essential concepts behind every PubNub-powered app in less than 5 minutes

Get Setup

Get Started

The PubNub docs will get you up and running, regardless of your use case or SDK

What is Geohashing?

darryncampbell — Mon, 27 Nov 2023 16:03:38 +0000

Geohashing (or geohash) is a geocoding method used to encode geographic coordinates (latitude and longitude) into a short string of digits and letters delineating an area on a map, which is called a cell, with varying resolutions. The more characters in the string, the more precise the location.

What is an example of a geohash?

Geohash is a public domain of encoding coordinates. An example of a geohash is the coordinate pair 28.6132,77.2291 being converted into a geohash of ttnfv2u.

What is the maximum length of a geohash?

The maximum length of a geohash is 12.

How Does Geohashing Work?

Geohash algorithm and calculation

Geohashes use Base-32 alphabet encoding (characters can be 0 to 9 and A to Z, excl "A", "I", "L" and "O”). Imagine the world is divided into a grid with 32 cells. The first character in a geohash identifies the initial location as one of the 32 cells. This cell will also contain 32 cells, and each one of these will contain 32 cells (and so on repeatedly). Adding characters to the geohash sub-divides a cell, effectively zooming in to a more detailed area.

The precision factor determines the size of the cell. For instance, a precision factor of one creates a cell 5,000km high and 5,000km wide, a precision factor of six creates a cell 0.61km high and 1.22km wide, and a precision factor of nine creates a cell 4.77m high and 4.77m wide (cells are not always square).

Geohashing Examples and Use Cases

Geohashing was originally developed as a URL-shortening service but it is now commonly used for spatial indexing (or spatial binning), location searching, mashups and creating unique place identifiers.

Geohash benefits

A geohash is shorter than a regular address, or latitude and longitude coordinates, and therefore easier to share, remember and store.

Social Networking: Chat with people near you within a particular cell, and to create chat apps.
Proximity Searches: Find nearby locations using API mapping or geolocation APIs and identify places of interest, restaurants, shops and accommodation establishments in an area.
Digital Travels: Geohashers go on global expeditions to meet people and explore new places. The twist: the destination is a computer-generated geohash and participants in this turnkey travel experience have to write up and post their story on the internet.
Custom Interactive Apps: Geohashing can be used to create real-time, interactive apps.

You can try it out here. Use the coordinates 40.748440989 and -73.985663981 for a view of the area around the Empire State Building in New York. Change the precision factor to increase or decrease the resolution (size of the cell), or use the geohash dr5ru6j2c5fqt for a 13-digit precision factor to zoom right in.

What are the drawbacks to using a geohash?

The drawbacks to geohashing include:

Using a grid based geohashing algorithm does need meet high-precision requirements
Geohashing deviation changes as latitude increases because the Earth is an irregular ellipse.

Ready to incorporate real-time functionality in your app? Create a free PubNub account today.

Other Resources

How can PubNub help you?

This article was originally published on PubNub.com

Our platform helps developers build, deliver, and manage real-time interactivity for web apps, mobile apps, and IoT devices.

Experience PubNub

Check out Live Tour to understand the essential concepts behind every PubNub-powered app in less than 5 minutes

Get Setup

Get Started

The PubNub docs will get you up and running, regardless of your use case or SDK

What is HTTP Streaming?

darryncampbell — Mon, 27 Nov 2023 09:09:44 +0000

What is HTTP streaming?

HTTP streaming, also known as HTTP-based streaming or HTTP live streaming, is a technique used to deliver real-time multimedia content, such as audio or video, over the Internet. This protocol allows continuous data transmission from a server to a client device, enabling users to consume media content without requiring complete file downloads.

Unlike traditional file download methods, where the entire file needs to be downloaded before playback can begin, HTTP streaming allows for the immediate playback of media content. It works by dividing the media file into smaller segments, or chunks, which are then continuously delivered to the client device.

HTTP streaming utilizes the Hypertext Transfer Protocol (HTTP) as its communication protocol. It leverages existing web infrastructure and uses HTTP servers' scalability, caching, and load-balancing capabilities. This makes it an efficient and flexible solution for delivering real-time content to many users.

How does HTTP streaming work?

At a high level, HTTP streaming works by dividing the media file into small chunks and transmitting it to the client over an HTTP connection. The client, typically a web browser or a media player, continuously requests and receives these chunks, allowing for the seamless playback of the media.

There are two main approaches to HTTP streaming: progressive download and adaptive streaming.

1. Progressive Download:

Progressive Download lacks the adaptability that adaptive streaming provides. The entire media file is downloaded with progressive download before the client can start playing it. This means that if there are any Wi-Fi network interruptions or fluctuations in bandwidth during the download process, the user may experience buffering or delays in playback. This can frustrate users and may result in a poor user experience.

2. Adaptive Streaming:

Adaptive streaming is a crucial technology for developers building real-time chat and messaging applications that deliver media files like video or audio. It ensures that the media content can be streamed smoothly and efficiently, regardless of the user's network conditions. This is especially important in today's digital landscape, where users expect high-quality, uninterrupted media playback but sometimes have different internet speeds.

Adaptive streaming typically involves the following steps:

Content-Encoding: The media file is encoded into multiple variations with different bitrates and quality levels. These variations are stored on the server.
Manifest File: A manifest file is created, which contains information about the available variations and their corresponding URLs.
Initial Request: The client requests the manifest file from the server, which provides information about the available variations of the media file.
Variant Selection: The client selects the desired variant based on network conditions and device capabilities. It then requests the corresponding media chunks from the server.
Chunk Delivery: The server delivers the chunked media to the client over an HTTP connection. The client continuously requests and receives these chunks, adjusting the playback quality if necessary.
Bitrate Adaptation: During playback, the client monitors the network conditions and dynamically adjusts the selected variant based on available bandwidth. It may switch to a lower bitrate variant if the network becomes congested or to a higher bitrate variant if the network conditions improve.
Seamless Playback: By continuously receiving and playing the media chunks, adaptive streaming provides a seamless playback experience, allowing users to enjoy the content without interruptions or buffering.

What benefits does HTTP streaming provide?

HTTP streaming provides several benefits for developers building real-time chat and messaging applications:

Real-time data delivery: HTTP streaming enables real-time delivery of data, allowing messages to be instantly sent and received by users. This ensures that users can have real-time conversations without any noticeable delays.
Scalability: HTTP streaming is highly scalable, able to handle many concurrent connections and deliver messages to multiple users in real time. This is essential for applications that support a growing user base and handle high message volumes.
Reduced network overhead: With HTTP streaming, only the necessary data is sent over the network when new information is available, reducing network overhead. This contrasts with other approaches like polling, where constant requests are made even when no new data exists.
Efficient resource utilization: HTTP streaming allows for efficient utilization of server resources as it eliminates the need for frequent polling requests. This can reduce server load and improve performance, especially in applications with many active users.
Better user experience: HTTP streaming enhances the user experience for chat and messaging applications by providing real-time updates and instant message delivery. Users can have more interactive and engaging conversations without the frustration of delays or missed messages.
Security: HTTP streaming can provide a secure communication channel by leveraging existing security measures like HTTPS. This ensures that sensitive user data and conversations are protected from unauthorized access and eavesdropping.

What are the drawbacks of using HTTP streaming?

There are several drawbacks to using HTTP streaming for real-time chat and messaging applications:

Latency: HTTP streaming relies on a continuous connection between the client and the server. This can introduce latency as the server needs to maintain an open connection and send data in chunks. As a result, there might be a delay in delivering real-time messages to the users, which can impact the user experience.

Scalability: HTTP streaming can be resource-intensive for the client and the server. Maintaining a large number of open connections can put a strain on the server and limit its scalability. Additionally, clients need to handle and process incoming data streams, which can also be demanding on their resources.

Compatibility: All devices or networks may not support HTTP streaming. Some firewalls or proxies may block or interfere with the streaming connection, leading to communication issues. This can restrict the availability of the chat application to a subset of users.

Reliability: Since HTTP streaming relies on a long-lived connection, interruptions or network failures can disrupt the streaming process. If the connection is lost, the client may need to reestablish it, potentially resulting in message loss or duplication.

Security: HTTP streaming does not inherently provide encryption or security measures for data transmission. Without additional layers of security, sensitive information exchanged through the chat application may be vulnerable to eavesdropping or unauthorized access.

Battery Consumption: Continuous connections and data streaming can quickly drain a mobile device's battery. This can be a concern for users of real-time chat applications, especially when using these applications for extended periods.

Developers need to consider these drawbacks when choosing a technology for real-time chat and messaging applications. While HTTP streaming provides some benefits, such as leveraging existing security measures, developers should weigh these advantages against the potential drawbacks and assess whether they align with their specific use case and requirements.

What are some alternatives to HTTP Streaming?

Some alternatives to HTTP streaming include:

WebSockets: WebSockets is a communication protocol that provides full-duplex communication channels over a single TCP connection. It allows for real-time, two-way communication between the client and the server, making it suitable for applications that require constant and low-latency data updates.
WebRTC: WebRTC (Web Real-Time Communication) is an open-source project that enables real-time communication between browsers and mobile applications. It provides APIs for voice and video calling and peer-to-peer data sharing, making it a popular choice for video conferencing and live streaming applications.
MQTT (Message Queuing Telemetry Transport): MQTT is a lightweight messaging protocol designed for the Internet of Things (IoT). It is optimized for low-bandwidth and unreliable networks, making it suitable for IoT devices with limited resources. MQTT allows for efficient and real-time communication between IoT devices and backend systems.
RTMP (Real-Time Messaging Protocol): RTMP is a streaming protocol developed by Adobe Systems for delivering audio, video, and data over the internet. It has been widely used for live streaming and video-on-demand applications, but its usage has decreased in recent years due to the rise of HTTP-based streaming protocols.
HLS (HTTP Live Streaming): HLS is an adaptive streaming protocol developed by Apple for delivering media content over the internet. It breaks the content into small, segmented files the client can download and playback in real time. HLS is widely used for streaming live events on-demand video, providing high-quality video playback on different devices and network conditions.
SPDY (pronounced "speedy"): SPDY is a deprecated networking protocol developed by Google to improve the speed and security of web browsing. It aimed to reduce latency and optimize web content delivery by introducing features such as multiplexing, header compression, and prioritization of requests. However, SPDY has been superseded by HTTP/2, which incorporates many of its features.
WebSocket++, Boost.Asio and other libraries: These libraries and frameworks provide low-level APIs and tools for building real-time communication applications using protocols like WebSocket. They offer more flexibility and customization options than higher-level protocols like HTTP streaming but require more development effort and expertise.

It's important to consider your application's specific requirements and constraints when choosing an alternative to HTTP streaming. Scalability, security, compatibility, and developer familiarity should be considered to ensure the best fit for your use case.

What is the difference between TCP and HTTP streaming?

TCP (Transmission Control Protocol) and HTTP (Hypertext Transfer Protocol) streaming are popular protocols for transmitting data over the internet. While TCP is a reliable, connection-oriented protocol, HTTP streaming is a more recent approach for streaming media content. Let's dive into the differences between these two:

Connection-oriented vs. Connectionless:

TCP is a connection-oriented protocol, which means that it establishes a direct, reliable, and persistent connection between the sender and receiver. This ensures that data packets are delivered in the order they were sent and without any loss or duplication. HTTP streaming is based on a connectionless model, where the client sends individual HTTP requests to the server, and the server responds with data chunks, typically in real time.

Data Delivery:

TCP ensures the reliable delivery of data by using various mechanisms like error detection, retransmission of lost packets, and flow control. It guarantees that the receiver gets all the data in the same order it was sent. HTTP streaming focuses on real-time multimedia content delivery, such as audio or video. It prioritizes low latency and responsiveness over guaranteed delivery of every data packet.

Port and Protocol:

TCP uses port numbers to identify applications or services running on a device. The default port for TCP is 80 for HTTP communication. On the other hand, HTTP streaming typically uses higher-level protocols like HTTP Live Streaming (HLS) or Dynamic Adaptive Streaming over HTTP (DASH) for media content delivery. These protocols operate over standard HTTP ports (80 or 443).

Scalability:

TCP is designed for point-to-point communication between two devices. It may face scalability challenges when trying to handle many simultaneous connections. HTTP streaming can leverage load balancing techniques and content delivery networks (CDNs) to distribute the streaming workload across multiple servers, enabling scalability for handling high traffic volumes.

Security:

TCP provides inherent security features such as encryption and authentication. However, additional security measures like SSL/TLS can be implemented on top of TCP to ensure secure communication. HTTP streaming can also use SSL/TLS for secure data transmission. Additionally, content protection technologies like digital rights management (DRM) can be applied to protect multimedia content.

Compatibility:

All major operating systems, networking equipment, and programming languages widely support TCP. It is a foundational protocol for Internet communication. HTTP, as an application-layer protocol, is also widely supported, and HTTP streaming can be implemented on top of existing HTTP infrastructure.

In conclusion, while TCP is a reliable and connection-oriented protocol suitable for general data transmission, HTTP streaming is a recent approach focused on real-time multimedia content delivery. Each protocol has its strengths and weaknesses and is used in different scenarios. Developers building real-time chat and messaging applications can choose between TCP and HTTP streaming based on their specific requirements for data delivery, scalability, security, and compatibility.

HTTP streaming vs. REST

HTTP streaming and REST are two different approaches to data delivery in the context of web applications.

HTTP streaming refers to continuously sending data from a server to a client over an HTTP connection. It enables real-time delivery of multimedia content such as audio or video streams. HTTP streaming typically involves using protocols like HTTP Live Streaming (HLS) or Dynamic Adaptive Streaming over HTTP (DASH) to deliver the content in small chunks. This approach allows the client to play the media as it is being received without waiting for the entire file to download.

On the other hand, REST (Representational State Transfer) is an architectural style for designing networked applications. It is based on principles and constraints that aim to make web services scalable, stateless, and interoperable. RESTful APIs (Application Programming Interfaces) use HTTP as the underlying communication protocol but do not involve real-time data streaming. Instead, REST APIs follow a request-response model, where a client sends a request to a server, and the server responds with a representation of the requested resource.

HTTP Streaming vs. Long Polling

When comparing HTTP streaming and long polling for real-time chat and messaging applications, there are several factors to consider. Both methods have advantages and limitations, so it's important to understand their differences to make an informed decision.

HTTP streaming is a technique where the server pushes data to the client over a single, long-lived connection. This allows for real-time message delivery, as updates can be sent to the client as soon as they are available. HTTP streaming is especially effective for applications that require low latency and high concurrency.

On the other hand, long polling is a technique where the client sends a request to the server, and the server holds the request open until new data is available or a timeout occurs. This approach simulates real-time communication by continuously requesting the server to check for updates. Long polling can be useful for applications that don't require immediate updates and can tolerate slightly higher latency.

One advantage of HTTP streaming over long polling is its ability to handle high concurrency and scale effortlessly. In contrast, long polling may struggle to efficiently handle large numbers of concurrent connections.

Regarding performance, HTTP streaming has the advantage of low latency as updates are pushed to the client as soon as they are available. This real-time delivery of messages ensures that users receive the most up-to-date information without any delay. Conversely, long polling introduces a slight delay as the client needs to request the server to check for updates continuously.

What are the differences between HTTP streaming and other streaming protocols?

HTTP streaming, also known as HTTP-based adaptive streaming, is a streaming protocol that delivers multimedia content over regular HTTP (Hypertext Transfer Protocol) connections. It is different from other streaming protocols, such as RTSP (Real-Time Streaming Protocol) and RTMP (Real-Time Messaging Protocol), in several ways:

Transport Protocol: HTTP streaming uses the HTTP protocol, widely supported by web servers, proxies, and firewalls. In contrast, RTSP and RTMP use transport protocols, which may require special configurations or dedicated infrastructure.

Portability: Since HTTP is a standard protocol, HTTP streaming can be easily accessed by various devices and platforms without needing specific client libraries or plugins. RTSP and RTMP, on the other hand, may require specialized clients or plugins to access the streamed content.

Scalability: HTTP streaming leverages standard web infrastructure, allowing content delivery networks (CDNs) to distribute the streaming content across multiple servers and locations efficiently. This enables better scalability and a wider reach for the streaming application. RTSP and RTMP, however, may require more complex setups to achieve similar scalability.

Adaptive Bitrate Streaming: HTTP streaming supports adaptive bitrate streaming, which dynamically adjusts the video quality based on the viewer's available bandwidth and device capabilities. This ensures a smooth viewing experience, even under varying network conditions. RTSP and RTMP typically do not offer built-in adaptive bitrate streaming.

Firewalls and Proxies: HTTP streaming over HTTP can easily traverse firewalls and proxies since it uses the standard HTTP port (port 80 or 443 for HTTPS), usually open in most network configurations. In contrast, RTSP and RTMP may require specific port configurations or additional network setup to bypass firewalls and proxies.

Overall, HTTP streaming offers advantages in terms of ease of implementation, portability, scalability, adaptive bitrate streaming, and compatibility with firewalls and proxies. These factors make it a preferred choice for developers building real-time chat and messaging applications that require scalability and security.

How do I set up an HTTP streaming server?

Setting up an HTTP streaming server requires a few steps to ensure a seamless and efficient streaming experience. Here's a step-by-step guide to help you get started:

Choose a Streaming Protocol: There are several streaming protocols available, including HTTP Live Streaming (HLS), Dynamic Adaptive Streaming over HTTP (DASH), and Smooth Streaming. Select the protocol that best suits your requirements and target devices.
Install a Web Server: To stream content over HTTP, you'll need a web server. Popular options include Apache HTTP Server, Nginx, and Microsoft IIS. Choose one compatible with your operating system and install it on your server.
Configure the Web Server: Once installed, you must configure your web server to handle streaming requests. This typically involves modifying the server's configuration file. Refer to the documentation specific to your chosen web server for detailed instructions on how to set up streaming.
Prepare Your Content: Before streaming, you must prepare your content for delivery. This involves encoding your video files into the appropriate format, creating playlists, and segmenting the content into smaller chunks. Various encoding tools and software, like FFmpeg or Adobe Media Encoder, can assist you with this process.
Set Up Content Delivery: Consider using a content delivery network (CDN) to deliver your streaming content to users efficiently. A CDN caches your content on servers closer to your audience, reducing latency and improving streaming performance.
Test and Monitor: After setting up your streaming server, it's essential to test and monitor its performance. Conduct thorough testing to ensure your streaming content is delivered smoothly and without glitches. Monitor key metrics such as buffering time, video quality, and network latency to identify potential issues and make necessary adjustments.
Scale as Needed: As your user base grows and the demand for your streaming service increases, you may need to scale your server infrastructure to handle the load. Consider using load balancers, clustering, or cloud-based solutions to ensure scalability and high availability.
Secure Your Streaming Server: To protect your streaming content and ensure the security of your server, implement appropriate security measures. This may include using encryption protocols such as HTTPS, enforcing access controls, and regularly updating your server software to patch any vulnerabilities.

What software is needed for HTTP streaming?

To enable HTTP streaming, several software components are required. The specific software needed depends on the streaming protocol being used. Here are the key components for the two most common HTTP streaming protocols:

HTTP Live Streaming (HLS):

Media Encoder: Software like Apple's macOS-based media encoding tool "Compressor" or FFmpeg can encode video and audio files into the required formats compatible with HLS.
Media Segmenter: This software divides the encoded media into small, manageable chunks called segments. Apple's "mediafilesegmenter" or open-source tools like "Bento4" can perform this task.
Web Server: A web server, such as Apache or Nginx, is needed to serve the media files to clients over HTTP.
CDN (Content Delivery Network): A CDN may distribute the media files across multiple servers geographically, reducing latency and improving scalability.

Dynamic Adaptive Streaming over HTTP (DASH):

Media Encoder: Similar to HLS, a media encoder must encode video and audio files into DASH-compatible formats. FFmpeg is a popular choice here as well.
DASH Packager: This software packages the encoded media into the necessary DASH format (MPD - Media Presentation Description). Open-source tools like MP4Box or commercial solutions like Bitmovin can perform this task.
Web Server: A web server is still needed to serve the media files to clients over HTTP, just like in HLS.
CDN (Content Delivery Network) A CDN can distribute the media files across multiple servers for improved scalability and reduced latency.

In addition to these components, you may need other software or tools depending on your specific requirements. These include media players for clients to play the streamed content, DRM (Digital Rights Management) systems for content protection, and analytics tools for monitoring and analyzing the streaming performance.

When considering the software for HTTP streaming, it's important to choose reliable and well-supported options compatible with your streaming infrastructure and meet your specific needs regarding features, scalability, and security.

What are some APIs you can use for HTTP streaming?

Several APIs are available for HTTP streaming that developers can use to implement video content delivery in their applications. Here are some popular APIs for HTTP streaming:

Media Source Extensions (MSE): MSE is a web API allowing JavaScript to generate playback media streams. It provides a way to dynamically switch between different media sources and adapt to network conditions. MSE is widely supported by modern browsers, making it a popular choice for HTTP streaming.

Video.js: Video.js is an open-source JavaScript library that provides an HTML5 video player with support for HTTP streaming. It abstracts away the underlying video playback technology and provides a consistent API for developers. Video.js supports HLS and other streaming formats like MPEG-DASH and Smooth Streaming.

Dash.js: Dash.js is a reference client implementing the MPEG-DASH standard for HTTP streaming. It is an open-source JavaScript library that provides a feature-rich video player with support for adaptive bitrate streaming, DRM, captions, and more. Dash.js is widely used for MPEG-DASH streaming and offers extensive customization options.

ExoPlayer: ExoPlayer is an open-source media player library for Android that supports HTTP streaming. It provides developers with a flexible and extensible API to build custom media playback experiences. ExoPlayer supports HLS, MPEG-DASH, and Smooth Streaming, among other formats.

AVPlayer: AVPlayer is a framework Apple provides for its iPhone’s iOS and macOS platforms. It supports HTTP streaming, including HLS, and offers advanced features such as adaptive bitrate streaming, subtitles, and offline playback. AVPlayer provides a high-level API for developers to integrate HTTP streaming into their applications easily.

These are just a few examples of APIs available for HTTP streaming. Other APIs may better suit your needs depending on your specific requirements and the platforms you are targeting. It's important to research and evaluate different options to choose the one that aligns with your project goals and provides the necessary features and performance.

What Programming languages can you use for HTTP streaming?

There are several programming languages that you can use for HTTP streaming, depending on your specific needs and preferences. Here are some popular options:

JavaScript: JavaScript is widely used for web development and is the primary language for client-side scripting in web browsers. It is commonly used for implementing HTTP streaming on the front end, utilizing APIs such as Media Source Extensions (MSE) and libraries like Video.js and Dash.js.

Java: Java is a general-purpose programming language popular for building enterprise-level applications. It can be used for HTTP streaming on the server side, with frameworks like ExoPlayer and libraries that support streaming protocols such as HLS, MPEG-DASH, and Smooth Streaming.

Swift: Swift is a programming language developed by Apple for iOS, macOS, watchOS, and tvOS app development. It can be used for HTTP streaming on Apple platforms, leveraging frameworks like AVPlayer that provide advanced features and support for streaming protocols like HLS.

C#: C# is a programming language developed by Microsoft and primarily used to build Windows applications. It can be used for HTTP streaming on the server side, with frameworks and libraries that support streaming protocols such as HLS, MPEG-DASH, and Smooth Streaming.

Python: Python is a versatile programming language known for its simplicity and readability. While it may not be the most common choice for HTTP streaming, libraries and frameworks such as Flask and Django can facilitate streaming implementations.

Ruby: Ruby is a dynamic, object-oriented programming language known for its simplicity and productivity. Although it may not be as commonly used for HTTP streaming, libraries like EventMachine and Celluloid can be used for building streaming applications in Ruby.

Go: Go is a statically typed, compiled programming language designed for simplicity and scalability. It strongly focuses on concurrency and can be a good choice for building high-performance streaming applications. Some libraries like Gin and Revel can be used for HTTP streaming in Go.

PHP: PHP is a server-side scripting language widely used for web development. While it may not be the most popular choice for HTTP streaming, frameworks like Laravel and libraries like ReactPHP can be used to implement streaming functionality in PHP.

Rust: Rust is a systems programming language known for its performance, reliability, and memory safety guarantees. While it may not be the most commonly used language for HTTP streaming, there are libraries like Tokio and Actix that can be leveraged for building streaming applications in Rust.

Kotlin: Kotlin is a statically typed programming language developed by JetBrains and officially supported for Android development. It can be used for HTTP streaming on Android platforms, using libraries like ExoPlayer that support streaming protocols such as HLS and MPEG-DASH.

Which HTTP streaming protocol is more flexible and offers interoperability?

Dynamic Adaptive Streaming over HTTP (DASH) is the HTTP streaming protocol that offers more flexibility and interoperability than other protocols. DASH is not tied to a specific platform or device, making it suitable for various applications and devices. It allows for efficient encoding and adaptive bitrate streaming, ensuring a smooth playback experience across different network conditions.

One of the key advantages of DASH is its interoperability. It is supported by the MPEG industry consortium, which consists of major industry players, including Microsoft, Netflix, and Google. This widespread support ensures that DASH is compatible with various devices, platforms, and browsers. It provides a standardized approach to streaming video content, making it easier for developers to implement and maintain.

In contrast, other streaming protocols like HTTP Live Streaming (HLS) are tied to specific platforms, such as Apple devices and browsers. While HLS is widely supported on Apple devices and browsers, its usage may be limited on other platforms. This can pose challenges for developers who want to ensure their video content reaches a broader audience.

DASH also offers more flexibility in terms of encoding options. It supports various video codecs and formats, allowing developers to choose the most efficient and suitable options for their specific use cases. This flexibility enables developers to optimize video quality and file sizes, enhancing the streaming experience for end-users.

Furthermore, DASH's adaptive bitrate streaming capability allows for seamless playback across different network conditions. It adjusts the quality of the video in real time based on the viewer's internet connection, ensuring a smooth streaming experience even with varying network speeds. This adaptive bitrate streaming feature is crucial for real-time chat and messaging applications, allowing uninterrupted user communication.

Another advantage of DASH is its ability to handle DRM and content protection. DASH supports various DRM systems, such as Microsoft PlayReady and Google Widevine, which are essential for protecting copyrighted content from unauthorized access. This ensures that developers can securely deliver their video content without compromising security.

DASH is the more flexible and interoperable HTTP streaming protocol for developers building real-time chat and messaging applications. Its compatibility with a wide range of devices, platforms, and browsers, along with its support from major industry players, makes it a reliable choice. Additionally, its flexibility in encoding options and adaptive bitrate streaming capability further enhance the streaming experience for end-users.

HTTP Streaming vs. HTTP Live Streaming

When comparing HTTP Streaming and HTTP Live Streaming (HLS), developers need several key factors to build real-time chat and messaging applications.

First, HTTP Streaming is a more flexible and scalable protocol than HLS. While HLS is widely supported on Apple devices and browsers, its usage may be limited on other platforms. This can pose challenges for developers who want to ensure their video content reaches a broader audience. In contrast, HTTP Streaming is compatible with many devices, platforms, and browsers, making it a more reliable choice for developers.

Secondly, HTTP Streaming offers more flexibility in terms of encoding options. It supports various video codecs and formats, allowing developers to choose the most efficient and suitable options for their specific use cases. This flexibility enables developers to optimize video quality and file sizes, enhancing the streaming experience for end-users.

Furthermore, HTTP Streaming's adaptive bitrate streaming capability is crucial for real-time chat and messaging applications. It adjusts the quality of the video in real time based on the viewer's internet connection, ensuring a smooth streaming experience even with varying network speeds. This feature is essential for uninterrupted communication between users.

Another advantage of HTTP Streaming, specifically DASH, is its handling of digital rights management (DRM) and content protection. DASH supports various DRM systems, such as Microsoft PlayReady and Google Widevine, which are essential for protecting copyrighted content from unauthorized access. This ensures that developers can securely deliver video content without compromising copyright protection.

In comparison, HLS has limited support for DRM and content protection. While it does offer basic encryption options, it may not be sufficient for applications that require strong content protection measures. This can be a significant drawback for developers who need to ensure the security and integrity of their video content.

Additionally, HTTP Streaming provides better support for real-time streaming and low-latency applications. It allows for faster delivery of video content, reducing buffering and latency issues. This is particularly important for real-time chat and messaging applications, where a delay in video delivery can impact the user experience and communication effectiveness.

For several reasons, developers should consider using HTTP Streaming over HLS when building real-time chat and messaging applications. HTTP Streaming offers greater flexibility and scalability, compatibility with various devices and platforms, and support for various encoding options. Its adaptive bitrate streaming capability ensures a seamless streaming experience for users, regardless of their network connection. Moreover, HTTP Streaming's robust DRM and content protection features provide the necessary security measures for protecting copyrighted content.

Developers can ensure a reliable, scalable, and secure platform for high-quality video content by choosing HTTP Streaming as the streaming protocol for real-time chat and messaging applications.

What are the use cases for HTTP streaming?

HTTP streaming has many use cases, particularly in real-time chat and messaging applications. Here are some specific use cases for HTTP streaming:

Video Streaming Services: HTTP streaming is commonly used in subscription streaming platforms like Netflix, YouTube, and Amazon Prime Video. It allows for the seamless delivery of high-quality video content to users across different devices and platforms.
Live Streaming: HTTP streaming is widely used for live-streaming events, including sports tournaments, concerts, and conferences. It enables real-time video content broadcasting to a large audience, ensuring smooth playback and minimal buffering.
Video Conferencing: HTTP streaming is essential for video conferencing applications, where real-time communication and collaboration are crucial. It ensures smooth video playback and low-latency streaming, allowing participants clear, uninterrupted video calls.
Gaming: HTTP streaming is increasingly used in cloud gaming platforms, where games are streamed directly to users' devices over the internet. It allows gamers to play high-quality games without needing powerful hardware, as the processing is done on remote servers.
E-Learning: HTTP streaming is utilized in e-learning platforms that offer online courses and educational videos. It enables students to stream educational content seamlessly, regardless of location or device.
Social Media: HTTP streaming is employed by social media platforms like Twitch, LinkedIn, Facebook, Instagram, and Snapchat for live video streaming and sharing. It allows users to broadcast live videos to their followers and facilitates real-time interaction and engagement between users.

How secure is HTTP streaming?

HTTP streaming protocols, such as HLS (HTTP Live Streaming) and DASH (Dynamic Adaptive Streaming over HTTP), provide security for online streaming content. However, it is essential to understand that while HTTP streaming protocols support secure transport mechanisms like HTTPS, overall security depends on various factors and considerations.

Content Protection: HTTP streaming protocols do not inherently provide content protection mechanisms. Developers must implement additional measures such as DRM (Digital Rights Management) systems or encryption techniques to secure the streamed content. These measures ensure unauthorized users cannot access or replicate the streamed content.
Transport Security: HTTP streaming protocols can leverage secure transport mechanisms like HTTPS to encrypt the server and client communication. This encryption helps protect the integrity and confidentiality of the streamed content, preventing unauthorized interception or tampering.
Authentication and Authorization: HTTP streaming protocols can integrate with authentication and authorization systems to ensure only authorized users can access the streaming content. This can involve user authentication, access control policies, and user management systems to regulate access rights.
Server-Side Security: The security of the server infrastructure hosting the streaming content is crucial. Developers need to implement best practices for server security, including regular updates and patches, secure configurations, and monitoring for potential vulnerabilities.
Network Security: While HTTP streaming protocols offer secure transport mechanisms, the overall security of the streaming system can be affected by potential network vulnerabilities. Developers should consider implementing additional security measures such as firewalls, intrusion detection systems, and network segmentation to protect against unauthorized access and attacks.
Data Privacy: HTTP streaming protocols do not inherently provide data privacy features. Developers need to implement appropriate data privacy measures, such as encryption, to protect the personal and sensitive information transmitted during the streaming process.
Compliance: Depending on the content being streamed, developers may need to comply with industry-specific regulations and standards, such as GDPR (General Data Protection Regulation) or HIPAA (Health Insurance Portability and Accountability Act). Compliance with these regulations requires additional security measures and safeguards to protect user data.

What industries can use HTTP streaming?

HTTP streaming can be used in various industries that rely on real-time communication and content delivery. Some industries that can benefit from HTTP streaming include:

Media and Entertainment: HTTP streaming is commonly used for live streaming platforms and online audio content. This includes streaming services, online gaming platforms, and live sports events.
E-commerce: Online retailers can use HTTP streaming to showcase product videos and offer interactive shopping experiences. This enables customers to view product demonstrations and make informed purchasing decisions.
Education and e-learning: Educational institutions and e-learning platforms can leverage HTTP streaming to deliver live lectures, webinars, and on-demand educational content. This allows students and learners to access educational materials in real time or at their convenience.
Healthcare: HTTP streaming can be utilized in telemedicine applications, enabling providers to deliver remote consultations, video conferences, and medical training sessions. It can also be used for streaming patient education videos and medical content.
Banking and Finance: Financial institutions can stream HTTP for real-time financial data feeds, stock market updates, and online trading platforms. This ensures clients have access to up-to-date information and improves overall user experience.
Gaming: The gaming industry heavily relies on HTTP streaming for online multiplayer games, game updates, and streaming platforms for game streaming services. This allows gamers to play and access game content in real time without extensive downloads.
Social Media: Social media platforms often use HTTP streaming for their content creators to deliver live video broadcasts, live events, and real-time notifications to users. This enhances the user experience and allows for immediate interaction and engagement. Facebook Live is one example platform.
Communication and Collaboration: HTTP streaming can be used in communication and collaboration platforms, such as messaging apps, video conferencing tools, and file-sharing services. This enables real-time communication, file streaming, and collaborative workspaces.
Transportation and Logistics: HTTP streaming can be applied to track and monitor vehicles, shipments, and inventory in transportation and logistics industries. This allows for efficient management and optimization of logistics processes.
IoT (Internet of Things): HTTP streaming can be utilized in IoT applications to transmit real-time sensor data, monitor and control signals, and device updates. This enables the integrating of IoT devices and systems for various industries, such as smart home automation, industrial automation, and environmental monitoring.

Overall, HTTP streaming can benefit any industry requiring real-time communication, content delivery, and data transmission. By leveraging HTTP streaming, businesses can enhance their services, improve user experience, and stay competitive in today's digital landscape.

PubNub and HTTP Streaming

PubNub is a leading platform that provides reliable and scalable real-time messaging and streaming services. It utilizes HTTP streaming as one of its core technologies to deliver seamless and efficient communication for developers building real-time chat and messaging applications.

Developers can rely on the platform to handle millions of concurrent connections and deliver messages in real time, regardless of the geographical location of the users, thanks to the 15 Points-of-Presence (PoPs) worldwide...without any concurrency limits.

Security is also a top priority, as the platform offers end-to-end encryption and authentication mechanisms to ensure that messages and data transmitted through HTTP streaming are secure and protected from unauthorized access.

PubNub provides developers a scalable, secure, and feature-rich platform for building real-time chat and messaging applications. By leveraging our infrastructure, SDKs, and extensive library of tutorials, developers can focus on creating innovative and engaging user experiences while PubNub takes care of the underlying complexities of real-time communication.

How can PubNub help you?

This article was originally published on PubNub.com

Our platform helps developers build, deliver, and manage real-time interactivity for web apps, mobile apps, and IoT devices.

Experience PubNub

Check out Live Tour to understand the essential concepts behind every PubNub-powered app in less than 5 minutes

Get Setup

Get Started

The PubNub docs will get you up and running, regardless of your use case or SDK

What is ngrok?

darryncampbell — Fri, 24 Nov 2023 09:16:09 +0000

ngrok is a cross-platform application that enables developers to expose a local development server to the Internet with minimal effort. The software makes your locally-hosted web server appear to be hosted on a subdomain of ngrok.com, meaning that no public IP or domain name on the local machine is needed. Similar functionality can be achieved with Reverse SSH Tunneling, but this requires more setup as well as hosting of your own remote server.

Ngrok pricing

Ngrok has a free pricing tier for developers who are bringing new projects to life. The downside to the free tier is the lack of custom domain support.

How does ngrok work?

ngrok is able to bypass NAT Mapping and firewall restrictions by creating a long-lived TCP tunnel from a randomly generated subdomain on ngrok.com (e.g. 3gf892ks.ngrok.com) to the local machine.

After specifying the port that your web server listens on, the ngrok client program initiates a secure connection to the ngrok server and then anyone can make requests to your local server with the unique ngrok tunnel address. The ngrok developer's guide contains more detailed information on how it works.

What is ngrok tunneling?

Various tunnel servers are available around the world and locations include: US (Ohio), Europe (Frankfurt), Asia (Singapore), and Australia (Sydney). Alternatively, the ngrok server software can be self-hosted on a VPS or dedicated server.

How to use ngrok?

By default, ngrok creates both HTTP and HTTPS endpoints, making it useful for testing integrations with third-party services or APIs that require valid SSL/TLS domains. Other use cases include: quickly showcasing local demos to clients, testing mobile application backends, and running personal cloud services from your home PC.

One praised feature of ngrok is the ability to track and replay HTTP requests via ngrok's web console . The replay functionality is highly useful when testing API calls or webhooks as one can easily inspect all header content and request/response data in one place via the console UI.

A premium version of ngrok, ngrok link, is available for developers to use in production and offers features such as API automation and credential management, making it suitable for remote IoT management in a professional setting.

Ngrok Security

Ngrok is a highly secure platform for remote access and is safe to use. Ngrok provides tunneling, as mentioned above, which enables users to access local-hosted servers from outside the machine.

Do hackers use ngrok?

Ngrok is regularly abused by hackers for cyber attacks. Hackers choose ngrok because of its capabilities and reputation to maneuver while bypassing security protocols. They will see this as a way to complete a phishing attack, as they are able to create fake login pages or deliver malicious URLs.

Resources

How can PubNub help you?

This article was originally published on PubNub.com

Our platform helps developers build, deliver, and manage real-time interactivity for web apps, mobile apps, and IoT devices.

Experience PubNub

Check out Live Tour to understand the essential concepts behind every PubNub-powered app in less than 5 minutes

Get Setup

Get Started

The PubNub docs will get you up and running, regardless of your use case or SDK

How Fast is Real-Time? Human Perception and Technology

darryncampbell — Wed, 22 Nov 2023 23:10:26 +0000

In 1968 Robert Miller published his classic paper Response time in man-computer conversational transactions in which he described three different orders of magnitude of computer mainframe responsiveness:

A response time of 100ms is perceived as instantaneous.
Response times of 1 second or less are fast enough for users to feel they are interacting freely with the information.
Response times greater than 10 seconds completely lose the user’s attention.

From this Miller concluded that a consistent 2-second response would be ideal. Years later this same value of 2 seconds has been used as a performance target for web-based applications. Today’s real-time applications, however, require near-instantaneous responsiveness. Does even 100ms cut it? The answer depends on the context.

How Fast Can a Human Process Input?

As humans beings, we have the curious inborn ability to observe and experience the persistent passage of time. The architecture of our human brains, however, limits our sensory perception in a way that prevents us from reacting to our perceptions within a certain short timeframe. This timeframe is commonly known as Reaction Time.

Human Reaction Time

The average human reaction time is on the order of a quarter of a second (250 milliseconds). Don’t believe it? You can test your own reaction time with this little test.

As you know, some humans have better reaction times than others. Fighter pilots, Formula One drivers, and championship video game players fall into the 100 – 120ms bucket on the left side of the curve.

How much of that time is spent receiving data versus mentally processing and physically reacting?

Real-Time Latency: From Eye to Brain

Reaction time is a complex subject and includes several different components of mental processing including:

Sensory perception
Receipt of input into our consciousness
Context applied to the input
Decisions made based on processing output.

To really understand how fast real-time is to the human brain we’ll focus on the Sensory Perception phase. This is where our senses receive the incoming data from the outside world whether that be visual or auditory.

For example, the time that an image of a tiger arriving on your retina takes to travel down your optic nerve into the visual cortex is incredibly fast. New studies show that humans can interpret visual cues seen for as little as 13 ms (about 1 in 75 frames per second).

As the brain receives the incoming data stream, an asynchronous process acknowledges the input and admits it into our consciousness. Now aware of the incoming data stream, another part of the brain applies context to the stream so that a decision can be made about how to react. All this happens very quickly. (Cats are nearly twice as fast.)

How Does Unwanted Latency Impact Human Performance?

While there is more involved in human reaction time than just mental processing, the important concepts here are:

1. The fastest rate at which humans appear to be able to process incoming visual stimuli is about 13 ms. Receiving a stream of data faster than this will only underscore the limits of our perception.

2. Increasing latency above 13 ms has an increasingly negative impact on human performance for a given task. While imperceptible at first, added latency continues to degrade a human’s processing ability until approaching 75 to 100 ms. Here we become very conscious that input has become too slow and we must rely on adapting to conditions by anticipating input rather than simply reacting to input.

In a duel, for example, a 100 ms lag matters. Especially if it is random and cannot be anticipated.

Implications for Real-Time Application Developers

Real-time applications have varying tolerances to data stream latency. Typically those applications with very demanding targets include:

It is these types of applications in which real-time human perception and interaction are required. Given the resources required to build and maintain a real-time data stream network to support these types of applications, many developers make the strategic decision to outsource the messaging layer in order to focus more on the application itself.

Online Gaming

While turn-based games, role-playing and strategy games typically do not rely on real-time movements or actions and they can tolerate latencies of up to 500ms or more, for Massive Multiplayer Online Gaming (MMOG), real-time is a requirement.

As online gaming matures, players flock to games with more immersive and lifelike experiences. To satisfy this demand, developers now need to produce games with very realistic environments that have very strict data stream latency requirements:

300ms < game is unplayable
150ms < gameplay degraded
100ms < player performance affected
50ms > target performance
13ms > lower detectable limit

A delay of even 100 ms reduces player performance in Twitch games by a measurable amount. It becomes noticeably difficult to track targets effectively and forces players into predicting movements.

Overall game enjoyment continues to decrease as latency increases and players experience jerky playback, ghosting, and out-of-sync behavior that ultimately ruin the game for all players involved.

Given these parameters, to be successful MMOG architecture must consider network performance as a fundamental requirement to ensure Quality of Experience for gamers. This architecture needs to be capable of delivering thousands of simultaneous data streams with latencies as low as 50 ms or better and to make it even more challenging, it must do so at scale for players in different geographic regions, on different access networks, using a range of devices.

“PubNub allows us to focus on our application, rather than the backbone network that supports it and the worries that accompany that. Knowing that we don’t have to setup a whole monitoring system to make sure our backbone network is running and sending messages is amazing; no crashing, no hardware reboots, and no worries,” said James Ross, co-founder and Operations Manager of NodeCraft Hosting.

Bidding & Auctions

With any auction, every split second counts. Success depends on making all bidders feel there’s seamless, reliable, real-time engagement for everyone, wherever in the world they may be. Having the right end-to-end platform for delivering real-time capabilities like real-time notifications, alerts, and updates is an imperative part of the user experience.

A reliable and scalable platform solution is the key to online auction success because:

It gives users the confidence they’re on top of every fresh bid or other development during an auction so they can make their next decisions about bidding (or not)
It maximizes participation among prospective bidders, contributing to higher top bids
It strengthens the credibility of the auction and the organization conducting it
It keeps bidders continuously engaged by delivering the same lively, competitive feeling as a real-world, in-person auction

Real-Time Collaboration

Another interesting example of the use of real-time data is in the area of e-learning and collaboration. In the online classroom, it’s essential to have reliable real-time communication between devices. When dealing with a classroom full of students with short attention spans, devices need to be able to signal between each other as quickly as possible. Otherwise, you’re losing the attention of the entire class. However real-time data stream network design is an outside core competency for most.

“We didn’t know whether we should hire people to do that, whether we’d have to increase the size of the team, and eventually we just sat down and thought, ‘this isn’t what the core of our business is about. We shouldn’t really be spending loads of money and time trying to make real-time work, when we should be focusing on our own business challenges. Real-time was a requirement for our business, but not a business challenge that we should have to solve,” said Liam Don, Co-founder and CTO of ClassDojo.

Live Entertainment

The hallmark of a successful virtual event is if the audience is engaged with not only the performers but also each other. But just because virtual events have been part of our reality for a few years now, doesn’t mean that every event organizer (or platform) provides the necessary features to allow for engaging, interactive events.

A few key features that developers should keep in mind when building apps for virtual events are:

In-app chat with capabilities such as Presence, Reactions, and Typing Indicators
Notifications and alerts to ensure viewers don’t miss a minute of the show—or messages from other participants
Unlimited conversation streams and concurrent users so there’s no need to worry about the event being too successful and crashing
Flexible, real-time profanity filtering and translation to allow participants to communicate civilly, regardless of where they are located.
Interactive polls, quizzes, and leaderboards synced in real time to keep your audience excited and truly engaged

PubNub and Real-Time

Our real-time data API is flexible, so you can roll out these features quickly. Our globally available messaging infrastructure then lets you expand as your app, and needs, evolve. Getting started is easy. And, if you have any questions, we offer support 24/7 worldwide. With PubNub, you can start building real-time...today.

How can PubNub help you?

This article was originally published on PubNub.com

Our platform helps developers build, deliver, and manage real-time interactivity for web apps, mobile apps, and IoT devices.

Experience PubNub

Check out Live Tour to understand the essential concepts behind every PubNub-powered app in less than 5 minutes

Get Setup

Get Started

The PubNub docs will get you up and running, regardless of your use case or SDK

7 Alternatives to REST APIs

darryncampbell — Wed, 22 Nov 2023 00:49:44 +0000

Representational State Transfer (REST) is an architectural style and protocol for building web services. It is a popular approach for designing web application programming interfaces (APIs) because it emphasizes scalability, simplicity, and modifiability.

Unlike the strict frameworks that govern API protocols like Simple Object Access Protocol (SOAP) or Extensible Markup Language remote procedure call (XML-RPC), REST protocols have historically been employed to simplify the process of API development, as they can be built using virtually any programming language and support a variety of data formats.

However, the emergence of several REST alternatives is forming a new flashpoint for API development over the next decade. This trend includes other protocols, patterns, and technologies like event-driven APIs, GraphQL, and gRPC. As these protocols achieve maturity and broader acceptance, it will become essential for API developers to understand how best to deploy REST alternatives across various platforms.

Let's explore what makes these REST alternatives so popular and the most common use cases that support their use in the field.

Why are REST alternatives gaining steam?

RESTful APIs are popular because they are flexible, easy to understand, and compatible with any programming language or platform that supports Hypertext Transfer Protocol (HTTP). They are also well-suited for building scalable and distributed systems, as they can leverage HTTP's caching and load-balancing capabilities.

REST prioritizes easy modification through resources and Uniform Resource Identifiers (URIs) representing data. This means developers can change the API structure without breaking existing client applications.

So why would developers use anything else? There are a few compelling reasons:

Evolving complexity. REST APIs are designed to overcome the complexity of earlier API protocols like SOAP, but they can become challenging to maintain as the number of endpoints and resources increases. This can make it difficult for developers to understand and modify the API over time.
Performance. For some use cases, REST APIs are scalable and can handle many requests. However, there may be better choices for real-time or low-latency applications, as they rely on multiple round-trip requests to retrieve data.
Evolving data requirements. REST APIs may require significant changes to support new use cases or data structures, leading to versioning and compatibility issues and increasing complexity and development time.
Specific use case requirements. There are specific use cases, such as real-time data streaming or low-power Internet of Things (IoT) devices (as mentioned above), where other protocols may be better suited than REST.
Developer preference. Developers may prefer using alternative protocols because they are more familiar with them or offer specific features or benefits that REST does not provide.

Alternatives to REST APIs

Here are the seven REST alternatives you need to know:

GraphQL

GraphQL is a runtime and query language for APIs that allows clients to request and receive only the data they require, making it more efficient than REST. With GraphQL, clients can specify the exact data they need and get it in a single request instead of making multiple requests to different endpoints as in REST. It's a great choice for data-driven applications with complex and evolving data requirements.

It may not be the best choice for applications that require strict data validation, applications that need to support a wide variety of clients, or user-facing apps like social media. Still, it is an excellent alternative to REST in situations requiring flexible and efficient data retrieval and manipulation. This is particularly true in applications with complex data models or mobile applications with limited connectivity.

gRPC

gRPC is an open-source framework developed by Google for building RPC APIs. It allows developers to define service interfaces and generate client and server code in multiple programming languages. gRPC uses protocol buffers and a language-agnostic data serialization format for efficient data transfer, making it ideal for high-performance applications.

gRPC may not be the best choice for a large volume of data manipulation or for applications that need to support a wide variety of clients. However, gRPC is known for its high performance and low overhead, making it a good pick for applications that require fast and efficient communication between services.

WebSockets

The WebSocket protocol enables bi-directional, real-time communication between clients and servers. Unlike REST, which is request/response-based, WebSockets allow servers to push data to clients as soon as it becomes available, making it ideal for applications that require real-time updates, such as chat applications and online gaming.

WebSockets may not be the best choice for applications that require complex data manipulation or for applications where scalability is a concern. However, it shines where real-time communication and low latency are required, thanks to a full-duplex, persistent connection between client and server. REST uses a less efficient request/response model.

MQTT

MQTT is a lightweight, open-source messaging protocol designed for IoT devices. It's a pub/sub protocol with a small packet size and low bandwidth, making it ideal for constrained networks and devices with limited processing power. MQTT can also handle intermittent network connectivity and supports Quality of Service (QoS) levels to ensure reliable message delivery.

It's not the best choice for complex interactions or data manipulation applications. But for use cases with lower bandwidth and battery life preservation — MQTT allows devices to "sleep" between messages, extending the battery life of IoT devices — it offers excellent capability.

Event-Driven Architecture (EDA)

In EDA, events trigger and communicate changes between different components or services within a system. This allows for real-time and reactive data processing and can reduce the need for repeated polling of resources, which can be resource-intensive and time-consuming in REST-based systems.

EDA is a good REST alternative for applications that require real-time data processing, scalability, and loose coupling between different components or services within a system. It's also well-suited for microservices architectures, allowing each microservice to operate independently and communicate with other services through events. This enables better scalability, flexibility, and resilience of the overall system.

FALCOR

Companies can also drive innovation in development. FALCOR is a JavaScript library developed by Netflix for building efficient and flexible APIs. It uses a "path-based" approach to data retrieval, representing data as a graph of interconnected paths rather than individual resources accessed through HTTP requests. It offers benefits like:

WebSocket support, which allows real-time data updates to be pushed to clients without requiring repeated polling
Declarative data fetching, where the client specifies the data it needs and the API responds with the requested data. This simplifies the client-side code and reduces the amount of data sent over the network.

The name "FALCOR" does not stand for anything. It was chosen by the developers at Netflix to represent the library's approach to data retrieval. It's inspired by the character of the same name from the 1980s movie "The NeverEnding Story," a dragon-like creature that can travel through different dimensions and worlds, much like FALCOR's ability to navigate complex data graphs.

Functions

PubNub’s Functions are JavaScript event handlers that can be executed on in-transit PubNub messages or in the request/response style of a RESTful API over HTTPS. For those who are familiar with Node.js and Express, the On Request event handler development is very similar. Functions can deploy new or additional real-time functionality like chat moderation and language translation.

Run your code within our network or leverage our existing integrations to transform, re-route, augment, filter and aggregate messages to detect and block spammers, measure averages, and more. All code is run at the edge for low latency and is robust enough that you don’t need to build your own infrastructure.

Facilitating innovation one REST alternative at a time

REST alternatives are gaining popularity due to their ability to solve challenges REST APIs face. Each alternative has its own unique strengths and use cases. When choosing a REST alternative, it's essential to consider the specific needs and requirements of the application — whether its scale for live events or real-time capabilities for Web3 — as well as the strengths and limitations of each REST alternative.

Ultimately, by exploring these alternatives, developers can build more efficient and effective APIs that meet the needs of modern applications.

Make your app come to life. Contact us to discuss your real-time project or, better yet, see it in action for yourself.

How can PubNub help you?

This article was originally published on PubNub.com

Our platform helps developers build, deliver, and manage real-time interactivity for web apps, mobile apps, and IoT devices.

Experience PubNub

Check out Live Tour to understand the essential concepts behind every PubNub-powered app in less than 5 minutes

Get Setup

Get Started

The PubNub docs will get you up and running, regardless of your use case or SDK

What is Long Polling?

darryncampbell — Tue, 21 Nov 2023 10:51:31 +0000

What is long polling?

Long polling is used in real-time web applications to achieve near-instantaneous communication between the client and the web server. It is particularly useful in chat and messaging applications where real-time updates are crucial.

In traditional HTTP communication, the client sends a new request to the server and waits for a response. This is known as short polling. However, in real-time scenarios, short polling may not be efficient as it requires frequent requests to the server, resulting in unnecessary network overhead and increased latency.

On the other hand, long polling improves efficiency by keeping the request open for an extended period until new data is available. The server holds the request open and waits until it has new information to send back to the client. Once the server has new data, it responds to the client, which can then process the data and initiate a new long polling request.

By maintaining a long-lived connection between the client and the server, long polling reduces the number of requests, minimizes latency, and improves real-time communication. This makes it ideal in use cases that require an effective technique for building scalable and responsive chat and messaging applications as well as other apps that make use of real time data like games.

How does long polling work?

Long polling is a technique used in real-time communication to achieve near-instantaneous message delivery between clients and servers. It is particularly useful in building chat and messaging applications where low latency and real-time updates are crucial.

Traditionally, web browsers use a pull-based approach to fetch data from servers. The client sends a request to the server, which responds with the requested data. This approach, known as short polling, can create delays in communication as the client has to send requests to check for updates repeatedly.

On the other hand, long polling is a push-based approach that allows the server to send updates to the client as soon as they are available. Here's how it works:

The client initiates a request to the server, typically through an HTTP request.
Instead of immediately responding, the server holds the request open, keeping the connection active.
If no new data is available, the server waits until it has something to send back.
Once the server has new data or a predefined timeout occurs, it responds to the client with the latest information.
Upon receiving the response, the client immediately sends another request to the server to maintain the connection.
This cycle of sending requests and receiving responses continues, ensuring real-time updates.

Long polling effectively simulates a real-time connection between the client and the server by keeping the request-response cycle open for an extended period. It allows the server to push updates to the client as soon as they are available and eliminates the need for the client to check for updates repeatedly.

What technologies are used to implement long polling?

Long polling is a technique to achieve real-time communication between a client and server. It is commonly used in chat and messaging applications where instant updates are crucial. Several technologies can be used to implement long polling, each with advantages and considerations. Let's explore some of the common technologies used in implementing long polling.

HTTP long polling:

This is the most basic and widely used approach to implement long polling. It leverages the HTTP protocol to establish and maintain a long-lived connection between the client and server. The client sends a request to the server, and the server holds the request open until new data is available or a certain timeout is reached. Once new data is available, the server responds with the updated information, and the client immediately sends another request to continue the cycle. This approach is easy to implement and requires no special server-side technologies.

WebSocket:

WebSocket is a full-duplex communication protocol that enables real-time communication between the client and server over a single, long-lived connection. It provides a more efficient and low-latency alternative to long polling. WebSocket enables bidirectional data flow, allowing the client and server to send messages asynchronously. It eliminates the need for frequent HTTP requests and reduces network overhead. WebSocket is well-suited for applications requiring instant updates and real-time interaction.

Server-Sent Events (SSE):

SSE is a unidirectional communication technology that allows the server to push data to the client over a single, long-lived HTTP connection. With SSE, the server can send multiple updates to the client without requiring the client to make requests continuously. The server initiates the connection and sends data as a series of events. The client receives these events and can handle them as needed.

When choosing a technology for implementing long polling in your application, there are several factors to consider:

Scalability: Ensure that the chosen technology can handle a large number of concurrent connections and can scale as the user base grows. WebSocket and SSE are generally more scalable than HTTP-based long polling, as they allow for more efficient use of server resources.
Security: Consider the security implications of the chosen technology. WebSocket and SSE can be secured using encryption protocols such as SSL/TLS, ensuring data privacy and integrity. HTTP-based long polling can also be secured but may require additional authentication and access control measures.
Browser support: Check the browser compatibility of the chosen technology. WebSocket and SSE have better browser support than HTTP-based long polling, which may require additional techniques or fallback options for older browsers.
Implementation complexity: Evaluate the chosen technology's ease of implementation and maintenance. HTTP-based long polling is relatively simple, while WebSocket and SSE may require more advanced knowledge and infrastructure. Consider the level of expertise available in your development team and the resources required for implementing and maintaining the chosen technology.

Long polling vs. WebSockets

Long polling and WebSockets are techniques to achieve a real-time connection between a client (such as a web browser) and a server. Although they serve a similar purpose, the two have significant differences.

Long polling is a technique where the client makes a request to the web server, and the server keeps the connection open until it has new data to send back. The server may respond immediately if it has new data available or wait for a specified timeout period before sending an empty response. In either case, once the client receives the response, it immediately makes another request to the server to establish a new connection. This process repeats continuously, allowing the server to push updates to the client as soon as they become available.

On the other hand, WebSockets provide a persistent, bidirectional communication channel between the client and the server. Unlike long polling, where a new connection is established for every request, a WebSocket connection is established once and kept open indefinitely. This allows real-time, low-latency communication in both directions. The server can push data to the client anytime, and the client can also send data to the server without waiting for a response.

Similarities between long polling and Web Sockets:

1. Real-time updates: Both long polling and WebSockets enable real-time communication between the server and client, allowing instant updates without continuous polling or refreshing.

2. Reduced server load: Both techniques minimize unnecessary requests by only sending data when it is available, reducing server load and improving scalability.

3. Wide language and framework support: Many popular programming languages and frameworks support long polling and WebSockets, making them accessible to developers across different ecosystems.

Differences between long polling and Web Sockets:

1. Latency: Long polling introduces latency as there is a delay between the server sending a response and the client receiving it. WebSockets provide bidirectional, low-latency communication, allowing for faster real-time capabilities.

2. Resource consumption: Long polling requires the server to maintain open connections with each client, potentially causing resource consumption and limiting the number of concurrent connections. WebSockets, however, use a persistent connection, reducing the overall resource consumption.

3. Scalability: Long polling's need to keep connections open for an extended period can pose challenges for horizontally scaling the server. With their persistent connection, WebSockets allow for better scalability as they do not require many open connections.

Both long polling and WebSockets offer real-time updates and reduce server load, but they differ in latency, resource consumption, and scalability. Web sockets provide faster, bidirectional communication with reduced resource consumption, making them suitable for applications that require low latency and high scalability. On the other hand, long polling can be a good alternative when low latency is not critical, and the number of concurrent connections is relatively small. Developers should consider these factors when choosing between the two techniques for their real-time chat and messaging applications.

Long polling vs. Server-Sent Events (SSE)

SSE is similar to long polling in terms of its simplicity and ease of implementation, but it provides a more efficient and standardized way of achieving server-to-client communication. Let’s look at some additional similarities and differences between the two technologies.

Similarities between long polling and SSE:

Real-time updates: Both long polling and SSE enable real-time communication between the server and client, allowing instant updates without continuous polling or refreshing.
Reduced server load: Both techniques minimize unnecessary requests by only sending data when it is available, reducing server load and improving scalability.
Wide language and framework support: Many popular programming languages and frameworks support long polling and SSE, making them accessible to developers across different ecosystems.

Differences between long polling and SSE:

Latency: Long polling introduces latency as there is a delay between the server sending a response and the client receiving it. On the other hand, SSE provides a continuous stream of data from the server to the client, reducing latency and improving real-time capabilities.
Resource consumption: Long polling requires the server to maintain open connections with each client, potentially causing resource consumption and limiting the number of concurrent connections. SSE, however, uses a single long-lived connection, reducing the overall resource consumption.
Scalability: Long polling's need to keep connections open for an extended period can pose challenges for horizontally scaling the server. With its single connection per client, SSE allows for better scalability as it does not require many open connections.

How can you optimize long polling?

Long polling is used in real-time chat and messaging applications to provide near-instantaneous client updates. However, it can be resource-intensive and cause scalability issues if not optimized properly. Here are several techniques that can be used to optimize long polling for better performance and scalability.

Batched responses: Instead of sending a response for each request, batch multiple updates together and send them in a single response. This reduces the number of HTTP requests and helps to minimize the overhead.

Compression: Compressing the data before sending it over the network can significantly reduce the payload size, resulting in faster transmission and lower bandwidth consumption. Techniques like Gzip compression can be used to achieve this.

Caching: Implementing a caching layer can help reduce the load on the database or other data sources. By caching the frequently requested data, subsequent requests can be served from the cache itself, reducing the response time and improving scalability.

Connection pooling: Maintaining a pool of reusable connections instead of creating a new connection for every request can improve the efficiency of the long polling mechanism. This eliminates the overhead of establishing a new connection for each request, resulting in better performance.

Throttling and rate limiting: Implementing throttling and rate limiting mechanisms can prevent excessive requests from overwhelming the server. This ensures fair resource allocation and prevents abuse, improving performance and scalability.

Load balancing: Distributing the incoming requests across multiple servers using load balancing techniques can help distribute the load and prevent any single server from becoming overwhelmed. This improves the overall performance and scalability of the long polling system.

Monitoring and optimization: Regularly monitoring the performance of the long polling system and identifying any bottlenecks or areas of improvement can help optimize the system for better performance and scalability. Techniques like profiling, load testing, and performance tuning can be used to identify and address any performance issues.

Asynchronous (Async) processing: Offloading time-consuming tasks to asynchronous processes or background workers can help free up resources and improve the responsiveness of the long polling system. You can get this via message queues, worker processes, or distributed computing.

Connection timeouts: Implementing appropriate connection timeouts can help prevent idle connections from consuming unnecessary resources. By closing idle connections after a certain period of inactivity, the system can free up resources for other clients and improve scalability.

Scalable infrastructure: Ensuring the underlying infrastructure is scalable and can handle the expected load is crucial for optimizing long polling. This may involve using technologies like cloud computing, auto-scaling, or containerization to dynamically allocate resources based on demand.

What Programming Languages are Compatible with Long Polling?

Several programming languages are compatible with implementing long polling in real-time chat and messaging applications. Here are a few examples:

JavaScript: Long polling is commonly combined with JavaScript, allowing for seamless client-side implementation. JavaScript frameworks like React, Angular, and Vue.js provide libraries and tools that simplify implementing long polling in your application.
PHP: PHP is a popular server-side language often used in web development. It provides features and libraries that enable developers to implement long polling efficiently. The PHP framework Laravel, for example, offers support for long polling through its event broadcasting system.
Python: Python is another versatile language that can be used for implementing long polling. Python frameworks like Django and Flask provide the tools and libraries for building real-time applications using long polling techniques.
Ruby: Ruby is a dynamic, object-oriented programming language well-suited for web development. A popular web framework, Ruby on Rails, supports long polling through various libraries and extensions.
Java: Java is a widely used language in enterprise development and provides support for long polling. Java frameworks like Spring and Java EE offer libraries and tools for implementing long polling in real-time applications.
.NET/C#: The .NET framework, with its programming language C#, is commonly used for building web applications. It provides libraries and frameworks like ASP.NET SignalR that simplify the implementation of long polling techniques.

These are just a few examples of programming languages that support long polling. Many other languages and frameworks can also implement long polling in real-time chat and messaging applications.

When choosing a programming language for implementing long polling, there are a few factors to consider. First, consider the specific requirements of your application and choose a language that best fits those requirements. Consider scalability, performance, and ease of implementation on the backend.

Additionally, consider the community and ecosystem surrounding the programming language. A strong and active community can provide support, tutorials, documentation, and resources to make implementing long polling in your application easier.

How is long polling used in real-time applications?

One of the primary advantages of long polling is its efficiency in delivering real-time updates. Minimizing the number of requests sent by clients significantly reduces network latency and improves overall performance. Additionally, it allows servers to push updates to clients immediately, ensuring that messages and notifications are delivered promptly.

Furthermore, long polling facilitates scalability in real-time applications. Reducing the number of open connections enables servers to handle more concurrent clients. This is particularly crucial in chat and messaging applications, where the number of users constantly fluctuates.

Long polling helps save on system resources. With traditional polling, each request requires the server to process and respond, even without updates. This constant processing can strain server resources and negatively impact performance. In contrast, long polling only triggers server processing when new data is available or a timeout occurs. This minimizes the strain on system resources and allows for better scalability and reliability.

However, implementing long polling can also introduce some challenges. One challenge is managing server resources effectively. Since long polling involves keeping connections open for extended periods, handling multiple concurrent connections requires significant server resources. This can be addressed using technologies like cloud computing, auto-scaling, or containerization to dynamically allocate resources based on demand. By automatically scaling resources up or down based on the number of active connections, developers can ensure that the server can handle the expected load effectively.

Another challenge is handling timeouts and connection failures. In long polling, the server holds the request open until new data is available or a timeout occurs. If a timeout occurs, the server must handle it gracefully and close the connection to free up resources. Additionally, if a connection fails, the server should be able to detect it and handle reconnection attempts appropriately. By implementing robust error handling and connection management mechanisms, developers can ensure the reliability of long polling in real-time applications.

Security is another important consideration when implementing long polling in real-time applications. Since long polling involves maintaining persistent connections between clients and servers, it is crucial to secure these connections to protect sensitive data. Implementing secure socket layers (SSL) or transport layer security (TLS) protocols can help encrypt data transmitted over long polling connections and prevent eavesdropping or unauthorized access.

With over 15 points of presence worldwide supporting 800 million monthly active users and 99.999% reliability, you’ll never have to worry about outages, concurrency limits, or any latency issues caused by traffic spikes. PubNub is perfect for any application that requires real-time data.

How can PubNub help you?

This article was originally published on PubNub.com

Our platform helps developers build, deliver, and manage real-time interactivity for web apps, mobile apps, and IoT devices.

Experience PubNub

Check out Live Tour to understand the essential concepts behind every PubNub-powered app in less than 5 minutes

Get Setup

Get Started

The PubNub docs will get you up and running, regardless of your use case or SDK

Announcing the New PubNub Chat SDK

darryncampbell — Fri, 13 Oct 2023 10:20:51 +0000

We are excited to announce the new PubNub Chat SDK, the easiest way for developers to create new chat applications or add in-app chat to existing applications.

We frequently see customers implementing the same chat features over and over again with our traditional SDKs; for example, most chat apps will handle group messaging, show a typing indicator, and have a way of organizing users. Rather than expect everyone to implement these high-level features using our traditional APIs, we thought: “Why not expose a dedicated API specifically for chat features?” The Chat SDK was created to address that need. Best of all, since the Chat SDK is still built on the PubNub platform, you get all the great PubNub features we offer, such as compliance, scalability, and security (through the Access Manager).

Any developer targeting TypeScript can use the Chat SDK in their project. We are framework-agnostic, so the Chat SDK is a great choice regardless of whether you are working with Angular, React, Vue, Ionic, or any other framework that supports TypeScript, both on web and mobile platforms. Additionally, the Chat SDK is UI agnostic, so it gives you the freedom to create unique chat experiences with either your own UI or any pre-built Chat UI available on the market. Why TypeScript? This was in response to customer feedback and where we saw the most demand for what is our first use-case-specific API.

Features of the PubNub Chat SDK

To add chat functionality to your application, the PubNub Chat SDK has been built to support the following features out of the box:

Conversation (Channel) Management: Create direct 1:1 conversations or larger group chats, inviting others to join or leave as needed.
Manage Users: Detect users’ online status, mention them in conversations, and moderate their content.
Basic Messaging: Send and receive messages, emojis, links, and files to any number of recipients.
Advanced Messaging: Forward, Quote, or Pin messages. The SDK also supports read receipts, unread message counts, and a typing indicator to see if other channel participants are typing.
Mobile app support: Easy integration with Android (FCM) and iOS (APNS) push messaging, giving you a consistent approach to offline messages without worrying about platform-specific implementation.

The Chat SDK natively supports concepts such as ‘Users,’ ‘Messages,’ and ‘Channels,’ which are used as follows:

Users

Create a User to represent anybody who will participate in a chat conversation. Users have all the properties you would expect of a chat system, like nickname & avatar, and you can assign custom properties as needed, such as an external ID to match the user with your external system.

// reference the "chat" object and invoke the "createUser()" method
const user = await chat.createUser(
  "support_agent_15",
  {
    name: "John Smith",
    profileUrl: "https://randomuser.me/api/portraits/men/1.jpg",
    custom: {
      title: "VP Marketing",
      linkedInUrl: "https://www.linkedin.com/mkelly_vp"
    }
  }
)

Once created, APIs exist to retrieve, update, delete, or listen for any changes to a user’s properties.

Channels

Having created representations for the system’s users, they need to be added to conversations. The Chat SDK uses the concept of channels to represent a conversation between users, and a conversation can either be direct, between just two individuals, a group conversation amongst multiple users, or a public group without any limits on membership.

Assuming two users have already been created, you can create a group conversation as follows. Note that optional metadata can be added to the channel, such as name and purpose.

// reference both agents you want to talk to
const user1 = await chat.getUser("agent-007")
const user2 = await chat.getUser("agent-008")

// add the channel ID, name, and topic
const { channel, hostMembership, inviteeMembership } = await chat.createGroupConversation(
  {
    users: 
    [
      user1,
      user2
    ],
    channelId: "group-chat-1"
    channelData: {
      name: "Weekly syncs on customer XYZ"
    },
    membershipData: {
      custom: {
        purpose: "premium-support"
      }
    }
  }
)

The users passed in at channel creation only represents the initial state, you can join or leave an existing conversation at any time or other users can invite you to an existing channel. Other APIs also exist to manage both the channel and channel membership.

Messages

With the user now a member of one or more channels, messages can be sent and received over that channel.

Note: You don’t strictly need to be a member of a channel to send and receive messages, but membership is required for more advanced features such as presence and mentions, so it is recommended. See our documentation for more details.

To receive messages, first connect to the channel and specify a callback to be invoked when new messages arrive.

// reference the "channel" object
const channel = await chat.getChannel("support")
// invoke the "connect()" method
channel.connect((message: Message) => {
  console.log(
    "Message received on channel: " + message.content.text
  )
})

Sending messages is very flexible, but in its simplest form, the sendText method is used to send a basic text message:

// reference the channel where you want to send a text message
const channel = await chat.getChannel("support")
// invoke the "sendText()" method on the "channel" object
const message = await channel.sendText(
  "Hi, Everyone!",
  {
    storeInHistory: true,
    meta: {
      messageImportance: "high"
    },
  }
)

Note that the message is (optionally) persisted with storeInHistory and additional metadata can be associated with the message.

You are not limited to sending textual messages, you can send files and links, or react with emojis. Messages can be forwarded, quoted, pinned, and you can even spawn threads or receive read receipts in reply.

Combining Users, Channels, & Messages

Having configured your Users and Channels, as well as each user’s channel memberships, you can take advantage of powerful capabilities offered by the Chat SDK:

User Presence and Online Status: Increase interactivity by tracking which users are active or away with the ‘online status’ capability. See who is available on each channel or globally using the ‘channel presence’ feature.

@Mention Users in Messages: When composing messages, users can see a list of suggested users when they start typing ‘@,’ sourced either globally or from only members of that channel. Since the SDK stores the list of mentioned users as a separate data structure alongside the message, you can use it to send notifications according to your business logic.

History: All messages and files can be optionally stored in persistent storage and assigned a custom time to live (ttl). Message storage is certified compliant, meaning you can build applications that require GDPR, HIPAA, or SOC2 with the Chat SDK. This list is incomplete, so please contact our expert team if you have specific requirements.

Edit and Delete Messages: After messages are sent, they can be optionally edited or deleted as needed. As the developer, you have complete control - you can choose to retain deleted messages in your history or delete them entirely.

Moderation: The Chat SDK allows you to report or flag offensive messages.

Typing Indicator: Dedicated APIs for start and stop typing allow you to track who is typing on any individual channel. The Chat SDK will handle all the timeout logic for you, so you don’t have to worry about keeping track of arrays of actively typing users.

Mobile Push Notifications: PubNub has built-in support for mobile push notifications, so regardless of whether your users are running Android or iOS, you can notify users with the same platform-agnostic Chat API you use to send messages.

Implementation Details

These are the high-level building blocks of the chat SDK. Please check out our docs or the full demo to learn more about the implementation details. Additional resources can also be found at the end of this article.

Selecting an SDK: The Chat SDK or Another PubNub SDK?

Until the release of the PubNub Chat SDK, developers would use one of our traditional SDKs, for example, the JavaScript SDK, and build chat functionality based on the general JavaScript API. This is certainly possible to do, and thousands of developers have already done so, but the process was more complicated than it needed to be for developers specifically developing chat apps.

Let’s take, for example, how to implement an unread message counter in both the JavaScript SDK and the Chat SDK:

An unread message counter using the JavaScript SDK:

For the current user, viewing some specific channel, set the timetoken to indicate the latest message the user viewed.

The recommended way to do this is to use PubNub AppContext to store metadata associated with the user’s membership of the channel, setting lastReadTimetoken as required.
To determine the number of unread messages on any specific channel, call the messageCounts API and pass in both:

The channel name and the last viewed timetoken, as stored in PubNub AppContext and set in the previous step.

This is how our showcase app implements the unread message count when the app first loads; specifically, please see message.js and chat.js.

An unread message counter using the Chat SDK:

As detailed in the Chat SDK documentation for unread messages:

For the current user’s channel membership, specify the last read message using the setLastReadMessage() API, which takes a Message as an argument.

The concepts of ‘channel membership’ and ‘messages’ are inherent parts of the Chat SDK.
Call the getUnreadMessagesCount() API on the current user’s channel membership.

Also, the Chat SDK unread messages documentation features an ‘Under the hood’ section, which explains how the Chat SDK is implemented… You will see that ‘under the hood’ the two implementations are identical. This means neither approach is better from a technical standpoint, but the Chat SDK is easier to understand and more user-friendly.

Getting Started with the PubNub Chat SDK

The PubNub Chat SDK is available from yarn and can be installed as follows:

yarn add @pubnub/chat

Please see our Getting Started walkthrough or video tutorial to start integrating the PubNub Chat SDK with your application.

Resources and Next Steps

The following resources are available for our Chat SDK:

Documentation:

Our documentation page contains a full API reference, a getting started walkthrough, and a primer to explain some of the concepts and terminology used by the SDK.

Demos:

We have several pre-built demos, which can all be found in the js-chat monorepo on GitHub.

The React Native Group Chat Demo is a fully featured showcase for the Chat SDK and should be your first stop to understand the capabilities of the API.

For a simple introduction to the Chat SDK, please see the Getting Started demo, which is referenced by the documentation & video tutorial and uses React.

See the readme file associated with each demo for instructions on how to get started.

Feedback Welcome

We encourage you to download and try out the new SDK for your real-time applications and provide us with valuable feedback. You can contact PubNub support or the developer relations team directly at devrel@pubnub.com. Be on the lookout for additional Chat SDK resources in the future, and happy coding!

Socket Programming in Python: Client, Server, and Peer-to-Peer libraries

darryncampbell — Fri, 24 Feb 2023 11:56:21 +0000

In this tutorial, you'll learn how to exchange data between a client and a server using Python socket programming and the Socket API. Later, this tutorial will discuss exchanging data directly between two or more Python clients using a hosted provider. The source code used in this tutorial can be found within the GitHub repository.

Socket programming connects two sockets (a client socket and a server socket) and allows them to communicate bi-directionally in real-time. Direct socket connections can benefit all real-time applications since data can be sent or received anytime.

Environment Set up

You will need a stable version of Python version 3.x installed on your machine. If you are a Windows user, you have the option of adding Python to your PATH.

You will also need a code editor to follow along with this tutorial. Visual Studio Code is a popular open-source and free code editor that supports many languages and frameworks, including Python. VSCode also supports extensions for Python to help with code completion and debugging.

Build and run a Python socket application

Let’s build a straightforward socket application using Python. Python provides a native socket class (socket module), so developers don’t need to depend on external libraries. Begin by setting up the Python socket client and server:

Create the file client.py in the project directory. To use sockets, import the Python socket library and create a new socket object that connects to a specified IP address (in this case, localhost on port number 8080, but you can select any ipv4 address). Create a new connection to the socket server, send data to the TCP server, and close the socket connection.

Your client.py file should look like this:

import socket
client = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
client.connect(('0.0.0.0', 8080))
client.send("I am CLIENT\n".encode())
from_server = client.recv(4096)
client.close()
print (from_server.decode())

You will need a socket server to listen for incoming connections and messages from your client. Create the file server.py and add the following contents:

import socket
serv = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
serv.bind(('0.0.0.0', 8080))
serv.listen(5)
while True:
  conn, addr = serv.accept()
  from_client = ''
  while True:
    data = conn.recv(4096)
    if not data: break
    from_client += data.decode('utf8')
    print (from_client)
    conn.send("I am SERVER\n".encode())
  conn.close()
  print ('client disconnected and shutdown')

Server.py binds the socket object to the hostname (localhost) on port 8080 and continually listens for new client connections. When a client connects to this address, the server accepts the connection and reads any data. Once the data is successfully read from the client, the server provides a data response, at which point the client terminates the connection.

To test this out yourself, open two terminal windows simultaneously. In one window, run:

python3 server.py

In the second window, run:

python3 client.py

Notice that the server continues running and will establish a new connection every time you run the client and append any new output.

The client will send the "I am CLIENT" string to the server and wait for a reply. The server will read the client’s message, output it to the terminal, and send back a response to the client.

Socket Programming in Python using PubNub

So far, this tutorial has covered exchanging messages between a server and a client, but what if you need to communicate directly between Python clients?

Sending data directly between two or more client devices is tricky because you run into many scaling and security considerations as your number of devices increases. A client-server architecture is used to moderate and manage your client-to-client communication. If you don’t have a web server or you worry about your server scaling to meet your application’s demands, you should opt for a hosted, real-time communication solution such as PubNub. PubNub is a globally distributed and scalable cloud platform, so you don’t have to worry about deploying and maintaining servers. PubNub’s cross-platform SDKs, including Python, can identify users and send messages to specific channels, which only subscribed clients will receive.

Client-to-Client Python socket programming

So, how would the simple app presented previously be written with PubNub to exchange messages directly between two clients? It’s essential to understand that although PubNub uses the ‘publish’ and ‘subscribe’ architecture (pub/sub) to send and receive bidirectional messages between endpoints, it still uses sockets behind the scenes. PubNub gives you the benefits of socket communication without worrying about the details of Python network programming and maintaining an always-on connection between your clients regardless of the operating system.

To integrate PubNub into the project, install the PubNub package with pip in the terminal; this will allow you to use the PubNub Python SDK and communicate with the PubNub infrastructure.

pip3 install 'pubnub>=7.1.0'

You will need to create two clients to connect to and communicate over the PubNub network. Create a file pn_client_1.py and add the following code:

from pubnub.callbacks import SubscribeCallback
from pubnub.enums import PNStatusCategory
from pubnub.pnconfiguration import PNConfiguration
from pubnub.pubnub import PubNub
import time
import os
pnconfig = PNConfiguration()
userId = os.path.basename(__file__)
pnconfig.publish_key = 'demo'
pnconfig.subscribe_key = 'demo'
pnconfig.user_id = userId
pnconfig.ssl = True
pubnub = PubNub(pnconfig)
def my_publish_callback(envelope, status):
  # Check whether request successfully completed or not
  if not status.is_error():
    pass
class MySubscribeCallback(SubscribeCallback):
  def presence(self, pubnub, presence):
    pass
  def status(self, pubnub, status):
    pass
  def message(self, pubnub, message):
    if message.publisher == userId : return
    print ("from device " + message.publisher + ": " + message.message)
pubnub.add_listener(MySubscribeCallback())
pubnub.subscribe().channels("chan-1").execute()
## publish a message
while True:
  msg = input("")
  if msg == 'exit': os._exit(1)
  pubnub.publish().channel("chan-1").message(str(msg)).pn_async(my_publish_callback)

Create the file pn_client_2.py and add the same code as you used for pn_client_1.py

The code above uses 'demo' keys, but you can obtain your custom PubNub keys for free.

Run both pn_client_1.py and pn_client_2.py simultaneously in two different terminal windows

python3 pn_client_1.py

python3 pn_client_2.py

Each client initializes its connection to the PubNub network and subscribes to receive new messages whenever they are published to the chan-1 channel. You can think of this as sending data over a TCP socket in Python; behind the scenes, PubNub is creating and managing the socket for you and routing your message to all clients who are listening for it. Once the remote client receives the message, the received message is displayed on the command line.

And that’s all there is to it! For more information on developing with PubNub, check out their range of tutorials and demos. Alternatively, check out the PubNub interactive live tour to understand how the platform provides real-time interactivity to applications. PubNub supports TCP and UDP datagram communication, as well as Linux, Unix, and Windows.

WebSocket Programming Examples with and without Node.js

darryncampbell — Fri, 24 Feb 2023 11:56:14 +0000

If you are reading this, you are probably interested in creating an application or service that needs two-way communication that either side can initiate. Node JavaScript (Node.js) can be used to quickly develop and deploy this application using WebSockets, which were designed with this use case in mind.

What is a WebSocket?

A WebSocket is a computer communications protocol providing duplex communication channels over a single TCP connection. In other words, it allows internet-capable devices to communicate with each other, one acting as the client and the other acting as the server, with both able to initiate communication.

What types of applications use WebSockets?

The WebSocket protocol is used wherever a solution requires real-time communication and example use cases include:

WebSockets create a TCP socket connection between multiple devices or processes. As will be discussed later, similar functionality can be implemented using HTTP Long Polling or a hosted pub/sub service, but let’s build a simple example first using WebSockets and Node.js.

Node.js WebSocket API Example - a basic chat application

All source code associated with this blog is hosted on GitHub

To build a basic chat application with WebSockets, you will need both a client and server component.

For the server, we will use Node.js, and the client-side code will run within a web browser such as Chrome.

Node.js WebSockets Code (Server)

This application will require both a web server (HTTP server) and a WebSocket server (wss). The web server allows us to load our client (running in a browser), and the WebSocket server handles the bidirectional chat messages.

Create the Node.js app and install both the Express.js and ‘ws’ packages which will provide our web server and WebSocket server, respectively.



npm init
**Follow the prompts and accept the defaults**
npm install --save express
npm install --save ws

The web server portion will serve a single web page to the client, websocket-client.html, on port 3000:



const express = require('express')
const webserver = express()
 .use((req, res) =>
   res.sendFile('/websocket-client.html', { root: __dirname })
 )
 .listen(3000, () => console.log(`Listening on ${3000}`))

A WebSocket server can be created in only a few lines using the Node.js WebSocket library (ws) package:



const { WebSocketServer } = require('ws')
const sockserver = new WebSocketServer({ port: 443 })

After creating an instance of a WebSocket server and specifying the port to run on, we can define any action that should happen after the WebSocket connection is established. In our case, we write connection events to the console and forward any messages we receive to previously connected clients. WebSockets are designed to run on the same ports as HTTP/HTTPS (i.e., 80 and 443).



sockserver.on('connection', ws => {
 console.log('New client connected!')
 ws.send('connection established')
 ws.on('close', () => console.log('Client has disconnected!'))
 ws.on('message', data => {
   sockserver.clients.forEach(client => {
     console.log(`distributing message: ${data}`)
     client.send(`${data}`)
   })
 })
 ws.onerror = function () {
   console.log('websocket error')
 }
})

And with that, the server portion is complete. Every time a message is received on any socket, it proxies the message to every connected client, which is the basis of any group-chat app.

The full index.js code looks like this:



const express = require('express')
const webserver = express()
 .use((req, res) =>
   res.sendFile('/websocket-client.html', { root: __dirname })
 )
 .listen(3000, () => console.log(`Listening on ${3000}`))

const { WebSocketServer } = require('ws')
const sockserver = new WebSocketServer({ port: 443 })
sockserver.on('connection', ws => {
 console.log('New client connected!')
 ws.send('connection established')
 ws.on('close', () => console.log('Client has disconnected!'))
 ws.on('message', data => {
   sockserver.clients.forEach(client => {
     console.log(`distributing message: ${data}`)
     client.send(`${data}`)
   })
 })
 ws.onerror = function () {
   console.log('websocket error')
 }
})

And you can run it from the command line as follows:



node index.js

Messages can either be text or JSON encoded (JSON.stringify). This code uses the utf-8 charset by default.

WebSockets Code (Client / Browser)

We don’t have to install additional software or packages to use WebSockets with modern web browsers. Create a listening WebSocket by providing a correctly formatted URI:



const webSocket = new WebSocket('ws://localhost:443/');

And defining an event handler for when a message is received from the server:



webSocket.onmessage = (event) => {
  console.log(event)
  document.getElementById('messages').innerHTML += 
    'Message from server: ' + event.data + "<br>";
};

That’s it. We can now receive data from the WebSocket server.
To send a message event from the client, you send() on the socket object:



webSocket.send('hello')

To make our chat app functional, we just need to add an input field and a ‘send message’ button, so your frontend HTML code should look something like this (styling & css omitted for brevity):



<html>
<body>
  <form id="input-form">
    <label for="message">Enter Message:</label>
    <input type="text" id="message" name="message"><br><br>
    <input type="submit" value="Send">
  </form>
  <div id="messages"></div>

  <script>
    const webSocket = new WebSocket('ws://localhost:443/');
    webSocket.onmessage = (event) => {
      console.log(event)
      document.getElementById('messages').innerHTML +=
        'Message from server: ' + event.data + "<br>";
    };
    webSocket.addEventListener("open", () => {
      console.log("We are connected");
    });
    function sendMessage(event) {
      var inputMessage = document.getElementById('message')
      webSocket.send(inputMessage.value)
      inputMessage.value = ""
      event.preventDefault();
    }
    document.getElementById('input-form').addEventListener('submit', sendMessage);
  </script>
</body>
</html>

Make sure you name your HTML page websocket-client.html and launch a few tabs in your browser, navigating to http://localhost:3000. You should see messages received in every tab:

Creating the same chat app without a Node.js server

If you don’t have a server or have concerns about scaling your server infrastructure to meet your application’s future demands, you should opt for a hosted real-time communication solution such as Socket.io or PubNub.

PubNub is a globally distributed and scalable cloud platform, so you do not have to worry about deploying and maintaining servers on your backend. PubNub SDKs can identify users and send messages to specific channels, which only subscribed users will receive.

So, how would the simple chat app presented above be written with PubNub? One benefit of PubNub is that it is protocol agnostic. PubNub uses the ‘publish’ and ‘subscribe’ architecture (pub/sub) to send and receive bidirectional messages.

First, include the PubNub JavaScript SDK in the header, available from their CDN:



<script src="https://cdn.pubnub.com/sdk/javascript/pubnub.7.2.2.min.js"></script>

To subscribe for incoming messages, create a PubNub instance, define the channel, and add a listener:



var pubnub = new PubNub({
  publishKey: 'demo',
  subscribeKey: 'demo',
  userId: "" + Math.random()
})
pubnub.subscribe({
  channels: ['ws-channel']
})
pubnub.addListener({
  message: payload => {
    console.log()
    document.getElementById('messages').innerHTML += 
      'Message from client: ' + payload.message + "<br>";
  }
})

The page will be updated whenever messages are received on the ws-channel. The code above uses 'demo' keys, but you can obtain your custom PubNub keys for free.

To send messages over PubNub, publish on the same channel you subscribed to previously, in this case:



pubnub.publish({
  channel: 'ws-channel',
  message: inputMessage.value
})

The channel model is very flexible and extensible, but for this simple example, it is sufficient to just send and receive messages on the same channel. Your message will be delivered anywhere in the world in under 100ms.

Since there is no server component required with a PubNub deployment, the entire application, written in HTML and JS, is contained within a single file:



<html>
<head>
  <script src="https://cdn.pubnub.com/sdk/javascript/pubnub.7.2.2.min.js"></script>
</head>
<body>
  <H1>This code uses PubNub, which is protocol agnostic</H1>
  <form id="input-form">
    <label for="message">Enter Message:</label>
    <input type="text" id="message" name="message"><br><br>
    <input type="submit" value="Send">
  </form>
  <div id="messages"></div>

  <script>
  var pubnub = new PubNub({
    publishKey: 'demo',
    subscribeKey: 'demo',
    userId: "" + Math.random()
  )
  pubnub.subscribe({
    channels: ['ws-channel']
  })
  pubnub.addListener({
    message: payload => {
      console.log()
      document.getElementById('messages').innerHTML += 
        'Message from client: ' + payload.message + "<br>";
    }
  })
function sendMessage(event) {
  var inputMessage = document.getElementById('message')
  pubnub.publish({
    channel: 'ws-channel',
    message: inputMessage.value
  })
  inputMessage.value = ""
  event.preventDefault();
}
document.getElementById('input-form').addEventListener('submit', sendMessage);
  </script>
</body>
</html>

You can see a live version of this code at
https://pubnubdevelopers.github.io/nodejs-websocket-examples/pubnub-client.html