Forem: CFE.dev

What Developers Need to Know About JWTs

Brian Rinaldi — Tue, 16 May 2023 11:56:30 +0000

Let's delve into JSON Web Tokens, their history, and the problems they solve. We'll discuss the importance of thorough validation, as some developers tend to overlook this aspect. Additionally, we'll cover Bearer Tokens, their issues, and possible remedies. Common mistakes made with JSON Web Tokens will also be addressed, followed by an exploration of Refresh Tokens – a related, yet distinct, concept. Lastly, we'll touch upon JSON Web Token revocation. So, let's dive in.

This article is based upon a presentation from Dan Moore of FusionAuth.

What are JWTs?

JSON Web Token, or JWT (pronounced "jawt" as per the specifications), is a widely recognized standard in the tech industry.

JSON Web Tokens (JWT) belong to a family of specifications around RFC 7519, which include RFC 7516, 7517, 7518, 7520, and others. These standards, developed by the IETF, focus on various aspects of tokens, signing, and encryption. So, when discussing JWTs, it's essential to remember that they are part of a broader set of related standards.

JSON Web Tokens (JWTs) first appeared around 2015-2017 and can be either signed or encrypted. A signed JWT ensures the integrity of the token, meaning you can confirm it hasn't been altered, but its contents are not secret. It's crucial not to include sensitive information like social security numbers in signed JWTs. On the other hand, encrypted JWTs keep the payload secret, as the name suggests. Understanding the difference between signed and encrypted JWTs is essential when working with these tokens.

While signed JWTs are more common, it's important to note that their contents are not secret, and sensitive information should never be included in them. If a signed JWT contains a secret, you should either convert it to an encrypted JWT or find a way to exclude the secret, as anyone who obtains the JWT can view that secret. Encrypted JWTs, on the other hand, keep the payload secret, ensuring the information remains confidential.

JSON Web Tokens (JWTs) are frequently used as stateless, portable tokens of identity. They are considered stateless because their authenticity and integrity can be verified without needing to communicate with a server to confirm their validity. This feature allows them to function independently, without constant server-side validation. JWTs are also portable due to their compatibility with HTTP, easily fitting into form parameters and headers.

JSON Web Tokens (JWTs) can be stored in cookies and are often used as tokens of identity at the end of an authentication event, representing the user. For example, this is how FusionAuth utilizes them. The holder of the token is usually the person who authenticated, although there are some caveats to this. JWTs are particularly useful for APIs and microservices, as you can generate a JWT in one location, pass it to a client, and then have the client present it to various APIs or microservices to access data or functionality on behalf of that user. This makes JWTs ideal for distributed systems where stateless, portable identity representation is essential.

JSON Web Tokens (JWTs) are widely used by various identity providers, including FusionAuth, Auth0, Firebase, and Cognito. These providers produce JWTs to grant access to specific functionalities for users holding the tokens. APIs and microservices can validate JWTs, ensuring that the access is provided to the intended users. This widespread support and adoption of JWTs across the industry reinforces their importance as a standard in stateless, portable identity representation.

Leveraging JSON Web Tokens in a Complex To-Do Application

JSON Web Tokens (JWTs) have gained widespread popularity, not only because they are supported by various identity providers, but also due to extensive documentation, library support, and general understanding within the tech industry. JWTs have been around for a while, and their primary purpose is to address the critical issue of distributed identity, making them an invaluable tool in modern applications.

In our example, we have a typical, slightly over-engineered to-do application. The architecture consists of clients on the left side, which could be browsers, native clients, or other types of clients. These clients communicate with various APIs, which in turn interact with a data store. The data store for the user API has a standard structure, containing users, roles, and a mapping between them. Meanwhile, the to-do API data store consists of a single table for to-dos. This table has some interesting features, demonstrating the complexity that can arise even in a simple to-do application.

The data store for the user API houses users, roles, and their mappings, while the to-do API data store contains a single table for to-dos with fields like text, status, and user ID. Notably, even in a simple to-do application, complexity can arise due to the distributed nature of the system. For instance, the user ID field cannot be a foreign key, as cross-database references are not supported. Although the user ID is always present and set to not null, it cannot reference the user table directly.

CREATE TABLE todos {
  id INT NOT NULL,
  text TEXT NOT NULL,
  status INT NOT NULL,
  user_id CHAR(40) NOT NULL,
  PRIMARY KEY (id)
);

In the over-engineered to-do application, accessing to-dos requires an authentication process. First, the user provides their credentials, such as a username and password, by posting the information to the user API. This authentication step is essential for ensuring secure access to the to-dos and maintaining the integrity of the system.

Securing User Data and To-dos: Opaque Tokens, Validation, and Architectural Considerations

In the over-engineered to-do application, linking the to-dos back to the user after the login event is crucial. During the login process, the user API communicates with the user database and retrieves the user's data. This user data plays a vital role in connecting the to-dos to the user, ensuring secure access and maintaining the integrity of the system.

In the case of JSON Web Tokens for developers, the details of the authentication process are not crucial. What matters is that at the end of the process, the developer receives a user object, usually in JSON format for modern web applications. The user object typically contains information such as an identifier, name, roles, and email, among other details. This user data is passed to the client and plays a vital role in connecting the user with their to-dos, ensuring secure access, and maintaining the integrity of the system.

{
  "user": {
    "id": "42",
    "name": "Dan Moore",
    "email": "dan@fusionauth.io",
    "roles": ["admin"]
  }
}

In the over-engineered to-do application, when the client wants to get the to-dos for user ID 42, they pass that identifier to the to-do API. The API then queries the to-dos database and finds that Dan has 25 different to-dos. It wraps them up as JSON and sends them back to the client, which then renders them. However, this approach is not ideal because you cannot trust the client to maintain the integrity of the system.

However, a potential security issue arises from the client directly requesting to-dos using the user ID. Attackers could inspect the client code or network traffic to identify that user ID 42 is being used, giving them insight into the number of users in the system and allowing them to iterate through user IDs to access other users' to-dos. To prevent this, the to-do API must not accept user IDs blindly and needs a mechanism to protect user privacy. One potential solution is using an opaque token approach.

In this scenario, the user API generates a long random string, known as an opaque token, along with the user data. The client is coded to present this token to the to-do API, rather than the user ID. While this approach addresses the issue of revealing the number of users in the system, it still leaves room for enumeration attacks. Attackers could potentially rent multiple EC2 instances on AWS and iterate through every possible token value to scrape all the to-dos. To prevent this, the to-do API cannot use the opaque token as an identifier in the to-dos database. Instead, it must present this token to the user API for validation.

The user API validates the opaque token by checking its database, as it generated the token initially. If the token is valid, the user API returns the user ID or the entire user object, allowing the to-do API to retrieve the to-dos. If the token is not valid, the user API sends an error code, indicating that an issue occurred, such as a bug or an attempt at privilege escalation, and the to-do API should not return the to-dos. This approach is valid, but like any engineering issue, there are trade-offs to consider.

Maintaining a user token mapping at the user API can range from trivial to a significant challenge, depending on the number of users. This approach can also be quite chatty, as the to-do API needs to present the token to the user API every time it receives a request, unless caching is implemented, which introduces additional complexities. Architecturally, this method heavily couples the user API to almost everything. Whenever requests come in from any user-related APIs, the token must be extracted and presented to the user API. Although this might work for systems with only a few APIs, it may become problematic as the number of APIs increases.

With a growing number of APIs, scaling the user API becomes increasingly necessary. This flow is similar to OAuth introspection, an IETF-approved RFC for determining the validity of client requests. It's worth noting that this process is not exactly OAuth introspection, but shares similarities with it. Understanding these patterns and trade-offs is crucial for managing complex systems and ensuring secure access to user information.

Leveraging JWTs for Simplified Authentication in Complex Systems

In the over-engineered to-do application, JWTs offer an alternative solution to the opaque token approach. The user API generates a JWT, which is then passed to the client. The client can extract the JWT and present it to the to-do API, allowing the to-do API to verify the signer, validity, recipient, and other attributes without needing to communicate with the user API. This approach simplifies the authentication process and reduces the coupling between the user API and other APIs, making it easier to manage in complex systems.

A JWT can be signed using a public-private key pair or a symmetric key, which will be discussed in more detail later. The main advantage is that the to-do API doesn't need to communicate with the user API, as the token's signature guarantees its correctness. Additionally, a JWT can hold various information, such as roles, subscription statuses, or plan details. However, it's important to note that the content of a JWT is visible to anyone who obtains it, so sensitive information should be avoided. Now, let's examine what a signed JSON Web Token looks like.

In the over-engineered to-do application, a signed JSON Web Token (JWT) consists of three parts: the header (green), the payload or body (blue), and the signature (white or tan). Both the header and the payload are Base64 URL encoded strings.

That's why you'll often see JWTs starting with "EYJ0", as it represents the URL-encoded string for "{" (curly brace). If you want to decode the Base64 URL encoded strings in a JWT, you can simply search for a Base64 decoder online, and it will work most of the time.

Understanding JWT Structure and Validation: Headers, Claims and Signatures

In a signed JSON Web Token (JWT), the header contains metadata, such as the algorithm used to sign it, the type of JWT, and sometimes a key identifier to determine which key was used for signing. The body, where things get interesting, can store any information as it is a JSON object. The keys of the JSON object are called claims, which provide details about the user. Some claims, like issuer, expiration, audience, and subject, are standardized, while others are not.

In a JWT, you can store a variety of data types, such as arrays, arrays of arrays, or arrays of objects of arrays. There's no specific limit on the JWT length, but the transport layer might impose a limit. For example, if you pass a JWT as a header, some browsers might have a maximum header length, which would effectively limit the JWT size. The flexibility of JWTs allows for the inclusion of non-standardized information, like names and roles, to be helpful to downstream JWT consumers like the to-do API.

In a JWT, the signature is an essential component, ensuring the integrity and authenticity of the token. Although you will most likely use a library to generate the signature, understanding its function is valuable. The process involves performing a cryptographic operation on the header and payload, followed by Base64 URL encoding the result. When passing a JWT as a cookie, you may be limited by the cookie size, requiring you to break it apart and reassemble it if necessary. Overall, the signature plays a crucial role in maintaining the security of JWTs.

The type of cryptographic operation performed on a JWT depends on the chosen algorithm, which can be symmetric or asymmetric. Libraries are typically used to handle these operations, but it's essential to understand that the signature is tied to the header and the body. If any alterations are made to the header or body, such as changing a role, the signature will no longer match. This leads to the process of JSON Web Token validation, which involves the necessary steps developers need to take when receiving a JWT to ensure its integrity and authenticity.

Securing User Data with JWT Validation

APIs like the to-do API receive JSON Web Tokens (JWTs) from the client to protect user data, such as personal to-dos. To ensure that only authorized users can access their to-dos, it is crucial to validate the JWT. The first step in this process is validating the signature. It is recommended to use a library for this task, as most programming languages have a dedicated JWT library available. By effectively validating the JWT, you can maintain the security and integrity of user data within the to-do application.

When working with JSON Web Tokens (JWTs), it's important to use a trustworthy library, preferably an open-source one, for validating the tokens. When using a library to check the JWT, if it returns an exception or error indicating that the signature doesn't match, the process should be stopped immediately. This error could result from a bug, privilege escalation, or other issues, but it signifies that the JWT is not valid for further processing.

After validating the signature, it's essential to validate the claims within the JWT. This is a step that some developers may overlook, but it is crucial for maintaining the security and integrity of user data within applications like the to-do API.

After checking the JWT signature, you must validate the values of the JSON object keys, which can be considered business logic. While some libraries can help with certain claims, you'll need to handle custom claims like checking for a paid user. This means you need to write the logic for verifying these custom attributes. Although you have to write this logic, you can create a custom library or module to avoid writing it for each individual API. This will ensure the security and integrity of user data within applications like the to-do API.

In a JWT, both standardized and custom claims exist. For example, the issuer claim verifies that the JWT was created by the expected entity, like the user API in a to-do application. It's essential to check the issuer to ensure the JWT is generated by the correct party. Another claim to validate is the expiration time, which is the number of seconds since the Unix epoch. If the expiration time is in the future, the JWT is most likely valid. Validating these claims helps maintain the security and integrity of the user data within applications.

In a JWT, it's crucial to validate claims like expiration time and audience. If the expiration time is in the past, the JWT is invalid, and processing should stop, just as if the signature validation failed. The audience claim indicates the intended recipient of the JWT. In the given scenario, the to-do API is the intended recipient. If the audience claim doesn't match the expected value, processing should be halted. This is important because there could be multiple APIs with different user roles, and validating the audience ensures that the JWT is being used for the correct API.

{
  "iss": "fusionauth.io",
  "exp": 1619555018,
  "aud": "238d4793-70de-4183-9707-48ed8ecd19d9",
  "sub": "19016b73-3ffa-4b26-80d8-aa9287738677",
  "name": "Dan Moore",
  "roles": ["RETRIEVE_TODOS", "ADMIN"]
}

When dealing with JWTs in applications, it is essential to ensure the proper access level is granted, depending on the user's role in various APIs. For example, an admin in the todo API might have access to view other users' to-dos, while an admin in the accounting API may have the ability to send out checks. It is crucial that a JWT created for the to-do API is not presented to the accounting API, as it could lead to unauthorized access. To handle this, you can implement validation code which outlines a simple validation scenario to maintain the security and integrity of user data across different APIs.

// the todo api
const options = {
  algorithms: "HS256",
  ignoreExpiration: false,
  issuer: "fusionauth.io"
}

const verified = jwt.verify(token, hmac_key, options);

// addl verification checks
if (verified.aud != 'myapp.example.com') {
  throw "invalid audience";
}

if (!verified.premiumUser) {
  throw "invalid access";
}

In the given example, if the HMAC key is changed, it demonstrates what happens when the signatures of JWTs don't match. By creating a new HMAC key (const new HMAC key), the signature will be different, as the time has changed. This mismatch in signatures helps illustrate the importance of validating JWTs to ensure the security and integrity of user data within applications like the to-do API. When signatures don't match, it's crucial to halt further processing and investigate potential issues.

Securing User Data: The Importance of Bearer Token Security

We've examined the architectural benefits of using JSON Web Tokens (JWTs) and learned about their structure. We've also discussed the importance of validating JWTs to ensure the security and integrity of user data in applications like the to-do API. Now, let's shift our focus to bearer tokens in general, which are closely related to JWTs. Bearer tokens are authentication tokens that grant access to resources on behalf of the bearer or holder, making them a convenient method for authorizing users in various scenarios.

Bearer tokens are a broader concept than JSON Web Tokens (JWTs), as JWTs are often used as bearer tokens, but bearer tokens can exist independently of JWTs. The main idea behind a bearer token is that possession of the token serves as proof of access. A helpful analogy to understand this concept is a car key – possessing the key grants access to the car.

If you have a simple car key without a biometric sensor, anyone holding the key can access and drive the car. Therefore, you must be cautious about the key's location, as leaving it unattended could allow unauthorized individuals to take the car. Similarly, in the world of JWTs, you need to safeguard the JWT issued by the user API. If a malicious actor acquires the JWT, they can present it to the JWT API and potentially gain unauthorized access.

The JWT API will consider a stolen JWT as valid because the malicious actor didn't create a new JWT but instead used an existing one. This can lead to unauthorized access to user data, making it essential to be cautious about storing and transmitting tokens. Protecting tokens during transit is crucial, and HTTPS serves as the gold standard for ensuring secure communication.

Securing JWTs: Common Issues

Avoid placing sensitive JWTs in URLs, as they may be cached as query parameters. For token storage, server-side storage is generally recommended, although use cases may vary. If a browser is used as a client, store JWTs as HtttpOnly secure cookies to eliminate cross-site scripting issues. For native clients, use secure storage options like keychains or Android's secure storage.

You should also be aware that a JSON Web Token (JWT) can leak information even if you don't store secrets in it. For example, if the audience claim is a number instead of a UUID, it may reveal the number of users in the system. Since JWTs are not secret, using a number like 42 for the audience ID or subject claim could inadvertently disclose that there are at least 42 users. To avoid this, use UUIDs for such claims. Additionally, when using symmetric key signing, like HMAC, the secret is shared between the entity signing the JWT and the entity verifying it, so take extra care to ensure the integrity and confidentiality of shared secrets.

It is possible to brute force a JSON Web Token (JWT) secret simply by using the token itself. This involves trying different secrets and running the cryptographic algorithm on the header and body, checking if the signature matches. This process continues with various secrets until a match is found. A GitHub project demonstrates this technique in C. The time taken to brute force the secret depends on the HMAC key length; shorter keys are quicker to crack, while longer keys take more time. This highlights the importance of avoiding the "none" algorithm for signing headers, as it leaves JWTs without any signature, making them vulnerable to exploitation.

Essentially, when there is no integrity check, anyone who discovers an API that accepts a JWT with no algorithm or signature can create any payload they want, base64 encode it, and present it to the API. This results in a completely valid JWT, emphasizing the importance of having proper checks in place to maintain the security and integrity of user data within applications.

Enhancing Security and Integrity of JWTs with Asymmetric Key Pairs

When dealing with JWTs, it's crucial to avoid using the "none" algorithm, as it's like accepting unsanitized user input – but worse. This is because unsanitized credentials might lead to unauthorized access, compromising the security and integrity of user data within applications. To further enhance security, it's important to understand the differences between asymmetric and symmetric key pairs. Asymmetric key pairs involve a public and private key, while symmetric key pairs use a shared secret for both signing and verifying JWTs. Knowing the appropriate key pair type for your scenario can help maintain the confidentiality and integrity of your JWTs.

In our example, the user API generates a JSON Web Token (JWT) and passes it to the client, which then presents it to the to-do API. As previously mentioned, HMAC is a common algorithm used for this purpose, which means that both the user API and the to-do API share a secret. This shared secret is essential for maintaining the security and integrity of user data across different APIs.

Bearer tokens, such as JSON Web Tokens (JWTs), need to be protected and securely stored. When using symmetric key signing, like HMAC, sharing and distributing the secret securely is crucial. One option is to distribute multiple secrets during deployment for rotation purposes. However, asymmetric key pairs can help avoid this complexity. Asymmetric key pairs, like RSA or elliptic curve cryptography, use a private key to sign the JWT and a public key to verify it. This approach simplifies the distribution and management of keys, enhancing the security and integrity of user data across different APIs.

Using asymmetric key pairs can significantly improve scalability in various situations. For instance, sharing a secret between a small development team maintaining two or three APIs might not be a big issue. However, if you have two different companies or departments, sharing a secret becomes a much more significant concern. In our example, let's say Pied Piper owns the user API, and Hooli owns the JWT API. Using asymmetric key pairs in such a scenario simplifies the distribution and management of keys, ultimately enhancing the security and integrity of user data across different APIs.

The user API generates a JWT, signs it with a private key, and passes it to the client. The client then presents the JWT to the JWT API, which can obtain the public key (since it's public) and verify the JWT. This approach allows for scaling out to include other organizations. Security is also a crucial factor. When using a symmetric key pair, the same secret is used to sign and verify the JWT. This increases the attack surface, as every entity verifying a JWT needs to be as secure as the user API. Using asymmetric key pairs, like RSA or elliptic curve cryptography, can help mitigate this issue and maintain the security and integrity of user data across different APIs.

When using symmetric key signing like HMAC for JWTs, the security implications are significant. If someone steals the shared secret, they can create JWTs with any desired access, such as super admin privileges, until the secret is rotated. This poses a substantial risk to user data and API security. However, asymmetric key pairs for JWTs are between three and 10 times slower than symmetric key signing. If you're processing a large number of JWTs, the trade-off between security and performance might be worth considering.

The Importance of Refresh Tokens

Refresh tokens are essential because access tokens, especially for users, are meant to be short-lived, lasting only seconds or minutes. In contrast, refresh tokens can be long-lived. They are presented to the user API, which then generates new JSON Web Tokens (JWTs) as needed. This helps maintain the security and integrity of user data within applications.

Refresh tokens play a crucial role in maintaining a balance between security and user experience. Without them, you'd have two undesirable alternatives. One is having short-lived access tokens, like five minutes, which would force users to log in frequently, leading to a poor user experience. The other option is having long-lived tokens, such as a month, which compromises security. Refresh tokens help avoid these issues by allowing users to obtain new JWTs without constantly logging in, ensuring a more seamless and secure experience.

Balancing Security and User Experience with Refresh Tokens in JWT-Based Systems

If someone manages to steal a JSON Web Token (JWT), they would have an extended period to exploit the token, pulling down data and potentially causing harm. This is why it's essential to strike a balance between security and user experience using refresh tokens, which allow users to obtain new JWTs without logging in frequently. This approach ensures a more seamless and secure experience, minimizing the risk of unauthorized access and protecting user data.

Refresh tokens operate by being requested during the authentication process and being sent along with the JWT. The JWT is then presented to the API, and data is exchanged. This pattern continues throughout the JWT's lifetime.

Eventually, the JSON Web Token (JWT) will expire, and the two APIs will deny further access to the user's to-dos. At this point, the client can present the refresh token to the user API. The user API checks if the user is still valid, logged in, and has paid their bill, then issues a new JWT. The new JWT is sent to the client, which can then present it to the to-do API for validation. This process allows for re-authentication of the user in a silent manner, ensuring a seamless and secure experience.

In a JSON Web Token (JWT)-based system, revoking access during logout is primarily done by revoking the refresh token. When a user chooses to log out, the client sends a message to the user API to stop issuing JWTs for the associated refresh token and then deletes them from its storage. However, revoking the JWT itself is more complex, and there isn't a straightforward method. RFC 7009 provides a way to revoke refresh tokens, but using this specification could result in losing certain capabilities.

Removing a public key from the list of public keys is one way to revoke access, as the library won't be able to find the valid public key, resulting in a failed signature check. Another option is using very short lifetimes for JWTs, making them almost one-time use. A third option is employing a deny list, which is a FusionAuth-specific feature, but similar implementations exist in other libraries. When a user logs out, an event is triggered and sent to different APIs, notifying them that a specific refresh token has been revoked. This helps maintain security and integrity across various APIs.

Conclusion

JWTs serve as a valuable tool in the architectural tool belt, offering both advantages and disadvantages. While they can be used alongside web sessions, web sessions come with their own set of costs and benefits, such as using cookies and being less scalable. Despite these limitations, web sessions are simpler to understand. JWTs are not a one-size-fits-all solution, but they provide a powerful option for maintaining security and user experience across various APIs.

Additional resources, such as the JWT debugger and sample code, can help deepen your understanding of JSON Web Tokens. In-depth articles are available for further reading, providing comprehensive insights into JWTs. The FusionAuth Community Edition, a free authentication and authorization server, can also be a valuable tool in your technical arsenal. If you have any questions or require clarification, don't hesitate to ask.

Additional Info:

Playing with Fire: Serverless PWA with Firebase

Brian Rinaldi — Thu, 04 May 2023 18:53:18 +0000

Firebase is Google's backend as a service (BaaS) for serverless work, simplifying the development and deployment of web and mobile applications. It helps front-end developers to create fully-featured applications without the need to manage servers or infrastructure.

Check out this presentation on serverless if you aren't familiar with what it is and what services fall under its umbrella.

Firebase offers various tools, including a web console, command-line interface, and numerous software development kits (SDKs) for different frameworks such as JavaScript, React, Angular, and Vue. Key features of Firebase include serverless databases (a Realtime Database and Firestore), built-in authentication, file storage, hosting, and serverless Cloud Functions. Additionally, Firebase provides tools to improve app quality, monitor performance, analyze crashes, and grow your business, including monetization options. Basically, Firebase is an all-in-one platform for building and launching applications.

This article is based upon a presentation from Michael Dowden, a Google Developer Expert in Firebase, based on his more than six years experience with the platform.

Getting Started

When you log into the Firebase console, you'll find a list of your projects. To create a new account and project, head to console.firebase.google.com. Once you have a project, you'll need to access your app configuration for essential information like your API key. Although it's not considered sensitive data, it's still better to keep it somewhat private.

Keep in mind that these IDs are not part of Firebase's security model, so you don't need to worry too much about them. The security model won't be discussed in depth here, but it's important to understand that you can safely commit these IDs to your repo and share them with others.

To enable a service, simply toggle it in the Firebase console. You can also access your project overview, project settings, and Firestore database. For instance, for our sample app, we can view chats and a list of users in our database.

Creating a Simple Chat Application with Firebase and Firestore

In order to dig into Firebase, we'll be exploring a simple chat application, allowing users to log in/out, send messages, upload files, and even run searches based on email addresses to display messages from specific users. This simple chat application serves as a framework to teach the principles and fundamentals of what you can do with Firebase.

Firebase offers two types of databases: the real-time database and Firestore. The real-time database is the original Firebase database, supporting multiple concurrent users and using pure JSON data structures. Firestore, on the other hand, is a true document database with documents and collections. It can support over a million concurrent users and has better security rules than the real-time database. Although working with pure JSON can be convenient for APIs and integrations, Firestore provides more value with improved querying and other features.

The pricing models for these databases differ, which can impact the choice between them. For our project, we'll use Firestore, as it is more popular and the default choice for most use cases.

Our Firestore database structure will consist of a collection of chats, each having a document ID and properties such as content text, display name, and timestamp. This allows sorting messages in chronological order. We'll also have a collection of users, with each user having their UID, display name, and a URL to their avatar for display in the app. To secure the database, we'll need to set up appropriate security rules.

Implementing Firestore Database Rules and Functionality for a Chat Application

Let's discuss Firestore database rules for a chat application. First, the base match pattern is set for all Firestore databases using the first three lines of code. Next, we create rules for the data model. For the chats collection, we require users to be authenticated for reading and creating documents (allow red, create), but editing and deleting are not allowed to prevent users from changing their posts.

For the users collection, we add a variable UID to represent the user's unique ID. Although currently, there are no nested documents, the important part is the UID rule. This rule allows authenticated users to read any user record but limits writing only to their own user record.

allow read: if request.auth != null
allow write: if request.auth.uid == uid

This is because the user information, such as avatars and display names, is used in the chat.

To read from the database using the Firebase SDK, you would obtain the Firestore object from the global Firebase object and perform a get operation to retrieve all documents in a collection. You can then print each document from the snapshot.

const db = firebase.firestore();
const snapshot = await db.collection('chats').get();
snapshot.forEach(doc => console.log(doc.data()));

This demo uses AngularFire, since this is an Angular-based demo, but you can accomplish similar tasks in other frameworks. By subscribing to changes on the collection, you can watch in real time as new data arrives and print it without requiring any special update checks. This subscription keeps you connected to the database, allowing you to receive updates as long as the code runs.

To create a chat object based on our data model, we utilize the SDK or AngularFire. With either method, adding the object to a collection is simple.

const chat = {
  contentText: "Hello, World!",
  displayName: "Michael",
  uid: "E4XcggIvYJQs2ZfhfXU3Cba18ye2",
  avatarUrl: "https://gravatar.com/avatar/02e8d17cca"
}

const db = firebase.firestore();
await db.collection('chats').add(chat);

Implementing Chat Image Functionality

The process begins by accepting text or a file for upload. After saving the chat, the ID reference of the created object is stored for later use. If a file is uploaded, the AngularFire storage object from the SDK is used to handle the file upload. Once the file is uploaded, the URL for displaying the image is obtained.

Next, the chat document needs to be updated with the ID reference. This involves adding content text, such as "user provided image," the file name of the image, and the URL for displaying the image.

async post(content: string | File, user: Profile): Promise<string> {
  const id = await this.saveChat(content, user);
    if (content instanceOf File) {
    const filePath = `/images/${user.uid}/${id}/${content.name}`'
        const snapshot = await this.fs.upload(filePath, content);
    const imageUrl = await snapshot.ref.getDownloadURL();
    this.db.doc(`/chats/${id}`).update({
      contentText: `User provided image: ${content.name}`,
      contentImageURL: imageUrl,
    })
    }
    return id;
}

Implement Authentication

The FirebaseUI library offers a simple way to implement authentication in JavaScript applications. You can easily configure it to support various authentication methods, such as Google, email/password, and Facebook, and it will display the appropriate buttons and manage the necessary processes. To adapt FirebaseUI for Angular, an Angular wrapper is available, which requires minimal configuration. In this case, auto-sign-on is disabled, and users must sign in manually. The module configuration specifies the use of Google Auth and email auth provider. Additionally, users are prompted to select their Google account each time to accommodate those with multiple accounts. The credential helper feature assists users in remembering their previous login methods and managing credential information. To integrate authentication into an application, simply add the configuration, import the necessary components, and the process is complete.

const firebaseUiAuthConfig: firebaseui.auth.Config = {
  signInSuccessfulUrl: '/',
  signInFlow: 'redirect',
  signInOptions: [
    {
      provider: firebase.auth.GoogleAuthProvider.PROVIDER_ID,
      customParameters: {
        prompt: 'select_account',
      },
    },
    {
      provider: firebase.auth.EmailAuthProvider.PROVIDER_ID,
      requireDisplayName: false,
    },
  ],
  credentialHelper: firebaseui.auth.CredentialHelper.GOOGLE_YOLO
}

Creating a PWA

Progressive Web Applications (PWAs) require three main components: capability, reliability, and installability. To create a PWA, you need a web manifest for app details, HTTPS for security, an icon to represent the app on devices, and a registered service worker, which is mandatory.

If you're unfamiliar with PWA, check out this presentation by Lee Warrick on Demystifying PWAs

In Angular, creating a PWA is straightforward by adding the Angular PWA package, which handles configuration, service worker addition, and manifest building.

To ensure your app icons display correctly on various devices, use a tool like maskable.app to generate safe icons. This tool provides full bleed icons that adapt to different cropping shapes, such as circles or squares. To manage and test your Progressive Web App (PWA), utilize Chrome DevTools for handling service workers and Lighthouse for checking the PWA's status and potential issues. For Apple device compatibility, include the Apple Touch icon without a transparent background. Although not mandatory, prompting users to install your PWA can enhance their experience. To do this, subscribe to the relevant event and use the appropriate platform module for your framework. Note that Android and iOS require slightly different approaches, with Android needing event bubbling prevention and an "Add to Home Screen" prompt.

In iOS, it's crucial to ensure that our app is not already a Progressive Web App (PWA) by checking it as a separate step. Additionally, when opening the prompts component, it's necessary to include a slight timeout if the service worker is running for the first time, as it may take some time to initialize. To demonstrate this, we will be working with an Angular project that also contains Firebase functions. Using Angular 13, we can add the necessary package using the Angular command line, which automates additional configuration steps that would not be covered by simply using npm to add the package. Once completed, we can proceed with the remaining tasks.

With a web manifest and an Angular service worker config file in place, testing the Progressive Web App (PWA) requires a different approach. Instead of running the Angular build, you'll need to use the Firebase local server, which supports HTTPS—necessary for testing PWAs. After building the app, run it inside the Firebase server, changing the port from 4200 to 5000. Once up and running, use Lighthouse to evaluate your PWA's performance, giving you valuable feedback and insights into any potential improvements.

public installPwa(): void {
  this.data.promptEvent.prompt();
    this.close()
}

public close(): void {
  this.bottomSheetRef.dismiss();
}

The results for our application show that the PWA is installable and optimized, with six out of seven criteria met. The only issue found was the absence of a valid AppleTouch icon. To fix this, it's a simple process of adding a link to the icon.

Implementing Push Notifications for Progressive Web Applications

With the PWA functioning, the final step is to incorporate push notifications using Firebase Cloud Messaging. Many projects use Firebase Cloud Messaging for push notifications even if they don't use Firebase for other purposes, as it is effective for web applications.

Firebase Cloud Messaging can be used with native applications or hybrid applications like Ionic. It allows you to send targeted messages to groups of devices. Each device must separately subscribe to receive push notifications. Typically, notifications are shown in-app when the app is loaded and on the device using the device notification system when the app is not loaded. Sending notifications to all subscribed devices can become complex, especially when data changes frequently.

For an Angular-based chat application, start by adding the Angular Fire Messaging Model. Next, enable the messaging API in the Google Cloud Console, as it is crucial for the functionality. Finally, choose the type of subscription or notification for your application. Keep in mind that this example demonstrates the basics, and more advanced use cases may require additional steps.

In this project, we utilize topic notifications, which may not suit all use cases. Device tokens for registered devices must be stored, and in our example, we keep an array of device tokens within the user record in our Firestore database. Notifications can contain images, links, and text messages, but the delivery order of multiple messages isn't guaranteed. It's essential to address the removal of stale tokens and require users to renew their device registration periodically. While topics are an excellent feature, it's crucial to implement exceptions, especially for user-centric events such as chat applications where users shouldn't receive notifications for their own messages. To begin, you'll need to enable the Firebase Cloud Messaging API in the Google Cloud Console.

AngularFire Messaging is used to subscribe to messages, and a snack bar displays the notification. The subscription takes place within the app, so it runs the entire time the app is active. When a message is received, the notification will be displayed. Ensure that the messaging module is added and the enable/disable notification buttons are functioning correctly. In the chat room component, if there is a user notification token, obtain the token and subscribe. When the enable notifications button is clicked, the device token will be registered.

The request token is a feature of AngularFire Messaging, which helps in managing notifications. To enable notifications, delete the token from AngularFire Messaging, record it in the database, and then subscribe the user to a topic. Conversely, to disable notifications, remove the token from the database and unsubscribe the user from the topic. A cloud function handles the actual sending and posting of notifications. This function subscribes to user updates, which change when users enable or disable notifications. When a chat is created, the cloud function sends the notification. The topic name used in this example is "chats."

When a chat is created, the admin SDK's messaging feature sends a message with the title "new chat message" to a topic, indicating the sender and the message content. This broadcasts the message to all subscribers of the topic, and they receive the notification. When the notifications are enabled and someone sends a message, an in-app notification appears thanks to Firebase Cloud Messaging (FCM)

Exploring Firebase Features and Integrations for Web and Mobile App Development

Firebase offers several useful features, such as the command line interface, which enables project creation and management, user account import/export, and database manipulation. The CLI provides extensive scripting and local management capabilities.

With just one command, Firebase Deploy, you can push your content to hosting and have it run in the wild. Firebase offers a range of configuration options, such as redirects, rewrites, and security headers. Additionally, you can easily roll back to a previous deployment using the web console.

Firebase Cloud Functions allow for serverless functions, enabling you to create triggers for various events such as database updates or creations. For example, you can create an emoji function that replaces "LOL" with a laughing emoji whenever a chat is created. This function is triggered after the initial text post, showcasing the capabilities of serverless functions in real-time.

It's quite common to mix and match Firebase components with other services, such as using Firebase Cloud Messaging (FCM) for push notifications or Firebase Hosting for static files. While Firebase Auth might be the hardest component to mix and match, it's still possible to integrate it with other authentication providers like Auth0 or Okta.

Firebase Hosting is a fast, CDN-backed static file hosting service. Although it's primarily for static files, it can also deploy Node.js applications through serverless Cloud Functions. For example, Angular Universal can use server-side rendering with Firebase Deploy, which bundles the app into a Cloud Function. However, this approach can lead to cold starts, so it's advisable to have a minimum instance of one Cloud Function running to minimize latency issues in low-traffic sites.

Firebase doesn't lock you into the platform, and you can easily export data from the Realtime Database as JSON or use built-in REST APIs to interface with the data. The Firebase free tier offers a generous amount of usage, and even when you move to a paid tier, you still get a free allowance of bandwidth and storage every month. The cost of Firebase depends on factors such as data usage and optimization, but with careful planning, it can be an affordable solution.

What the Heck is Edge Computing?

Brian Rinaldi — Sun, 23 Apr 2023 12:56:05 +0000

Edge compute is the distribution of serverless functions to thousands of locations worldwide, handling requests as close to the user as possible. This leads to programmable, dynamic, customizable responses with minimal latency, that can potentially make your application perform faster.

To better understand edge compute, let's first define compute. Compute refers to the amount of work computers must perform to generate output.

This article is based upon a presentation by Austin Gil, a web educator and developer at Akamai, one of the largest CDN providers, which also offers compute at the edge.

What do We Mean by Compute

There are various methods of generating output like HTML, PDF, dynamic images, or JSON. The main options include traditional servers for server-side rendering, client-side rendering, static site generation, and rendering in the cloud via Cloud Functions.

Traditional servers involve running software on a machine you've chosen to execute your code and generate the desired output. These servers offer predictable pricing, unbounded run times, and complete control. However, this also means you pay even when the server isn't actively working, and scaling can be complicated. Additionally, deploying to a single location might cause latency for users in other parts of the world. Moreover, you'll be responsible for maintaining server software in addition to your application. Think of traditional servers like a commercial workspace: you either own or rent the space, providing machinery and workers, but pay the bill regardless of the number of clients you serve.

Exploring Rendering Methods: Client-side, Static Site Generators, and Serverless Functions

Client-side rendering involves sending code to the user's browser, where it's responsible for generating the HTML for the page. This can be done using JavaScript or WebAssembly. The benefits include reduced latency, as the code is running on the user's machine, and potentially offline functionality with a service worker. However, there are downsides are that the initial download size is larger, secrets cannot be hidden in the code, you have less control over the environment, and performance depends on the user's device.

Static site generators, on the other hand, pre-build all pages in an application and generate static resources that can be deployed to various targets. When a user requests a page, there's no need to build it on demand – they simply receive a static asset. This approach falls under the server-side rendering umbrella, since it runs in environments like Node.js, Go, or PHP.

The main advantages of using pre-built assets are immediate response time, easy deployment to CDNs, and being fast, cheap, and secure due to static nature. However, there are some drawbacks, such as the inability to handle dynamic content like logins or authenticated content without combining client-side rendering or other dynamic rendering methods. Additionally, the build time for static site generators increases linearly with the number of pages, potentially causing delays in generating a large amount of content.

In terms of compute resources, Cloud Functions (also known as Lambda Functions or Serverless Functions) enable a cloud provider to manage the necessary computing and scaling of an application based on the user-created functions. The provider maintains the machines and server-side software, handles requests, and passes them to the designated functions. Often, a URL is provided for the function to receive requests.

Serverless Functions offer several advantages, including easy provisioning and infinite scalability. As traffic to a URL increases, service providers can allocate more machines to run the same function. The pay-for-what-you-use model can make pricing more affordable, especially for low-traffic applications. Since the service provider maintains server software, developers only need to focus on business logic and function code, simplifying maintenance.

However, there are drawbacks to serverless functions. Developers must adhere to conventions set by the provider, such as naming, file structure, and stateless principles. Due to the scaling nature, shared state, memory, or file systems cannot be relied upon. Service providers also determine available software and languages, limiting choices for developers. Finally, like traditional servers, Cloud Functions can suffer from latency if deployed far from the user requesting it.

Exploring Edge Compute: Cloud Functions, CDNs, and Performance Optimization

Now, let's discuss the edge part of edge compute. The edge is essentially a globally distributed network of computers capable of handling user requests. The importance of edge computing lies in reducing latency. By placing resources closer to users around the world, we can decrease the time it takes for users to make a request and receive a response, ultimately improving their experience.

A content delivery network (CDN) is an excellent example of the edge in web development. It consists of thousands of connected computers that deliver resources to users upon request. CDNs are particularly effective for reducing latency due to their proximity to users, often within a few miles. They work well for distributing static resources, such as CSS, JavaScript, images and fonts, generated by static site generators. However, CDNs are not suitable for applications that require dynamic content. A big part of the improved performance is in terms of reduced distance – the amount of distance a request and response must travel.

Latency, download size, and device capabilities are crucial factors affecting web performance. While edge compute doesn't solve the download and device issues, it can greatly improve the the distance. By bringing processing closer to the user, similar to Content Delivery Networks (CDNs), we can improve performance and reduce latency.

Exploring the Advantages of Edge Compute

Edge compute can be used to perform dynamic server-side functionality but executed as close to the user as possible. This not only improves user experience but also offers reliable location information that is not possible with a traditional server. Additionally, many edge compute providers grant access to a key-value store for persistent data on the edge, enabling more customized and dynamic experiences for users without the limitations of pre-built or static options.

Moving rendering to edge compute versus client-side rendering also means less data downloading. This results in better battery life and performance for users' devices.

For developers, edge functions are easy to set up, have a low barrier of entry for proofs of concept, and provide consistent environments. Dynamic content can be based on users' locations without worrying about secret leaks, and JavaScript is the most widely supported programming language on the edge. Additionally, maintaining server infrastructure is unnecessary.

For business owners, dynamic compute at the edge reduces the workload on origin servers, improving performance, reliability, and potentially reducing costs. Edge compute provides automatic scaling and global request distribution. Most edge compute providers offer a pay-as-you-go model, which can save costs. However, it's essential to remember that no solution is perfect and should be considered as the ultimate answer.

Challenges and Considerations in Edge Computing Adoption

Edge compute has its challenges that need to be understood before adopting it. One issue is that many edge compute providers rely on V8 isolates — JavaScript running on the edge — which differs from Node.js. While more resources are being added to edge providers, edge compute offerings usually have fewer resources in terms of function runtime and memory, so avoid long-running functions.

Furthermore, most prominent edge compute providers only allow HTTP as the networking protocol, limiting actions like database access. For instance, establishing a Postgres connection at an edge node might not be possible. However, an edge node can connect to a proxy server that communicates with the database.

It's essential to consider the architectural decisions, as proximity to the user doesn't always guarantee better performance. To illustrate this, imagine a proxy service in the same region as a database (the target). The user (computer emoji in the image below), server (waiter emoji), edge (knife emoji), and target (origin server). Running multiple sequential requests that go from the edge to the server and back is actually slower than running those requests on the server. Careful planning is required to ensure optimal performance in such scenarios.

Traditionally, browsers cannot communicate directly with a database, so an intermediary is needed to handle requests and responses. In a typical setup, a user sends a lengthy request to a server located near the database, resulting in a short connection. With edge computing, the user connects to an edge node with a short hop, but the request might still have a long distance to travel if the target service is far away.

The duration of a request-response cycle depends on the distance between the target and the edge compute node. While edge computing reduces some time due to the shorter distance, multiple sequential or waterfall requests can still cause delays. For example, a request may go from the user to the proxy, then to a service, back to the proxy, and so on before finally reaching the user.

Exploring Edge Computing: Use Cases

Edge computing allows for various optimizations, such as modifying requests to inject ads without client-side ad blocking interference or enabling fast static searches with low latency. This is useful for auto-complete suggestions and store locators, as they don't change frequently. Geolocation-based enhancements can also be made, like preemptively guessing a user's language or applying region-specific policies. Redirect management can be improved, reducing the round trip time for users. Edge nodes are also well-suited for token-based personalization, A/B testing, feature flags, and stateless authentication with JSON web tokens. Additionally, they can serve as proxies for API orchestration and securely store sensitive information like API keys. The introduction of edge computing as a new paradigm opens up exciting possibilities for the next phase of web development.

Edge compute is gaining traction in various frameworks as a potential, primary, or main target, and many already support edge compute deployment. While this is exciting, it can also lead to an increase in mental overhead and architectural decisions, with developers weighing the costs and benefits for performance.

Exploring Edge Computing: Choosing a Provider

Edge computing offers a variety of solutions to save time and optimize performance for various applications. However, the importance of saving 300 milliseconds depends on the use case. For large enterprises like Walmart, it's a significant problem to solve, but for a personal blog site, it might not be worth the effort.

When choosing an edge computing provider, factors like cost, network size, and security should be considered. The big three providers are Akamai, Cloudflare, and Fastly. Cloudflare offers a low-cost entry point and is easy to get started with, while Akamai is known for its large network and strong security. However, the best provider will depend on your specific use case and requirements.