Forem: Torsten Dittmann

Preview npm packages from any PR

Torsten Dittmann — Tue, 20 May 2025 14:42:28 +0000

I recently launched pkg.vc to solve a problem that’s been a pain for years:

How do you test npm package changes from a PR or feature branch, without local builds or polluting the public registry with throwaway versions?

What is pkg.vc?

It’s a service that lets you:

Build and share installable npm packages from any branch or PR
Get a unique install URL for each commit, branch, or PR
Integrate with GitHub Actions or use it locally
Private packages

I was inspired by pkg.pr.new (which is great!), but I needed something that worked with private packages and works outside of GitHub Actions.

How it Works

1. Install the CLI:

npm install -g pkg.vc
# or just use npx
npx pkg.vc --organization my-org ./my-package

You’ll need an API key (see the docs for setup).

2. Publish a Preview

After publishing, you get a unique install URL:

npm install https://pkg.vc/-/@my-org/my-package@5541c6f

You can use this URL in any project to test the package - no need to publish to npm.

3. Automate with GitHub Actions

You can set up a workflow so every PR automatically publishes a preview package.

- uses: pkg-vc/publish-action@main
  with:
    organization: my-org
    directory: my-package
    secret: ${{ secrets.PKG_VC_SECRET }}
    github-token: ${{ secrets.GITHUB_TOKEN }}

Now, every PR gets a comment with an install link.

What’s Next?

I’m still iterating on this, and I’d love feedback.
If you’ve struggled with this workflow, or have ideas for improvement, let me know!

Docs
Discord

New Operators to Query Documents More Efficiently

Torsten Dittmann — Wed, 12 Apr 2023 13:13:26 +0000

Query operators are essential to search and retrieve data from Appwrite. To allow more advanced queries, we have added new operators to the mix with Appwrite 1.3.

🤔 New to Appwrite?

Appwrite is an open-source back-end-as-a-service that abstracts all the complexity of building a modern application by providing you with a set of REST, GraphQL, and Realtime APIs for your core back-end needs. Appwrite takes the heavy lifting for developers and handles user authentication and authorization, databases, file storage, cloud functions, webhooks, and much more!

Operators

This section will introduce each of these operators and provide examples of how they affect results.

`isNull`

The isNull operator is used to query for documents with null or missing values. This operator is handy when you need to identify incomplete data or records that lack specific fields. For example, if you're managing customer data and want to find customers who haven't provided their email addresses, you can use the isNull operator to retrieve those records.

Query.isNull('email')

[
  {
    "name": "Torsten Dittmann"
    "email": null,
    // ...
  }
]

`isNotNull`

The isNotNull operator, on the other hand, is used to query for documents with values. This operator is helpful when you want to exclude records with missing fields from your query results.

Query.isNotNull('email')

[
  {
    "name": "Christy Jacob"
    "email": "christy@example.com",
    // ...
  }
]

`between`

The between operator is used to query for documents within a range of values. This operator is helpful when retrieving records that fall between two specified values. It can be used with both string and numeric attributes.

Query.between('age', 28, 48)

[
  {
    "name": "Torsten Dittmann"
    "age": 30,
    //...
  }
]

`startsWith`

The startsWith operator is used to query for documents that begin with a specified string.

Query.startsWith('name','Chris')

[
  {
    "name": "Christy Jacob"
    //...
  }
]

`endsWith`

Conversely, the endsWith operator is used to query for documents that end with a specified string.

Query.endsWith('name','mann')

[
  {
    "name": "Torsten Dittmann"
    //...
  }
]

`select`

The select operator is used to select specific fields from a document. This operator is useful when you want to retrieve only certain fields from a document and exclude others.

Query.select(['name', 'email'])

[
  {
    "name": "Torsten Dittmann",
    "email": null
  },
  {
    "name": "Christy Jacob",
    "email": "christy@example.com"
  }
]

Learn More

Adding these database operators to Appwrite has improved the ability of developers to query data more efficiently. It allows for more complex queries with less code, which reduces development time and increases productivity.

Check out our documentation for more information and the release announcement for details on other awesome new features in the latest version of Appwrite.

You can also use the following resources to learn more and get help:

Announcing Appwrite 0.15 with Phone Authentication & More!

Torsten Dittmann — Tue, 28 Jun 2022 14:36:54 +0000

Today, we're thrilled to announce the official release of Appwrite 0.15, including one of our most anticipated features: Phone Authentication! 🥁

Besides Phone Authentication, Appwrite 0.15 also comes with some exciting features, including improved Metadata on all Resources, Webhook Signature, and multiple bug fixes.

🤔 New to Appwrite?

Appwrite is an open-source back-end-as-a-service that abstracts all the complexity of building a modern application by providing you with a set of REST and Realtime APIs for your core back-end needs. Appwrite takes the heavy lifting for developers and handles user authentication and authorization, databases, file storage, cloud functions, webhooks, and much more!

📱 Phone Authentication is Here!

With this release, we are introducing a feature, which has been one of our longest open issues on the Appwrite GitHub repository. Now you are able to authenticate Users, by sending them 6 digits to their phone and have them enter them in your application. Phone authentication will be the 6th authentication method that is now supported at Appwrite alongside email/password, anonymous login, magic links, team invites and JWT.

Initially there are 3 providers which can be used with Appwrite to send SMS verification codes:

The following Environment Variables have been added for configuring Phone Authentication:

_APP_PHONE_PROVIDER
Provider used for delivering SMS for Phone authentication. Use the following format: phone://[USER]:[SECRET]@[PROVIDER]. Available providers are twilio, text-magic and telesign.
_APP_PHONE_FROM
Phone number used for sending out messages. Must start with a leading '+' and maximum of 15 digits without spaces (like +123456789).

Once you have set up your environment variables with your favorite provider credentials, you can call the createPhoneSession method, passing the phone number of the User. This method will send a six digit code to your user as an SMS message.
Once received by your user and submitted back to your application or website , you can verify your user’s code - which is valid for 15 minutes - using the updatePhoneSession method.

We have also added the phone and phoneVerified attributes to the User Object. These new attributes will help you know the user verification status and decide whether you want to force the user to verify his phone or not.

In summary, following endpoints have been added to the Account Service:

createPhoneSession
For initializing the phone authentication.
updatePhoneSession
For completing the phone authentication.
updatePhone
For adding a phone number to your user’s account.
createPhoneVerification
For initializing phone verification.
updatePhoneVerification
For completing phone verification.

Additionally the Users Service has these new endpoints:

updatePhone
For updating the use phone number.
updatePhoneVerification
For changing verification status.

📀 Multiple Databases For Each Project

Until now, each Appwrite project could only use a single database. Starting with version 0.15, this is no longer the case. We're happy to introduce a new database layer to allow you to have as many database instances as you want under a single Appwrite project. Currently, all your project databases still have to use the same database adapter under the hood as defined in your Appwrite's instance environment variables, but this is our first step towards adding support for multiple database adapters under the same Appwrite project.

In the next few months, we will continue to expand our database adapters support, and we can already share that besides existing support for MariaDB and MySQL, we are working on adding support for both MongoDB and Postgres. With the new database layer in place, we will allow you to have multiple adapters under the same project. This will allow you to micro-optimize your project's performance, choose the best technology for your specific use case, and give your team the flexibility and comfort to work with the tools and technologies you already love and trust. All of that is on top of the same Appwrite experience you are already familiar with.

🎉 New `$createdAt` & `$updatedAt` Attributes

They are finally here, $createdAt and $updatedAt are now part of every Resource in Appwrite and contain a timestamp of the date of creation and the latest update. This comes in very handy, if the ordering of the resources is important or if you want to implement offline sync with your application. The new attributes are $createdAt and $updatedAt.

🔐 Webhook Signatures

Security is vital for everyone here at Appwrite, and we strive to give you easy-to-use tools to keep you and your users safe against web threats. This commitment to security is why with version 0.15, we have introduced the ability to validate webhooks with Appwrite. Webhooks now include a x-appwrite-webhook-signature which can be used in combination with a unique Signature Key found in the Webhook to validate the Webhook is actually coming from Appwrite.

🔮 New OAuth Provider for Dailymotion and Autodesk, Gitlab Got an Upgrade

We introduced new OAuth providers for Dailymotion and Autodesk. These new adapters are taking our count to a total of 32(!) supported OAuth providers in Appwrite. On top of the new providers, the GitLab adapter has also been upgraded to support a custom hostname for those of you who are using a self-hosted edition of GitLab.

🥳 And More!

We still haven't covered everything! There's lots more bug fixes and other micro optimizations we made under the hood to make your experience with Appwrite better!

To get all the details on Appwrite 0.15, check out all the changes in the changelog on GitHub.

⏭ What’s Next?

Appwrite 0.16 is not far away and we promise it's going to be even more exciting 🤩 We'll continue improving upon Appwrite's solid performance, flexibility and adding amazing new features.

Do you have ideas for an exciting new feature ? Open up a Github Discussion so you can get feedback from the core team, maintainers and our ever growing community.

You're welcomed to join us on Discord to get all the latest updates about new Appwrite versions and chat directly with the Appwrite team.

Can Appwrite handle 25 Million documents?

Torsten Dittmann — Fri, 07 Jan 2022 17:06:33 +0000

With the release of Appwrite 0.12 come many new features, most notable being a completely overhauled database service.

It's Here! Announcing the Release of Appwrite 0.12!

Eldad A. Fux for Appwrite ・ Jan 5 '22

#javascript #webdev #programming #news

The database didn't scale well with many documents in previous Appwrite versions. This was noticeable in collections containing upwards of ten thousand documents.

Since Appwrite uses the same database for internal resources like Users and Files, this limitation impacted every other service indirectly. The more the data, the slower the requests.

With 0.12, the database API introduces Indexes, which allow you to improve queries for specific documents by a lot. Complimentary, we also added a new and more powerful way of filtering with different query operators.

These improvements along with a complete rewrite of the database service enhanced Appwrite in a way that we were able to effortlessly query a collection of 25 Million documents with excellent response times.

🏁 Benchmark

k6 was used with 500 Virtual Users (VUs) over 30 seconds, with each request performing an EQUALS query against an attribute that is indexed with a Key index.

import http from 'k6/http'
import { Rate } from 'k6/metrics'

const config = {
    headers: {
        'X-Appwrite-Key': '[API_KEY]',
        'X-Appwrite-Project': '[PROJECT_ID]',
        'Content-Type': 'application/json'
    }
};

const myFailRate = new Rate('failed requests');

export let options = {
    vus: 500,
    discardResponseBodies: true,
    duration: '30s',
    thresholds: {
        'failed requests': ['rate<0.05'],
    },
};

export default function () {
    const res = http.get(
        `http://[APPWRITE_URL]/v1/database/collections/[COLLECTION_ID]/documents?queries[]=number.equal(${Math.floor(Math.random() * 25000000 + 1)})`,
        config
    );

    myFailRate.add(res.status !== 200);
}

📝 Collection

Here is a representation of what the collection looks like used for the benchmark, the $id is automatically generated, and number is a unique random value up to 25 million.

$id	number
61d7fe9f91d6fed26fc0	1
61d7fe73eb0142ab50ec	2
61d7fe73eb5142c4b5e5	3
...	...
61d86657b6179ea91483	25000000

⚙️ Request

Requests were performed against the list documents endpoint with an EQUALS query against the number attribute and a random number from 1 - 25,000,000. On top of the query itself, each request goes through authentication, authorization, and permissions validation.

/v1/database/[COLLECTION]/documents?queries[]=number.equal([RANDOM_NUMBER])

🔬 Results

The results speak for themselves. With 40k successful requests and 1,334 per second, we reached numbers that were very difficult to achieve with previous versions.

Not only is the new Database better at scale with simple Queries - but it also improves overall performance with more complex queries.

The Server were Appwrite was hosted had 8-Cores with 16GB memory and it's CPU usage during the benchmark peaked around 80%.

k6 was run on a Mac Mini M1 with 16 GB memory.

This performance boost is part of a series of steps we take to ensure developers can take full advantage of their Appwrite servers. We plan to share more data and insights from both the development process and Appwrite benchmarks with upcoming new versions.

📚 Learn more

You can use following resources to learn more and get help:

Announcing Appwrite Web SDK 5.0

Torsten Dittmann — Mon, 08 Nov 2021 16:55:55 +0000

We are very excited to announce the release of Appwrite's Web SDK version 5.0 with complete TypeScript coverage. It is now available on npm. With this version, each method will now return proper TypeScript definitions.

We hope this will help a lot of developers out there who are using our Web SDK in combination with TypeScript for building their applications. Having response definitions means you will know what method will return and what properties are available to you via autocomplete without leaving your IDE.

⚙️ Setup

First, you need to install the Appwrite SDK or upgrade it to the latest version via npm:

npm install appwrite@5.0.0

The next step is to import, instantiate and configure the SDK:

import { Appwrite } from "appwrite";

const sdk = new Appwrite();
sdk
    .setEndpoint('http://localhost/v1')
    .setProject('PROJECT_ID');

👥 Account

Let's start with the simplest example by getting the current user using the account.get() method. In previous versions of the SDK, this method returned a unknown type, but now you don't need to create your own definitions anymore, since the SDK will offer them out-of-the-box.

const user = await sdk.account.get();

The user object will now already contain all possible properties via a TypeScript definition. But there is more, since the User model also contains the prefs property containing all of the User's preferences. These can be set by the client, which means the SDK cannot provide you with typings yet.

Let's assume you store the users preferred theme for your application in their preferences. You will have Type like this:

type MyPreferences = {
    theme: "light"|"dark";
}

The new SDK allows you to pass MyPreferences via a Generic - this allows you to pass your own structure to the method.

const user = await sdk.account.get<MyPreferences>();

The new user object returned from account.get() using a generic is now automatically extended by your MyPreferences on the prefs property.

Generics can be used on any method, which can return a data structure that is allowed to be extended by the developer like the User's preferences or documents from the Database service.

📀 Database

Talking about Database, let's move on to some examples how the new SDK can be used in combination with it.

Assuming we have a collection containing Movies with following type:

type Movie = {
    title: string;
    published: number;
    genres: string[];
    gotAnOscar: boolean;
};

These are all properties that can be set as rules in a collection, but by default documents in Appwrite come with values like $id, $permissions and $collection.

We can easily import the Models from the SDK and merge Movie with the Document type.

import type { Models } from "appwrite";

type Movie = {
    title: string;
    published: number;
    genres: string[];
    gotAnOscar: boolean;
} & Models.Document;

Now that we have all our TypeScript definitions in place, let's use them by retrieving a Document from the Database using database.getDocument(). We can use Generics to tell TypeScript to use our Movie type:

const avatar = await sdk.database.getDocument<Movie>('movies', 'avatar');

For example with using the database.listDocuments, which will have 2 pre-defined properties called sum and documents, the type passed as a generic will be used for documents:

const movies = await sdk.database.listDocuments<Movie>('movies');

movies.sum; // The sum of all documents.
movies.documents; // Will use an array of our Movie type.

This can also be used with the subscribe() method for real-time updates:

sdk.subscribe<Movie>('collection.movies', response => {
    response.payload; // Will use the Movie type.
});

You can try it out by yourself using this StackBlitz.

The heavily improved TypeScript support of the new Web SDK allow you to kickstart the development of your Application and keep you focused without leaving your IDE.

If you have any issues or questions feel free to reach us on our discord.

📚 Learn more

You can use following resources to learn more and get help

Cover by Kevin Ku from Pexels

Lessons learned from building a WebSocket server

Torsten Dittmann — Tue, 14 Sep 2021 15:15:24 +0000

Appwrite is an open-source, self-hosted Backend-as-a-Service that aims to make app development easier with SDKs available in a variety of programming languages.

Before the release of a Realtime API with version 0.10.0, applications were only able to communicate with our REST API.

It's Here! Announcing Appwrite 0.10 and the new Realtime API!

Eldad A. Fux for Appwrite ・ Aug 31 '21

#opensource #flutter #android #javascript

Why did we build a Realtime API?

REST APIs have been a popular architecture for data delivery in the past. So why do we need Realtime now?

Our REST API works great and is very simple, but in order for us to allow more flexibility, and to allow developers to create new use-cases such as game development and reactive applications, we needed to add a new API layer for realtime interaction.

Rather than the API client only getting new data on their next query, the new data is pushed to them immediately. If a developer is already polling the REST API for data changes, it doesn't just mean they want to access the data faster, but is a strong indication that they really want a realtime API.

Realtime APIs provide a more enjoyable developer experience that can significantly reduce application processing overhead and code complexity. Once the data is transferred to the system in real time, you allow the developers to focus on adding value to the product.

Architecture

Since the Realtime Service is implemented on top of an already existing REST API, all messages sent over Realtime are triggered from the HTTP Server. This means that if a resource is created or updated, the WebSocket Server will be triggered to send this action to its subscribers.

The backbone of the data exchange between REST and WebSocket is a Redis instance. We are using a single Pub/Sub channel, which is the Source of Truth for the WebSocket Server. If a new resource is added via the REST API, the HTTP server will publish the payload alongside metadata in this channel. The WebSocket Server subscribes to the channel, processes the message, decides which client is allowed to receive the message, and sends it to the destined client.

Data Flow

At Appwrite, resources from the REST API are separated by projects, secured by permissions, and the events are categorized in channels. When a client establishes a connection to the realtime server, a project identifier is sent along with information to authenticate the connection to a user and channels via which the client will receive messages. In the following, we take the Car resource as an example, and tell the WebSocket Server to subscribe to the Cars channel.

The WebSocket Server now allocates all roles of the user, the project, and the channels to the unique connection identifier for the client.

If the Car resource now gets updated via the REST API, the HTTP Server publishes this event to the Redis channel with its payload. The WebSocket server will then receive this event and start checking who the receiver of this event will be.

The following conditions need to meet for a client:

The Project ID must be equal.
The Permissions of the resource must meet the user's roles.
The Channel must be subscribed to.

The WebSocket Server will then send the payload of the resource to all clients that meet the conditions.

Data Structure

Speed is vital for building Applications with Realtime Updates. Our Data Structure needs to be processed as quickly as possible to decide which Client is supposed and allowed to receive an event. For this, we are maintaining 2 Hashmaps in memory. One of them holding all Subscriptions, and the other, all Connections.

Looking at the previous conditions, we can see the pattern reflected in this tree. You may realize that this structure has a disadvantage, namely, there are many duplicate data entries of the connection ID. However, this disadvantage is intentional and has a specific reason – speed.

The tradeoff of memory for speed is essential in a WebSocket server. This structure allows us, even with a high number of subscribers, to quickly identify them and forward the message to them, even though this might use up more memory.

Below is an example of our implementation which distributes subscribers evenly across 20 different channels, then using one event to gather all subscribers for that event.

Subscriptions	Time used	Memory used
10,000	0.022ms	11MB
100,000	0.238ms	90MB
500,000	1.525ms	427MB
1,000,000	3.678ms	852MB
5,000,000	19.334ms	4,289MB

These numbers are more than fast enough for everyday applications, especially considering that a single WebSocket server is unlikely to maintain more than a million connections simultaneously. Since the WebSocket server is stateless and only manages its own subscriptions, it can easily scale horizontally and balance off the work.

Now we come to our next data structure and the reason why we need it in the first place.

Let's assume a client connects to our WebSocket server and subscribes to some channels. After some time, the client disconnects and we have to clean up after them and remove their connection from all channels.

To avoid endless loops of identifying every legacy, we have an auxiliary data table that holds the project and roles of each connection easily accessible for us. Using this data, we can remove all the information from the subscribers without much searching.

Stumbling Blocks

Of course, we didn't get everything right the first time. Every time we encountered and solved one hurdle, the next one was already there waiting for us.

Change of Permissions

One of the first hurdles we encountered was: What happens if a user's permissions change while they are connected? What if a user is deactivated and the connection is still open?

The WebSocket server would not know about this change and would continue to send all the messages that the user was allowed to receive at the beginning of the connection. This would result in exposing a resource to someone who is not authorized to read it.

To prevent this phenomenon, we have added a flag to the message sent to the WebSocket server, which indicates whether the permissions for a particular user have changed. When the WebSocket Server receives this message, it checks if this user is currently connected and matches their roles with those in the backend.

The Operating System

Linux’s networking stack comes with sane defaults for many workloads, but the stack isn't tuned for 1+ million concurrent connections. We expected to face some form of the C10k problem, so we prepared our systems in advance [1][2][3]:

Increased the default TCP buffer sizes for the system
Increased the default IPv4 port range
Increased the limit for open files and file handles

Despite this tuning, we hit a limit of around 260k connections - past that point, the HTTP server stopped responding to our clients. We observed that our server wasn't completing the TCP 3-way handshake: it would receive SYN packets from the client (as observed with tcpdump) but wasn't responding with ACK.

After hours of fruitless debugging, we tapped other maintainers to lend their eyes to the problem. Through the power of open-source collaboration, we had our culprit in a matter of minutes:

$ cat /proc/sys/net/netfilter/nf_conntrack_max
262144

Because websocket connections are long-lived, we needed to increase the connection tracking limit in the networking stack. Once increased, we cruised all the way to 1 million connections with ease.

Asynchronous Delivery

When we checked the performance of sending messages everything was going well, that is, until the moment we ran higher scaled tests and were surprised with very poor results. The culprit was the fact that we sent each message serially instead of in parallel.

Fortunately, the solution was only a few lines of code away.

Authentication with Cookies

The first implementation of the WebSocket Server only communicated 1-way, which was sending updates to clients. This turned out to be a problem in retrospect, as our current implementation uses the HTTP-only cookie which is transmitted to the WebSocket server with the handshake.

Later, when developing a demo application, we noticed that under certain circumstances this cookie is not sent, for example, when the client and server are on different domains.

After a bit of research, we came across the information that the handshake is not intended as a method for authentication at all. Reason for this can be found here from one of the maintainers of Chrome's WebSocket implementation. This was solved by additionaly authenticating via a message over the WebSocket protocol. If the user was not authenticated via the cookie, we decided to fall back to authentication via a message and send the token of the cookie to the WebSocket Server.

So, relying on the handshake for authentication alone was obviously a bad idea.

Takeaway

Of course, the above approaches might not apply for every use-case - but they are for us at this point. As Donald Knuth said in his book The Art of Computer Programming:

“The real problem is that programmers have spent far too much time worrying about efficiency in the wrong places and at the wrong times; premature optimization is the root of all evil (or at least most of it) in programming.”

We could micro optimize our data structure to achieve even better results with many more subscribers. However, it is easier to add another instance of the WebSocket Server behind a Load Balancer and scale horizontally.

As long as this works for us, we’ll follow Donald’s advice.

Credits

Thank you for your attention and we hope you enjoyed this article!

Here are some handy links for more informations about Appwrite:

Introducing Magic URL Login to Appwrite

Torsten Dittmann — Tue, 07 Sep 2021 10:17:45 +0000

Appwrite 0.10 introduces Magic URL as an authentication method, which allows users to create an account without providing a password and login with a URL sent via an E-Mail.

This feature is especially useful if you want to provide a passwordless authentication process for your application.

⚙️ Setup

Let's learn how we can add Magic URL Authentication to a Web Application using our Web SDK. The same can be done with our Flutter SDK and Android SDK.

The first step is to add our Web SDK to our project with NPM like this:

npm install appwrite --save

If you're using a bundler (like Rollup or webpack), you can import the Appwrite module when you need it:

import { Appwrite } from "appwrite";

To install with a CDN (content delivery network) add the following script to your HTML file before you use any Appwrite services:

<script src="https://cdn.jsdelivr.net/npm/appwrite"></script>

The next step is to initialize your SDK code with your project ID which can be found in your project settings page:

// Init your Web SDK
const appwrite = new Appwrite();

appwrite
    .setEndpoint('http://localhost/v1') // Your Appwrite Endpoint
    .setProject('455x34dfkj') // Your Appwrite Project ID
;

🎩 Create a Magic URL

Once your SDK is setup, access the Account service and call the createMagicURLSession() method. The method accepts an e-mail address and a redirect URL as arguments.

// Initiate the Magic URL login
appwrite.account.createMagicURLSession('name@example.com', 'http://localhost/')
    .then(response => {
        console.log(response); // Success
    }, error => {
        console.log(error); // Failure
    });

If the createMagicURLSession() method completes without error, the request sends the user an email with a URL containing a secret key for the next step. When the user clicks the link, they are redirected back to the URL you provided with the secret key and userId values attached to the URL query string. This link is valid for 1 hour. If the e-mail passed did not belong to any existing user - this request will also create a user for the passed e-mail address.

🔐 Login with a Magic URL

Now that the user is able to initialize the authentication process, we need to complete it by handling the redirect from the URL provided in the e-mail.

Use the updateMagicURLSession() method and call it with the secret and userId values from the URL's query string.

Please note that in order to avoid a Redirect Attack the only valid redirect URLs are the ones from domains you have set when adding your platforms in the console interface.

const urlParams = new URLSearchParams(window.location.search);
const userId = urlParams.get('userId');
const secret = urlParams.get('secret');

let promise = appwrite.account.updateMagicURLSession(userId, secret);

promise.then(function (response) {
    console.log(response); // Success
}, function (error) {
    console.log(error); // Failure
});

If the updateMagicURLSession() succeeded, the user is now logged in. Note that once a link is used, it cannot be used again.

🏁 Conclusion

If you need help or encounter any difficulties setting up Magic URL Login with Appwrite, please join our Discord.

🔖 References

Introducing the New Appwrite Anonymous Login

Torsten Dittmann — Fri, 04 Jun 2021 12:44:50 +0000

It is important to consider the user experience when someone first comes to your application. The registration process can often prove to be a hurdle for users to use an application, especially if parts of the application are bound to an authentication process and hidden behind a login screen. Moreover, in today's world, the issue of privacy and data protection is a delicate one, especially because of new data privacy regulation, like GDPR and CPRA.

Appwrite 0.8 introduces Anonymous Login as an authentication method, which allows users to create an account without providing personal information such as an email address, username or password.

Why do we need anonymous users?

This feature is especially useful if you want to provide an easy authentication process for an underage audience, mobile apps or use cases where you don't want to store personal information to ensure users anonymity for sensitive subjects like politics or religion.

With Appwrite you can offer your users the option to create an anonymous account, as well as the possibility to convert it into a full-fledged account with an e-mail address and password or linking to an OAuth2 service provider at a later stage.

Setup

Enough talking, let's learn how we can add Anonymous Authentication to a Web and a Flutter Application using our Web SDK and Flutter SDK. The same can be done with other client SDKs we might release in the future.

Web

The first step is to add our Web SDK to our project with NPM like this:

npm install appwrite --save

If you're using a bundler (like Rollup or webpack), you can import the Appwrite module when you need it:

import { Appwrite } from "appwrite";

To install with a CDN (content delivery network) add the following scripts to the bottom of your tag, but before you use any Appwrite services:

<script src="https://cdn.jsdelivr.net/npm/appwrite"></script>

The next step is to initialize your SDK code with your project ID which can be found in your project settings page:

// Init your Web SDK
const appwrite = new Appwrite();

appwrite
    .setEndpoint('http://localhost/v1') // Your Appwrite Endpoint
    .setProject('455x34dfkj') // Your Appwrite Project ID
;

Flutter

The first step is to add our Flutter SDK to our project.

Add appwrite: ^0.6.0 or the latest version in your project's pubspec.yaml file under dependencies.

dependencies:
    appwrite: ^0.6.0

Then run following command to download the dependencies or upon saving your IDE might automatically run this command.

flutter pub get

Next step is to initialize your SDK code with your project ID which can be found in your project settings page:

import 'package:appwrite/appwrite.dart';
final client = Client();

client
    .setEndpoint('http://localhost/v1') // Your Appwrite Endpoint
    .setProject('455x34dfkj') // Your Appwrite Project ID
;

Create an Anonymous User

Once your SDK is setup, access the Account service and call the createAnonymousSession() method.

Web

// Create an anonymous user and login
appwrite.account.createAnonymousSession()
    .then(response => {
        console.log(response); // Success
    }, error => {
        console.log(error); // Failure
    });

Flutter

// Create an anonymous user and login
final account = Account(client);
try {
    final res = await account.createAnonymousSession();
    print(res);
} on AppwriteException catch(e) {
    print(e.message);
}

If the createAnonymousSession() method completes without error, the request will create an anonymous user and automatically logs in the user, setting up the cookie for following requests. Now the user is authenticated and is allowed to access his or her private data and settings.

Convert to Permanent User

When an anonymous user signs up, you might want to allow them to continue their work with a permanent account. This will also allow the user to recover his account in the future and switch between devices.

With Appwrite there is 2 ways of doing so.

E-Mail

By updating the email address of an anonymous account, we can pass an email address and a password to the account.updateEmail method.

Web

let promise = appwrite.account.updateEmail('email@example.com', 'secret');

promise.then(function (response) {
    console.log(response); // Success
}, function (error) {
    console.log(error); // Failure
});

Flutter

try {
    final res = await account.updateEmail(email: 'email@example.com', password:'secret');
    print(res);
} on AppwriteException catch(e) {
    print(e.message);
}

This will convert the anonymous user to a permanent account with email@example.com as email and secret as his or her password.

OAuth2

By calling the account.createOAuth2Session method from an anonymous account, the user will be automatically converted.

Web

appwrite.account.createOAuth2Session('google', 'https://localhost/success', 'https://localhost/failure');

Flutter

account.createOAuth2Session(provider: 'google');

Conclusion

Both options offer users to transfer all their information to an account and use them on other devices.

If you need help or encounter any difficulties setting up Anonymous Login with Appwrite, please join our Discord.

References

Photo by Jason Dent on Unsplash

Appwrite's Isomorphic Web SDK

Torsten Dittmann — Thu, 03 Jun 2021 18:09:29 +0000

We at Appwrite recently announced our new web SDK and would love to share the improvements we introduced. This article is a quick summary of what has changed and things you need to keep in mind while using the latest version of our SDK.

For the developers not much has changed, but internally some things have been rewritten that enable completely new use cases.

Breaking Changes

Let's start with the first and only breaking change: the way our Web SDK is imported. We have switched from default to named exports.

This allows for innovative auto completion and takes away the user's decision making when importing. Every decision you need to make slows you down, which is why things like coding conventions lead to faster development.

It also unifies the importing process, which could be different depending on the bundler in our previous approach.

import * as Appwrite from "appwrite";

// is now

import { Appwrite } from "appwrite";

Isomorphic

Previously, our SDK was initialized in the window object of the browser. This meant that the SDK only worked in the browser and did not cooperate with technologies like Next.js, Nuxt.js or Svelte Kit, which also interact server-side in Node.js.

That's why we refactored parts of our SDK so that it acts isomorphically and according to the environment.

// Node.js
const { Appwrite } = require("appwrite");

// ESM - Modern Javascript
import { Appwrite } from "appwrite";

// IIFE - Browser
const { Appwrite } = window.Appwrite;

JWT

Talking about server-side rendering, when doing SDK calls in your users scope from the server, it is not possible right away, since the HTTP-only cookie used for authentication is saved in the user's browser. That's why the Web SDK now allows to use JWT for authentication.

const sdk = new Appwrite();
sdk.setJWT("eyJhbGciOiJIUzI1NiI.....");

// Prints out the user attached to the JWT
sdk.account.get().then(console.log);

Typescript

Also, the first preparations for the upcoming response models have been integrated, which is going to provide full Typescript coverage across every service.

For this release, the Promise<unknown> generic has been added to all methods that will receive a response from the server. This way it is easier for developers right now to implement their own definitions when working with Appwrite and Typescript.

type User = {
    $id: string;
    name: string;
}

const userA = await sdk.account.get<User>();
const userB: User = await sdk.account.get();

Learn More

You can find the new version of the NPM package here.
Checkout Appwrite's Github Repo.
Our Discord Server is the place to be if you ever get stuck.
You can find all our Documentation here.

#30DaysOfAppwrite : JWT Support in Appwrite

Torsten Dittmann — Wed, 26 May 2021 13:57:39 +0000

Intro

#30DaysOfAppwrite is a month-long event focused on giving developers a walk-through of all of Appwrite's features, starting from the basics to more advanced features like Cloud Functions! Alongside, we will also be building a fully-featured Medium clone to demonstrate how these concepts can be applied when building a real-world app. We also have some exciting prizes for developers who follow along with us!

What is a JWT

JWT (JSON Web Token) is a standard used to create access tokens for an application. It works this way: the server generates a token that certifies the user identity and sends it to the client. The client will send the token back to the server for every subsequent request, so the server knows the request comes from a particular identity.

A well-formed JWT consists of three concatenated Base64 url-encoded strings, separated by dots (.):

Header: contains metadata about the type of token and the cryptographic algorithms used to secure its contents.
Payload: contains verifiable security statements, such as the identity of the user and the permissions they are allowed.
Signature: used to validate that the token is trustworthy and has not been tampered with.

This architecture proves to be very effective in modern Web Apps, whereafter the user is authenticated, we perform API requests either to a REST or a GraphQL API.

Anyway, it is not always recommended to use JWTs for sessions. Using a regular server-side session combined with Cookies is usually much more efficient and less prone to data exposure.

So, why do we need a JWT then?

In the modern web, you will often have several entities communicating with each other. Certain features will naturally be restricted and require some sort of authorization mechanism. At Appwrite we use Cookies for the client-side to communicate with the backend.

Using a JWT, you will be able to authorize the user on the Server-Side within a Cloud Function, Microservice, or SSR.

Create a JWT

Version 0.8 of Appwrite introduced JWT, and it's really easy to generate using the Web or Flutter SDK. Because JWTs are used for authentication and authorization - we can only generate them when we are authenticated.

Web

appwrite.account.createJWT().then(response => {
    console.log(response); // Success
}, error => {
    console.log(error); // Failure
});

Flutter

account.createJWT().then((response) {
    print(response);
}).catchError((error) {
    print(error.response);
});

The createJWT() method will receive an object like this:

{
  jwt: "eyJhbGciOiJIUzI1NiIsInR5cCI6I..."
}

This JWT will be valid for 15 minutes and can only be generated 10 times every 60 minutes per user account.

JWT with Server SDK

Now that we can get our hands on a JWT, we can use it to do actions on the server on behalf of a user without needing to log in or provide an API Key.

For demonstration, let's try to get our current user with a Node.js script:

mkdir appwrite-jwt-test
cd appwrite-jwt-test
npm init -y

Now add node-appwrite as a dependency:

npm install node-appwrite

Create index.js file and put in the following content:

const appwrite = require('node-appwrite');
const client = new appwrite.Client();
const account = new appwrite.Account(client);

client
    .setEndpoint("[ENDPOINT]") // Your API Endpoint
    .setProject("[PROJECT_ID]") // Your project ID
    .setJWT("[INSERT_JWT_HERE]") // Your users JWT
;

account.get().then(r => console.log(r));

Remember to fill out the endpoint, project ID, and JWT. Keep in mind that a JWT is only valid for 15 minutes after generation.

Now we can execute this file with node index.js, and if everything goes well, we should see our user's object 👏

JWT With Cloud Functions

Remember from Day 23 that users can execute Cloud Functions over the Rest API? If a user does this, the Cloud Function will be passed a JWT in the APPWRITE_FUNCTION_JWT environment variable by default for the user who executed the function.

This way, we don't even have to create a JWT from the client-side and pass it around 🎉

JWT With SSR

With release 3.0.0 of the Web SDK for Appwrite, we have refactored it to be isomorphic. This is important in the ecosystem of JavaScript - since, with the rising popularity of SSR, libraries need to work in the Browser - as well as on the Server Side with Node.js.

That's why we have added the setJWT(jwt) method, found in the Server SDK, to the Web SDK as well - which allows developers to use the same SDK for Client and Server Side actions with Frameworks like Next.js, Nuxt.js and Svelte Kit.

Credits

We hope you liked this write-up. You can follow #30DaysOfAppwrite on Social Media to keep up with all of our posts. The complete event timeline can be found here

Feel free to reach out to us on Discord if you would like to learn more about Appwrite, Aliens, or Unicorns 🦄. Stay tuned for tomorrow's article! Until then 👋

#30DaysOfAppwrite : Cloud Function with CRON

Torsten Dittmann — Tue, 25 May 2021 13:13:26 +0000

Intro

#30DaysOfAppwrite is a month-long event focused on giving developers a walk-through of all of Appwrite's features, starting from the basics to more advanced features like Cloud Functions! Alongside we will also be building a fully-featured Medium clone to demonstrate how these
concepts can be applied when building a real-world app. We also have some exciting prizes for developers who follow along with us!

Creating Statistics

On Day 24, we created a Cloud Function that was triggered by an event. This comes in handy when you want to react to interactions from the Client Side. For Day 25, we are going to create a Cloud Function that will be triggered at particular intervals of time. We can accomplish this by adding a CRON Schedule to our Cloud Function.

For Day 25, we are creating a Cloud Function that will run every day and create statistics for our App. We are going to save the number of profiles and posts every day in a Collection - this data allows us to develop Graphs and Statistics to track.

First of all, we are going to create a new Statistics collection with the following Rules:

Profiles:
- ID: profiles
- Rule Type: Integer
Posts:
- ID: posts
- Rule Type: Integer
Timestamp:
- ID: timestamp
- Rule Type: Integer

The Permissions will be role:all for read, so anyone can retrieve the statistic, and we are going to leave the write permissions empty. Leaving the write empty will block anyone from writing to that Collection, except when they're using an API key.

Now that the Collection is prepared let's start with our cloud function. For this example, we are going to create another Node.js Cloud Function. As environment variables, we are going to add the following:

APPWRITE_PROJECT_ID: Insert your project ID.
APPWRITE_ENDPOINT: Insert your Appwrite endpoint.
APPWRITE_API_KEY: Insert an API key that has documents.read and documents.write permissions.
STATISTICS_COLLECTION: Insert the ID of the Statistics collection.
PROFILE_COLLECTION: Insert the ID of the Profile collection.
POST_COLLECTION: Insert the ID of the Post collection.

Under the Settings page of this Cloud Function, you also need to add a value in the Schedule (CRON Syntax) field. For our use case, we are setting it to 0 12 * * *, which will execute this function every day at 12:00.

We will again move to our project directory and use the Appwrite CLI to create a NodeJS 16.0 function called create-statistics.

appwrite init function

Within the src/index.js, put in the following content:

const appwrite = require("node-appwrite");

module.exports = async function(req, res) {
  const STATISTICS_COLLECTION = req.env.STATISTICS_COLLECTION;
  const PROFILE_COLLECTION = req.env.PROFILE_COLLECTION;
  const POST_COLLECTION = req.env.POST_COLLECTION;

  // Initialise the client SDK
  const client = new appwrite.Client();
  const database = new appwrite.Database(client);

  client
      .setEndpoint(req.env.APPWRITE_ENDPOINT) // Your API Endpoint
      .setProject(req.env.APPWRITE_PROJECT_ID) // Your project ID
      .setKey(req.env.APPWRITE_API_KEY) // Your secret API key
  ;

  // Get the sum of Profiles and Posts
  const profiles = database.listDocuments(PROFILE_COLLECTION, [], 0).then(r => r.total);
  const posts = database.listDocuments(POST_COLLECTION, [appwrite.Query.equal('published', true)], 0).then(r => r.total);


  // Waiting for all promises to resolve and write into the Statistics Collection
  Promise.all([profiles, posts]).then(([profiles, posts]) => {
      return database.createDocument(STATISTICS_COLLECTION, "unique()", {
          posts: posts,
          profiles: profiles,
          timestamp: Date.now()
      }, ['role:all']);
  }).then(res.send).catch((err) => res.send(err, 500));
};

We will then deploy the function to our instance, go ahead and change our directory to the project one and run the following command:

appwrite deploy function

Make sure to select our create-statistics function and deploy it.

Testing Our Cloud Function

We can easily test out our function by waiting for 12:00 or just executing it manually on your Functions page. If the function was executed successfully, you could head over to the Statistics Collection, and you should find a document like this:

With this data, we can create Charts and Statistics to monitor the usage of our application.

Feel free to share how you would leverage this data and implement it in the Medium clone!

Credits

We hope you liked this write-up. You can follow #30DaysOfAppwrite on Social Media to keep up with all of our posts. The complete event timeline can be found here

Feel free to reach out to us on Discord if you would like to learn more about Appwrite, Aliens, or Unicorns 🦄. Stay tuned for tomorrow's article! Until then 👋

#30DaysOfAppwrite : Our First Cloud Function

Torsten Dittmann — Mon, 24 May 2021 13:10:16 +0000

Intro

#30DaysOfAppwrite is a month-long event focused on giving developers a walk-through of all of Appwrite's features, starting from the basics to more advanced features like Cloud Functions! Alongside we will also be building a fully-featured Medium clone to demonstrate how these
concepts can be applied when building a real-world app. We also have some exciting prizes for developers who follow along with us!

Reading Time

The first Cloud Function we will implement in our Medium clone will be a function that calculates the reading time of a post. Calculating the reading time of posts as you browse can be quite an expensive task, depending on the length of the content. In order not to slow down your application unnecessarily, we will run this process on our server.

We will use a formula suggested in this blog post from Infusion Media.

First of all, we are going to add the following rule to our Posts collection:

ID: readingTime
Rule Type: String

Now that the Database is prepared let's start with our cloud function. For this, we will create a Cloud function with Node.js runtime. In your Function Dashboard under the Settings Tab we need to enable the trigger for the events database.documents.create and database.documents.update. As environment variables, we are going to add the following:

APPWRITE_PROJECT_ID: Insert your project ID.
APPWRITE_ENDPOINT: Insert your appwrite endpoint.
APPWRITE_API_KEY: Insert an API key with documents.write permission.
POSTS_COLLECTION: Insert the ID of the Posts collection.

To stay true to the language of our demo project, we will write it in Node.js.

Change your current directory to the project folder we created two articles ago. Then, we are going to create a new function using the Appwrite CLI:

appwrite init function

Set the name to reading-time, and the runtime should be node-16.0. This will create a new folder called reading-time with a ready-to-go template for nodeJS.

Move into the reading-time folder and edit the src/index.js file, then replace the entire content with the following:

const sdk = require("node-appwrite");

module.exports = async function(req, res) {
  const DATA = JSON.parse(req.env["APPWRITE_FUNCTION_EVENT_DATA"])
  const POSTS_COLLECTION = req.env["POSTS_COLLECTION"];
  const {
    $id,
    $collection,
    text,
    published
  } = DATA;

  if ($collection !== POSTS_COLLECTION || !published) {
    return res.send("Failed, invalid collection or not published", 400);
  }

  const client = new sdk.Client();
  const database = new sdk.Database(client);

  if (
    !req.env['APPWRITE_ENDPOINT'] ||
    !req.env['APPWRITE_API_KEY']
  ) {
    console.warn("Environment variables are not set. Function cannot use Appwrite SDK.");
  } else {
    client
      .setEndpoint(req.env['APPWRITE_ENDPOINT'])
      .setProject(req.env['APPWRITE_PROJECT_ID'])
      .setKey(req.env['APPWRITE_API_KEY'])
      .setSelfSigned(true);
  }

  let words = text.match(
    /[A-Za-z\u00C0-\u017F]+|[\u0400-\u04FF\u0500–\u052F]+|[\u0370-\u03FF\u1F00-\u1FFF]+|[\u4E00–\u9FFF]|\d+/g
  );

  words = words ? words.length : 0;

  let minutes = words / 200;
  let seconds = (minutes % 1) * 60;
  let readingTime = `${Math.floor(minutes)}m ${Math.floor(seconds)}s`;

  // Don't update Post if reading time has not changed
  if (readingTime === DATA.readingTime) {
    return res.send(`Post ${$id} has not changed`, 200);
  }

  await database.updateDocument($collection, $id, {
      readingTime: readingTime
  }).then((data) => res.json(data)).catch((err) => res.json(err, 500))
};

This function is triggered on every document write and update events, calculates the reading time, and saves it to the readingTime attribute. We are also checking if the reading time changes - this is necessary not to create an infinite loop and unnecessarily hit the API with our Cloud Function.

We will then deploy the function to our instance, go ahead and change our directory to the project one and run the following command:

appwrite deploy function

Make sure to select our reading-time function and deploy it.

Testing Our Cloud Function

We can quickly test our reading time calculation by navigating to the Posts collection and editing the text of a published post. You can either navigate to the Functions Dashboard and check the log or refresh the document we just updated and see how the readingTime attribute has magically been updated!

The only thing left for us is to add the reading time to our Medium clone at the top of each post:

// src/routes/Post.svelte

//...
<i>
  {post.readingTime}
</i>
//...

Credits

We hope you liked this write-up. You can follow #30DaysOfAppwrite on Social Media to keep up with all of our posts. The complete event timeline can be found here

Feel free to reach out to us on Discord if you would like to learn more about Appwrite, Aliens or Unicorns 🦄. Stay tuned for tomorrow's article! Until then 👋

Forem: Torsten Dittmann

Preview npm packages from any PR

What is pkg.vc?

How it Works

1. Install the CLI:

2. Publish a Preview

3. Automate with GitHub Actions

What’s Next?

New Operators to Query Documents More Efficiently

🤔 New to Appwrite?

Operators

isNull

isNotNull

between

startsWith

endsWith

select

Learn More

Announcing Appwrite 0.15 with Phone Authentication & More!

🤔 New to Appwrite?

📱 Phone Authentication is Here!

📀 Multiple Databases For Each Project

🎉 New $createdAt & $updatedAt Attributes

🔐 Webhook Signatures

🔮 New OAuth Provider for Dailymotion and Autodesk, Gitlab Got an Upgrade

🥳 And More!

⏭ What’s Next?

Can Appwrite handle 25 Million documents?

It's Here! Announcing the Release of Appwrite 0.12!

Eldad A. Fux for Appwrite ・ Jan 5 '22

🏁 Benchmark

📝 Collection

⚙️ Request

🔬 Results

📚 Learn more

Announcing Appwrite Web SDK 5.0

⚙️ Setup

👥 Account

📀 Database

📚 Learn more

Lessons learned from building a WebSocket server

It's Here! Announcing Appwrite 0.10 and the new Realtime API!

Eldad A. Fux for Appwrite ・ Aug 31 '21

Why did we build a Realtime API?

Architecture

Data Flow

Data Structure

Stumbling Blocks

Change of Permissions

The Operating System

Asynchronous Delivery

Authentication with Cookies

Takeaway

Credits

Introducing Magic URL Login to Appwrite

⚙️ Setup

🎩 Create a Magic URL

🔐 Login with a Magic URL

🏁 Conclusion

🔖 References

Introducing the New Appwrite Anonymous Login

Why do we need anonymous users?

Setup

Web

Flutter

Create an Anonymous User

Web

Flutter

Convert to Permanent User

E-Mail

Web

Flutter

OAuth2

Web

Flutter

Conclusion

References

Appwrite's Isomorphic Web SDK

Breaking Changes

Isomorphic

`isNull`

`isNotNull`

`between`

`startsWith`

`endsWith`

`select`

🎉 New `$createdAt` & `$updatedAt` Attributes