Forem: Teleport

Plans for Teleport #Part-5

Teleport — Mon, 11 Jan 2021 07:38:02 +0000

P2P is nice, it's free right?

From the video, P2P starts off being connected to a central broker server which helps negotiate network details to connect. Once the network details have arrived, each peer establishes a P2P pipe to the other peer and cuts its dependency with the broker server. It's like the 'chicken and the egg' problem - You have to be connected with a server to get a serverless connection XD.

So we need a highly available WebSocket server to be hosted in the cloud and this does not come for free. For now, Teleport's WebSocket server is hosted in a single click through Digital ocean's app platform with the free credits provided from DO. P2P works most of the time, but let's say in 20% of the cases direct if the users live under a complex network setup or symmetric NATs, direct P2P might fail. As a backup, we need a turn server, think of it as a central router that helps in delivering the files if P2P fails, which does not store the files, just routes it to the receiver. This is not free again as turn servers come at a cost. Most of the open-source projects above don't have a turn service for the same reason, leaving a section of users unattended.

Regardless of winning the hack or not, Teleport will be free to use forever in terms of unlimited file transfer, clipboard share and CLI(excluding premium features that would need funds). Since real money is involved, we have no plans as of now to run Teleport opensource. There are multiple reasons for this. Before Teleport, we used tools like Snapdrop or Sharedrop. We often just love the problem it solves and wants to use it but don't have time to look into the code and contribute. Most of the issues in Snapdrop can be summed up as it works in X device but not in the Y device and it works in this X browser and not in Y browser, in this platform and so on.

Implementing a new feature or even a UI design change always breaks something somewhere because of webRTC in some other browser and API's we use don't have the same behavior/support in all platforms. This problem is directly connected to having a comprehensive automated selenium test suite in BrowserStack and obviously. BrowserStack doesn't come for free as well it's around 26$/month. Not only that, the dev time for unpaid work, it is also pretty huge as it took 6 months to get a POC pr to be raised for a really important issue posted 7 months ago. This is perfectly explained by Havoc Pennington in his article Open source has a working-for-free problem, where we particularly endorse the quote.

We as developers will put our blood and sweat into building more features for Teleport for our users in record time and provide a reliable and quality service for individuals and organizations. We have great plans for Teleport in solving many problems in many areas not only as a personal file transfer tool but the mission is also to easily move files from point A to point B by ensuring security and privacy at both ends. This could be solving a particular problem in a video production company, working remote and sending videos instantly to peers for reviews, etc. We have discovered potential is from India where the education system still requires a photocopies for every single assignment, project or document from schools to colleges. Teleport can eliminate the usage of USB's in these shops which inturn prevents malicious files/virus being injected to the printer machine. This also introduces the convenience where users can pay online and schedule their photocopies or just scan a QR code in the shop to drop the files directly P2P via Teleport to the printer machine. We have also want to customize the Teleport to provide them some unique features including one were once they click the leave room button, the file is automatically erased from the Desktop which performed the photocopy to ensure there is no misuse of the documents.

There are a lot of places like these where Teleport can change traditional ways of transferring files. But we want this to be a tool that is accessible to a wider range of audience, like where we were using free open source tools. We will be moving/developing exclusive or existing premium features like remembering portal links, private portals, persistent push notification convenience(even if users close the app, they could join a portal in a single click from their notification bar) and more in the near future to reward the users who pay for us through the business plan. We aim at running it at a very reasonable and affordable price, enough to help us run Teleport with a smile on our face.

Our vision for Teleport is that 'Teleport me the files' gets added to people's vocab and becomes the new norm with constant help/support from the community. That's all folks!

Teleport's alpha version is up at [teleportme.in][https://teleportme.in], we have been very transparent with everything we did and we hope people try it and give us feedback/suggestions to improve. Thank you!

Technical Breakdown ⚒️ #Part-4

Teleport — Mon, 11 Jan 2021 07:35:25 +0000

We have gone through all the flaws in the existing solutions and innovated the following features to help address them. This article provides a deep breakdown of all the features.

🎯Dynamically switches between topologies

Teleport can dynamically switch its topology which makes it first of its kind. Why is this switch necessary? How does it make it better? Before answering these let's know the pros and cons of existing individual topology.

File streamed directly from sender	File shared using Web torrent API
All the receivers request files from only the sender and data is streamed via Webrtc.	It allows a torrent like a download in the browser where every peer helps each other to download files
Pros	Pros
Zero wait time. Since there is no pre-processing, file transfer begins the moment the connection establishes.	Scale as receiver list increases. For every new peer added it can receive files from n-1 other receivers.
No file limit, since file is streamed(broken into chunks, only that is loaded in main memory and sent). Perfect if users want to send medium / large files (files greater than 100MB) from their phones to PC. The receiver count is only 1, why use a torrent network and pay for the CPU processing + waiting time when all you want is to send a file to one or two peers.	Download rate increases with rising in peers.
Cons	Cons
Does not scale as the receiver list increases. At one point there is a throttle on the sender side as it has to cater to too many requests and fails. Won't work for a large no of users.	Considerable waiting time according to the CPU, mobile phones suffer a lot with webTorrent than a PC. Before a torrent can come into action, there is a pre-processing step known as *torrent creation time.* This varies based on file size and processing power. For an estimate, it takes more than around 2mins to create a torrent for a 500Mb file on OnePlus 7T which has a considerably good CPU for a mobile phone
The download rate reduces since only the sender shares the files with receivers.	It has a limit on file size. For the seeding process, the file should be loaded into main memory, since it is a web-app that runs in a browser. So for any file more than 700MB when tried to send a file in a phone browser, browser crashes occur due to out of memory reasons. Even low-end PC's which have a pretty bad CPU suffers a lot of waiting time, not to mention sending a file greater than 2GB in pc is not possible with web torrent as the entire file has to be loaded into your ram.

The hybrid approach towards this is what makes Teleport scalable and gets the best of both worlds. Initially, when file-sharing is initiated, the files are shared from sender to receivers through streaming mode(zero wait time) while torrent-creation is parallelly operated in a worker thread. If the peer count increases let's say more than 2 users and the worker thread has completed the torrent-creation, Teleport shifts to the torrent network where peers help each other. Existing peers who have downloaded files partially using streaming mode download what's left using the torrent protocol( this feature of memoizing chunk stores is something we added to web torrent's internals). Also, these peers seed their data, thereby helping newly joined peers download from existing peers and new peers to come. Thus we solve the initial waiting time faced in the torrent network by streaming the file from the master node and the receivers gracefully switch to web torrent at the right time through a hot memoization swap. We have our own forked modified version of webTorrent to achieve this.

How is this performant?

Assume 3 users are trying to download a 400MB file through their mobile devices, in a typical webTorrent scenario it takes 1.30 to 2mins waiting time and maybe another 1.30min to download from each other. With teleport, users receive 40% percent of the file in streaming mode instantly before the torrent creation time ends (1.30min). So now we download only the rest of the pieces in the webTorrent by swapping the new Store initialization with our existing partially downloaded store. Because of this web torrent doesn't start with 0% from scratch rather verifies the existing store with the hashes and moves on to download only the rest of the pieces which gets downloaded in 40 seconds. We save 50 seconds of transfer through this.

🎯Smart Environment Sensing

The torrent switch happens only if the room has more than 2 peers, otherwise, it works in streaming mode peacefully perfect for the phone to pc scenario mentioned above
If the file size is greater than 2GB, and a portal is created in the browser. We never initiate or use the torrent network rather just use the streaming mode. Take advantage of the hybrid topology for files greater than 2GB, by using teleport's CLI as we don't have any limitations of the browser there.
If a specific portal has switched to torrent mode, and let's assume the file size is greater than 700MB and a new phone joins, teleport knows that the phone won't be able to participate in seeding the file cause of main memory limitations. The phone now becomes a leecher and downloads the file directly to disk.

leecher - a peer who downloads from everyone but will not contribute back by uploading.

🎯Secure Private Portal Transfer with Authorization

In public mode, the portal is exposed to all peers in the local network. Anyone can join the portal. Whereas in the private mode, we provide the user with a personalized profile of settings and some extra features.

Firstly, the room name is updated to prevent earlier users from entering the portal.
Secondly, peers entering the portal get authorized via Google and since we enable the receiver prompt by default only desired people can enter the room.
Finally, these portals are not discoverable in the local network, meaning it provides anonymity to the sender.

Teleport - DO Hackathon submission #Part-1

Teleport — Mon, 11 Jan 2021 07:29:39 +0000

💡 What I built

A fully decentralized cross-platform P2P file sharing tool with unlimited capacity. Send files/clipboard across and between CLIs and browsers. Between? Yep start a file transfer in the CLI and receive it in your phone's browser! The CLI and the browser are just a frontend to teleport's core and they can connect between themselves or between each other. Teleport is a productivity tool built for users with security, privacy, and convenience in mind.

Category Submission: Built for business

📹 App Link : teleportme.in

🎬 Video demo :

📸 Screenshots

👩‍💻 Description

Why build Teleport?

Everyone loves Apple's Airdrop, a tech that allows users to drag files onto devices in your local network/wifi. The problem with Airdrop is that it is only available for the Apple ecosystem. We envisioned a tool that could be as cool and convenient as Airdrop but had no boundaries like your device or platform. You could just transfer a file from your Mac to Windows, Windows to Mac, Mac to Android, and so on, making it truly cross-platform. What if this file-sharing mechanism worked beyond your wifi? Airdrop does it with Bluetooth and under your wifi, so we wanted to achieve something like that on the world wide web using WebRTC for remote peers. We started working on this P2P file sharing client that is going to make everyone's lives easier.

Features

Here is the list of features that sets us apart from other traditional file-sharing tools :

Dynamically Switching Topologies
Local Network Detection
Unlimited Files Capacity
Unlimited peers - When peers grow teleport goes into the torrent mode
Secure Private Portals
Automatic Recommendation of Previous Portals
Real-time CLI Support
Clipboard and Text Content Sharing
PWA Support
Smart Environment Sensing
Highly Customizable
Cross-platform Support
Optimized for Instant Rapid File Transfer

📜 Link to Source Code

Source Code

🔒 Permissive License

Common Clause

🛤 Background

We have been working with WebRTC and file-sharing algorithms for more than a year. After experimenting on some small projects we decided to implement a P2P video streaming application in VR for Facebook’s Developer Circle Community Challenge in 2019. We were elated to win first place in India(regionals)🎉. This gave us the impetus to continue updating ourselves with all there is to know in the field.

Being undergraduates who constantly competed at hackathons or were involved in pair programming for projects related to our course, we often found it inconvenient to transfer files or simple code snippets. Moreover, things like links and clipboard content also required the use of apps like WhatsApp or Slack because google meet & zoom truncates text and formats code (removes linebreaks, etc) weirdly when sent through them. We wanted to build a tool that would help us achieve all this without having to leave the development environment from our VS Code's integrated terminal.

So we moved ahead to create Airshare an open-source python CLI based file client limited to the local network. It was released in PIP as a module that has over 15K downloads and is now available for use @ homebrew.

Although we got a good response from the community, there is always room for improvement. This inspired us to build a new file-sharing tool. Like any other project, we dedicated ourselves to explore all the currently existing solutions in the field and created Teleport consequently.

👩‍💻 How we built it

Digital Ocean App Platform made it very easy and reliable to host our website and server in one place retaining complete control over our sites underlying infrastructure. All this was done within a few clicks and made our life simpler.
Also, it was fascinating to see that it could automatically update our project when we push changes to our code which would help us concentrate more on upgrading and building on teleport without worrying about the deployment and scaling.

Thanks to Digital Ocean for making our work easier and as a developer, I would love to continue using Digital Ocean in my future projects.

🧾 Additional Resources/Info

🎥 Demo
🎬 Walkthrough
🌐 Website

Features of Teleport #Part-3

Teleport — Mon, 11 Jan 2021 07:25:41 +0000

Teleport

Teleport has a variety of features that make it the next best tool for file transfer.

🎬 Walkthrough of teleport and its features:

Dynamically Switching Topologies

Unlike traditional file sharing clients which have a fixed topology that either works on a streaming mode like sharedrop/snapdrop or uses webTorrent's API like blaze, Teleport can switch its nature dynamically which makes it first of its kind. We built a hybrid module that can dynamically switch between topologies to provide the best possible file transfer mechanism. This helps solve two major problems in the existing solutions -

Traditional bandwidth issues when more peers join
Zero wait time

CLI to Browser Synergy 🖥️ ❤️ 🌐

This is one of the primary features that allows us to transfer files across CLIs and browsers seamlessly thus making the tool truly accessible to regular users and developers alike. We built a npm module teleport which can be installed as a CLI tool. Now developers can send files from CLI to any browser and receive both file and clipboard content from browsers without exiting their development environment.

CLI Usage 🖥️

You can send and receive files and text/clipboard content using Teleport's npm module. Once you start Teleport in the CLI, it listens to portals created local rooms and dynamically refreshes on its own. This prevents users from closing and opening the tool again to see the changes.

Local Network Detection 🖧

Teleport was built to provide productivity and convenience to every user and this feature helps deliver this to them. When a portal is created, other users in the local network are notified of the existing portal. This helps users directly connect to the portal without having to share their portal codes.

Automatic Recommendation of Previous Portals 📝

Once you join a peer's portal, their user name and portal code is saved to a suggestions list. During future interactions to connect with the same peer's portal, the name of the user can be used to recommend the previous peer's portal code.

Secure Private Portals🕵🤫

Sharing confidential files is possible using private portals which enables the sender to validate the authenticated receivers before entering the portal. Furthermore, private portals are invisible to other peers in the local network.

Clipboard and Text Content Sharing 📋

Apart from files, Teleport can also be used to transfer text and clipboard content. We also provide auto clipboard copy feature which copies the clipboard content directly to the receivers, ensuring that a prompt doesn't hinder your productivity.

PWA Support 📱

Teleport is also available as PWA to emulate a native app environment for mobile users. Here is a list of the features available in PWA,

Integrated WakeLock API to prevents device screens from dimming or locking during file share to get a stable transfer speed.
Added Web Share Target to share any file directly from the gallery(or any native apps) via Teleport without actually opening the app.
Implemented push notifications that notify the user of certain events while the app runs in the background.

Highly Customizable ⚙️

Our main goal is to boost productivity and provide convenience to our users. Hence we have provided a wide variety of options as shown in the diagram below.

Smart Environment Sensing 💡

Sending files from one device to another always has many complications. We have successfully created our module to intelligently detect the environment and optimize the file transfer mechanism. All this is abstracted from the user to facilitate a rapid file transfer.

For more insights on Teleport and technical breakdown of various features and different use cases, read our follow up post, ' Technical Breakdown ⚒️'.

Exploring Existing Solutions🔍 #Part-2

Teleport — Mon, 11 Jan 2021 07:24:06 +0000

Several existing tools provide file-sharing capabilities:

WeTransfer

What it does?

To move a file from point A to point B, organizations that use 'WeTransfer' wait to upload their files to weTransfer(third party). This is then transferred to the clients by weTransfer through email. The receivers get an email to download the file from weTransfer's server

Problems:

Files are stored in WeTransfer servers which is a huge privacy concern for organizations.
Restricted file size for free and pro users.
Does not support instant file transfer, needs to be uploaded to the cloud before anyone can receive the files

weTransfer

Persistence is not required in most use cases because most of the clients download the resources almost instantly or within that specific working day. Thus P2P makes more sense in this scenario as there is no need to upload. It's an instant share and the file is never sent to any third-party server. Thus making it efficient and inexpensive.

Opensource solutions are really amazing; people have taken the time to build innovative projects. The problems with the existing solutions are already listed in the GitHub issues of each project. For instance, snapdrop doesn't work for remote peers and works only on LAN. If the user is planning to upload a file in a conference and send it to 50 - 60 users, the user has to individually click on every peer and send it which is not convenient. Moreover, files cannot preexist before the peers join and users can't send files to multiple devices at once.

Most importantly snapdrop and sharedrop use (1-N) streaming protocol as illustrated below:

The master who initiates the transfer performs the transfer to all clients. This becomes highly inefficient when the peer size and file size grow because one peer gets overloaded with requests and bandwidth gets significantly narrow. To counter this scenario, blaze used webTorrent's API which makes sure everyone who receives the file helps each other to download the file like a torrent (N-N).

Unfortunately in webtorrent, files are loaded in the memory which prevents the upload of files greater than 600Mb in a phone and a hard limit of 2GB for computers. This setback is attributed to the fact that the file has to exist in the ram. Choosing one topology over the other has its own merits and demerits. Choosing the streaming mode makes sure an unlimited file size and high performance when using mobile devices. Using webtorrent below 600Mb is advisable because the transfer speed is insane when multiple peers join the system which becomes performant and efficient but has a hard limit and torrent creation wait time. To put it in a single sentence, webTorrent is not mobile-friendly as it imposes a significant load on the CPU and depends on the main memory.

So seeing the existing problems we thought, what if we can combine both topologies, use the right topology at the right time, graciously switch progress between both topologies and use them efficiently?

What if we can provide complete synergy between CLI and the browser?

As developers ourselves, we spend most of our time using CLIs and having the ability to transfer files from your CLI(like Airshare) to any other platform on any device would help boost productivity and provide convenience.

Now we know what exists, let's see in the next part what teleport 👽 has!