<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>Forem: Mohamed Roshan</title>
    <description>The latest articles on Forem by Mohamed Roshan (@mroshank).</description>
    <link>https://forem.com/mroshank</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F3086627%2F046ab36a-a785-4304-b564-b8239cb1891d.png</url>
      <title>Forem: Mohamed Roshan</title>
      <link>https://forem.com/mroshank</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://forem.com/feed/mroshank"/>
    <language>en</language>
    <item>
      <title>Cant play game cos of Ads so I made my own Game - Ai Studio</title>
      <dc:creator>Mohamed Roshan</dc:creator>
      <pubDate>Sat, 29 Nov 2025 10:48:09 +0000</pubDate>
      <link>https://forem.com/mroshank/cant-play-game-cos-of-ads-so-i-made-my-own-game-ai-studio-3i1d</link>
      <guid>https://forem.com/mroshank/cant-play-game-cos-of-ads-so-i-made-my-own-game-ai-studio-3i1d</guid>
      <description>&lt;p&gt;I recently downloaded the &lt;strong&gt;Ball Sort&lt;/strong&gt; game which was infested with ads for every move and level. Got me really frustrated which made me think &lt;strong&gt;why not vibe code one&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Opened up my &lt;strong&gt;Ai Studio&lt;/strong&gt; and started giving different prompts about the design preferences and rules I wanted and unlike other vibe coding platform this time ai studio followed all of my instructions and after a few iterations back and forth I got my very own game &lt;strong&gt;ChromaSort&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fivp0bxva82d8yzb25zkn.jpeg" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fivp0bxva82d8yzb25zkn.jpeg" alt="google ai studio" width="800" height="403"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;It was pretty straight forward and took me a few weeks to complete as it was just a fun lil side project. Also the game additionally has no login nonsense or anything. Your progress is simply cached on your device, unlike a lot of these tiny casual games that somehow still want your email.&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;The stuff that makes it cooler : I made sure no two players ever get the same color combo every level is completely random. That way, I can come back anytime and still get a fresh challenge.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2F611g4v2wp9gd95cebuya.jpeg" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2F611g4v2wp9gd95cebuya.jpeg" alt="Game interface" width="800" height="526"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  What about hosting it ?
&lt;/h2&gt;

&lt;p&gt;I have my own e2 micro vm that handles all my fun lil side projects along with my portfolio website. I have set up a Github Action pipeline for deployment + CloudFlare as a proxy and Nginx to handle different routes within the VM hehe.&lt;/p&gt;

&lt;p&gt;Check out &lt;a href="https://chromasort.mroshank.me/" rel="noopener noreferrer"&gt;ChromaSort&lt;/a&gt;&lt;/p&gt;

</description>
      <category>ai</category>
      <category>gemini</category>
      <category>vibecoding</category>
    </item>
    <item>
      <title>This is so real, IS IT AI ?? - How to identify AI generated images</title>
      <dc:creator>Mohamed Roshan</dc:creator>
      <pubDate>Fri, 28 Nov 2025 15:24:36 +0000</pubDate>
      <link>https://forem.com/mroshank/this-is-so-real-is-it-ai-how-to-identify-ai-generated-images-3j3c</link>
      <guid>https://forem.com/mroshank/this-is-so-real-is-it-ai-how-to-identify-ai-generated-images-3j3c</guid>
      <description>&lt;p&gt;Since the release of Gemini 3 and Nano Banana Pro, AI-generated images have become insanely realistic and it’s only going to get better to the point where we won’t even be able to tell what’s real anymore&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fwhmo41kieanchmz79l6h.webp" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fwhmo41kieanchmz79l6h.webp" alt="Im Okay meme" width="500" height="280"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;But dont panic we got a way to differentiate (kinda) AI images created using GEMINI from the real ones humans click.&lt;/p&gt;

&lt;p&gt;We can use Google Lens and do a reverse search or use the Gemini app itself and ask if it AI generated or not&lt;/p&gt;

&lt;p&gt;Dont get me wrong it works (obviously exceptions exists)&lt;/p&gt;

&lt;h2&gt;
  
  
  How is it done ?
&lt;/h2&gt;

&lt;p&gt;Lets take this hyper realistic image I created using Gemini 3&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Foex69sbmwov3q06ssm8t.jpeg" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Foex69sbmwov3q06ssm8t.jpeg" alt="Image created using Nano Banan Pro" width="800" height="463"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Now lets do a reverse search on that &lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fj3plx8s0y5yyg2haio16.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fj3plx8s0y5yyg2haio16.png" alt="Reverse search image on Lens" width="627" height="1072"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;After uploading it on Lens let's click on &lt;strong&gt;About this image&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;AND VOILAA&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2F7rm6hdmkf20s9qgnn6w0.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2F7rm6hdmkf20s9qgnn6w0.png" alt="Result on Lens" width="591" height="917"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Additionally you can use the very own GEMINI you have used to create the AI image, just ask &lt;strong&gt;halo, is this ai ?&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fcl6rpmfgf4bprr3n3bzh.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fcl6rpmfgf4bprr3n3bzh.png" alt="Asking Gemini if its AI" width="592" height="993"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;If you had noticed, it's using SynthID to identify if the image is AI or not and not the visible watermark at the bottom&lt;/p&gt;

&lt;h1&gt;
  
  
  Tech behind the identification and it's limitation
&lt;/h1&gt;

&lt;p&gt;The main question, Whats &lt;em&gt;SynthID&lt;/em&gt; ?&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;&lt;p&gt;SynthID is an invisible digital watermark from Google DeepMind that's embedded into all the images created using Gemini.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;So when you ask Gemini or do a reverse search on lens they just look up the synthID present in the image and comes to a conclusion if its an AI generated or not &lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;The watermark is kinda robust and will stay intact against minor  to moderate edits, resize or compression.&lt;/p&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;And the last point from other POV gives you the limitation, if youre someone who knows how to edit the image well enough then there’s a good chance Gemini won’t be able to detect the SynthID watermark anymore.&lt;/p&gt;

&lt;p&gt;Follow me on &lt;a href="https://x.com/mroshanka" rel="noopener noreferrer"&gt;X&lt;/a&gt; for more cool updates and tech/infra related stuff &lt;/p&gt;

</description>
      <category>ai</category>
      <category>gemini</category>
      <category>nanobanana</category>
      <category>googlecloud</category>
    </item>
    <item>
      <title>Hosting OpenAI CLIP on GKE</title>
      <dc:creator>Mohamed Roshan</dc:creator>
      <pubDate>Wed, 01 Oct 2025 18:05:20 +0000</pubDate>
      <link>https://forem.com/mroshank/hosting-openai-clip-on-gke-4i47</link>
      <guid>https://forem.com/mroshank/hosting-openai-clip-on-gke-4i47</guid>
      <description>&lt;h2&gt;
  
  
  So what is this CLIP ?
&lt;/h2&gt;

&lt;p&gt;CLIP (Contrastive Language–Image Pretraining) basically is a model from OpenAI that can look at an image and a piece of text and figure out how well they match. It is kind of like a search engine for images, where you can give a description and the model figures out  which given image is the closest to it. &lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;We have different sizes of CLIP released by OpenAI and we can run it on just CPU than being dependent on GPU but as you've guessed there are trade offs.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;h2&gt;
  
  
  Hosting CLIP on GKE
&lt;/h2&gt;

&lt;p&gt;Since CLIP isnt generally available as a chat app or similar sort like the traditional AI tools, I decided to host it.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Host it where ?&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;A normal VM works for testing but let's go brrr and spin up some pods&lt;/p&gt;

&lt;h2&gt;
  
  
  Setting up GKE
&lt;/h2&gt;

&lt;p&gt;I have IaC handy that controls all the infra for my personal project, so i added a gke and did the &lt;code&gt;terraform apply&lt;/code&gt;&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Ffsd75eebp3qt4tluv1vx.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Ffsd75eebp3qt4tluv1vx.png" alt="Iac SS" width="659" height="452"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fk5zi5fvob7phlhu0vv64.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fk5zi5fvob7phlhu0vv64.png" alt="tf apply" width="800" height="203"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;I'm gonna be using &lt;code&gt;e2-standard-2&lt;/code&gt; for this example and I think it meets the minimum requirement for our model &lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;Now we have the cluster -&amp;gt; lets set up a script to handle the endpoint for the model &lt;/p&gt;

&lt;h2&gt;
  
  
  Fastapi and Docker
&lt;/h2&gt;

&lt;p&gt;So we need an endpoint to talk to the model and Im using fastapi for that. Let's wrap the CLIP with an endpoint like &lt;code&gt;/predict&lt;/code&gt; where we be sending the image with different texts and it sends back similarity score as response &lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;similarity score is range from -1 to 0 to 1, closer to 1 is how close the description is with the image&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;Once the script is ready (link to &lt;a href="https://github.com/roshan0099/clip-gke" rel="noopener noreferrer"&gt;repo&lt;/a&gt;) we can now start the containerizing process &lt;/p&gt;

&lt;p&gt;if we properly perform the multiple staging containerization we can bring the image size to around ~1.1GB  or maybe less &lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Faes5q3ia8lrigor1ahc2.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Faes5q3ia8lrigor1ahc2.png" alt="Docker image size" width="800" height="67"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;You can read &lt;a href="https://dev.to/mroshank/know-what-u-want-dockerfile-7gb-to-1gb-4ldm"&gt;here&lt;/a&gt; how I reduced the container size &lt;/p&gt;
&lt;/blockquote&gt;

&lt;h3&gt;
  
  
  Push the image to artifact
&lt;/h3&gt;

&lt;p&gt;Let's push the image to the artifact, it will be easy to pull and build the image later on the pod&lt;/p&gt;

&lt;p&gt;Let's create an artifact registry repo &lt;/p&gt;

&lt;p&gt;Adding the below to the IaC&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fzj70ojlnlb3znmcfrpq7.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fzj70ojlnlb3znmcfrpq7.png" alt="terraform file" width="543" height="181"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2F6e7agt7cozrw9nxhhsoy.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2F6e7agt7cozrw9nxhhsoy.png" alt="Repo" width="800" height="230"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Now we containerized the script and we have the artifact repo ready, et's write a deployment yml for our K8&lt;/p&gt;

&lt;h2&gt;
  
  
  Kubernetes -&amp;gt; GKE
&lt;/h2&gt;

&lt;p&gt;So as you know we need a &lt;code&gt;deployment.yml&lt;/code&gt; and &lt;code&gt;service&lt;/code&gt; to get our app to the public, for this demo we gon be using load balancer &lt;/p&gt;

&lt;p&gt;I don't think we are going to have that much of a load, so let's humble ourselves and just go with one replica &lt;/p&gt;

&lt;p&gt;You can find the yml file &lt;a href="https://github.com/roshan0099/clip-gke/blob/main/k8/deployment.yml" rel="noopener noreferrer"&gt;here&lt;/a&gt; &lt;/p&gt;

&lt;h2&gt;
  
  
  CI/CD pipeline with Github Actions
&lt;/h2&gt;

&lt;p&gt;So the idea is to build a pipeline that would build the image and push the latest one to the artifact and then we spin up a pod and pull the image from the artifact and run&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;build image -&amp;gt; artifact -&amp;gt; spin up pod -&amp;gt; pull the image -&amp;gt; run -&amp;gt; expose it to the world 

&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;pretty straight forward.&lt;/p&gt;

&lt;p&gt;You can find the yml for pipeline &lt;a href="https://github.com/roshan0099/clip-gke/blob/main/.github/workflows/deploy.yml" rel="noopener noreferrer"&gt;here&lt;/a&gt; &lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fqil8e7bfsnagbr4vttne.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fqil8e7bfsnagbr4vttne.png" alt="CI/CD Result" width="800" height="273"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  Issues I faced
&lt;/h3&gt;

&lt;ul&gt;
&lt;li&gt;The usual credential permission stuff in the pipeline &lt;/li&gt;
&lt;li&gt;Image not getting pushed to artifact partly cos of the above related issue&lt;/li&gt;
&lt;li&gt;Pod getting choked as the vm was intially e2-micro :) &lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;But hey we finally got it running .. .. .. &lt;/p&gt;

&lt;h2&gt;
  
  
  Final
&lt;/h2&gt;

&lt;p&gt;The model worked as expected, I mean with the advance ai models we have at disposal this might not come across that fascinating but still consider the time it was released (2021) it's cool and there's still a lot of stuff we can do with this and if all you need is to match text with images without burning a hole in your wallet, CLIP is still a solid little workhorse.”&lt;/p&gt;

&lt;p&gt;Lets see some examples and metrics now 😼 &lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fphtn710b36pe60ad3rz6.jpg" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fphtn710b36pe60ad3rz6.jpg" alt="cat" width="800" height="1068"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;was thinking what the above image is and just asked clip, is that a cat, dog or human ? 😳&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fof148flm7x5np4ssqdj2.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fof148flm7x5np4ssqdj2.png" alt="clip example" width="800" height="154"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;as you can see cat got 0.9 similarity making it cat (looks like cat is more similar to human than to dog) &lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;Dont spam the ip and choke my cluster &lt;br&gt;
&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fexghzkfzcqw6td8zjttc.gif" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fexghzkfzcqw6td8zjttc.gif" alt="sus kid meme" width="220" height="220"&gt;&lt;/a&gt;&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;Okay now to the metrics --&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2F6dzi8l8869kbgc19cfdv.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2F6dzi8l8869kbgc19cfdv.png" alt="metrics" width="800" height="263"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fbqet13dapt5r70rv1dwz.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fbqet13dapt5r70rv1dwz.png" alt="gke dashboard" width="800" height="324"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;&lt;em&gt;TODO : Adding monitoring/observability dashboard to this, preferably Prometheus+Grafana&lt;/em&gt;&lt;/p&gt;
&lt;/blockquote&gt;

</description>
      <category>ai</category>
      <category>openai</category>
      <category>kubernetes</category>
      <category>cloud</category>
    </item>
    <item>
      <title>Know what you want - Dockerfile 7GB to 1GB</title>
      <dc:creator>Mohamed Roshan</dc:creator>
      <pubDate>Mon, 15 Sep 2025 21:25:47 +0000</pubDate>
      <link>https://forem.com/mroshank/know-what-u-want-dockerfile-7gb-to-1gb-4ldm</link>
      <guid>https://forem.com/mroshank/know-what-u-want-dockerfile-7gb-to-1gb-4ldm</guid>
      <description>&lt;p&gt;Encountered the classic big image issue when I built a docker image with multi staging.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fg7o5tp08w09hy0gc0qtk.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fg7o5tp08w09hy0gc0qtk.png" alt="dockerfile" width="800" height="422"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Coming to what i was trying to achieve, I had this idea of hosting openai CLIP model for fun on gke so for that i was containerizing the following dependencies and app file, my requirement initially was&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;torch
torchvision
ftfy
regex
tqdm
fastapi
uvicorn
pillow
numpy&amp;lt;2
python-multipart

&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;A lil bit of context here, the idea is to have the CLIP model that runs on CPU and doesn't need any CUDA/GPU support. So this &lt;code&gt;requirement.txt&lt;/code&gt; installs the torch related packages with default GPU/CUDA which is not required and this made the image size to be 7gb &lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Flbq0j07vidkshv3sjnpd.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Flbq0j07vidkshv3sjnpd.png" alt="clip image" width="800" height="27"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;So like any other person I dug into it cos i was expecting the size to be around 2gb max. I found about the unwanted CUDA/GPU stuff being added while installing the torch packages. &lt;/p&gt;

&lt;p&gt;So I updated the &lt;code&gt;requirement.txt&lt;/code&gt; and mentioned that I just want CPU support and that just did it.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;torch==2.2.0+cpu
torchvision==0.17.0+cpu
-f https://download.pytorch.org/whl/cpu/torch_stable.html
ftfy
regex
tqdm
fastapi
uvicorn
pillow
numpy&amp;lt;2
python-multipart
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;now the image size got drastically reduced to ~1GB from `~7GB&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fbx6unhh7deap9s14rto1.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fbx6unhh7deap9s14rto1.png" alt="clip image optimised" width="762" height="97"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;and if you want you can further drill it down by removing the tests and example directories from the site-package and that will probably save u 50mb ~ 60mb&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2F2gzb6eb296xhxxo3u0za.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2F2gzb6eb296xhxxo3u0za.png" alt="dockerfile optimised" width="800" height="335"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;and now the size went from 1.07GB to 1.01GB&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fzve7yknl2zabzmiw5kmm.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fzve7yknl2zabzmiw5kmm.png" alt="clip image light" width="800" height="67"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;In fact you can go aggressive and might even reduce more if you're individually copying the packages from &lt;code&gt;site-packages&lt;/code&gt;&lt;/p&gt;

</description>
      <category>docker</category>
      <category>containers</category>
      <category>python</category>
      <category>devops</category>
    </item>
    <item>
      <title>Vibecoded a Game using AmazonQ CLI</title>
      <dc:creator>Mohamed Roshan</dc:creator>
      <pubDate>Sat, 31 May 2025 13:19:42 +0000</pubDate>
      <link>https://forem.com/mroshank/vibecoded-a-game-using-amazonq-cli-n3m</link>
      <guid>https://forem.com/mroshank/vibecoded-a-game-using-amazonq-cli-n3m</guid>
      <description>&lt;p&gt;I was checking out some AI tools out there and how efficient they are at building entire apps from scratch, literally anything. Then I came across Amazon Q and the new challenge to build a game using it.&lt;/p&gt;

&lt;p&gt;So, like anyone curious, I decided to give it a shot. Building a game just by prompting? Let’s see how far that goes.&lt;/p&gt;

&lt;p&gt;You can refer to the &lt;a href="https://docs.aws.amazon.com/amazonq/latest/qdeveloper-ug/command-line-installing.html" rel="noopener noreferrer"&gt;documentation&lt;/a&gt; for the download steps&lt;/p&gt;

&lt;p&gt;Getting started ..&lt;/p&gt;

&lt;h1&gt;
  
  
  Initial Phase - the idea
&lt;/h1&gt;

&lt;p&gt;I wanted to build a graphically rich game—something super simple that anyone could pick up and play, anywhere, anytime. The kind of game that keeps you going without overcomplicating things.&lt;/p&gt;

&lt;p&gt;So I decided to keep it basic and go with just &lt;strong&gt;HTML, CSS, and JS&lt;/strong&gt;. Now the big question was: &lt;em&gt;what exactly am I building?&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;I eventually landed on a shooting game, kind of like Fruit Ninja but instead of slicing fruits, you'd shoot shapes. Add some cool explosions and effects, and boom 💥 we've got something fun.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;&amp;gt; Create a game where i can click or shoot shapes (circle, rectangle, triangle so on) that fall from the top and for each click i should be getting a point.
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;and this gave me &lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fgzr36tc2qfas8ji48vnx.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fgzr36tc2qfas8ji48vnx.png" alt=" " width="800" height="498"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Very basic but yea, this is our base &lt;/p&gt;

&lt;h1&gt;
  
  
  Build Phase
&lt;/h1&gt;

&lt;p&gt;From here, I started adding a bunch of functionalities, rules and effects to make the game feel a lot more fun and polished. Also, I kept throwing prompts at Q to keep it thinking and building with me.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fhieb2yzzobiubs0cguco.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fhieb2yzzobiubs0cguco.png" alt=" " width="800" height="531"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;I gave the prompt and Q gave me the detail reports on the changes it made. When, why, where all answered &lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;Kept adding improvements and honestly I'm pretty impressed by Q&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fhxd1iu8ff07ooaf754yw.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fhxd1iu8ff07ooaf754yw.png" alt=" " width="800" height="396"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fh91vstp25dgahrfd8b02.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fh91vstp25dgahrfd8b02.png" alt=" " width="800" height="549"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;I added sounds, effects and kept tweaking going back and forth and Q just gave me exactly what I wanted.&lt;/p&gt;

&lt;p&gt;After multiple iterations, the game now looks like this:&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2F5l5mequxpiby68fiu1ss.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2F5l5mequxpiby68fiu1ss.png" alt=" " width="800" height="395"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fxay3ni2hsemg3mpbo74e.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fxay3ni2hsemg3mpbo74e.png" alt=" " width="800" height="395"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fya4hw27pe2k0s4cv31cu.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fya4hw27pe2k0s4cv31cu.png" alt=" " width="800" height="395"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Since this was built mostly using DOM and DOM manipulation, it had its issues. So, I decided to rework it by implementing everything using Canvas instead.&lt;/p&gt;

&lt;h1&gt;
  
  
  Re-implementation using Canvas
&lt;/h1&gt;

&lt;p&gt;The whole process went pretty smooth. I just asked Q to re-implement the game in Canvas and voila just like that Q did it.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;&amp;gt;reimplement the current game display and everything in canvas
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Obviously the first iteration had issues, so I picked each and asked Q to specifically fix them all and it just did it.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Finlzsjn3q2t79wcehtra.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Finlzsjn3q2t79wcehtra.png" alt=" " width="800" height="327"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Helped to clean up the files as well &lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2F2c3qihr9dex9jzelkusy.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2F2c3qihr9dex9jzelkusy.png" alt=" " width="800" height="396"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Pictures from the new one &lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fnm8gzfvytg86aoq2myc2.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fnm8gzfvytg86aoq2myc2.png" alt=" " width="800" height="396"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fwste2xl8c8pto774l3bb.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fwste2xl8c8pto774l3bb.png" alt=" " width="800" height="396"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fwslco2mi25j2517uwe88.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fwslco2mi25j2517uwe88.png" alt=" " width="800" height="396"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Check out the game here and do let me know your feedback &lt;a href="https://shape-shoot.vercel.app/" rel="noopener noreferrer"&gt;Shape Shoot&lt;/a&gt;&lt;/p&gt;

</description>
      <category>amazonqcli</category>
      <category>ai</category>
      <category>vibecoding</category>
      <category>cli</category>
    </item>
    <item>
      <title>Data transfer made easy - Rclone</title>
      <dc:creator>Mohamed Roshan</dc:creator>
      <pubDate>Sat, 17 May 2025 10:02:09 +0000</pubDate>
      <link>https://forem.com/mroshank/data-transfer-made-easy-rclone-2m3f</link>
      <guid>https://forem.com/mroshank/data-transfer-made-easy-rclone-2m3f</guid>
      <description>&lt;p&gt;Let me get straight to it, I used to transfer data from Files.com or any other platform where files were dropped to our cloud buckets using scripts. It was all okay when the file sizes were within a few MBs, but things got painful once they grew into GBs. It started taking a lot more time.&lt;/p&gt;

&lt;p&gt;To speed things up, I tried running the transfer on a VM, it did get faster, but not &lt;em&gt;faster faster&lt;/em&gt;, especially when the size crossed 400+ GB.&lt;/p&gt;

&lt;p&gt;That’s when I started looking for a better way to connect my GCP/AWS buckets directly with these storage platforms, something that could make the transfer process faster and more reliable. And that’s where rclone came into the picture.&lt;/p&gt;

&lt;h1&gt;
  
  
  Rclone
&lt;/h1&gt;

&lt;p&gt;I have set it up on my vm as a job that runs the backups/transfer with ease&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;sudo apt update 

curl https://rclone.org/install.sh | sudo bash

&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;blockquote&gt;
&lt;p&gt;the usual installation process, once done with is lets set up the config, this is the place where we mention the details of the storage place where we transferring the data from and to&lt;br&gt;
&lt;/p&gt;
&lt;/blockquote&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;rclone config
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;blockquote&gt;
&lt;p&gt;gonna throw options to set remote &lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fl612jymew7hw1q6b8oxp.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fl612jymew7hw1q6b8oxp.png" alt="rclone options" width="460" height="176"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;from here it will take you to a bunch of storage platform options supported by rclone that can be used to mount&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fswyooybu9gx7ajpoj59y.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fswyooybu9gx7ajpoj59y.png" alt=" " width="800" height="884"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;choose the one that's preferred, I used files.com and gave it a name which will be used to refer later on, did the auth using api here.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fepsljgab68ng65giulzd.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fepsljgab68ng65giulzd.png" alt=" " width="800" height="497"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;PS : You might not find the api option right away so wait for the edit advanced config option&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2F3bmfzzh65trnzpw4tm6y.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2F3bmfzzh65trnzpw4tm6y.png" alt=" " width="800" height="112"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;now we are done with one remote, moving on to the next, follow the similar steps as the first one. rclone config -&amp;gt; new remote -&amp;gt; pick the one you want and provide the auth method. I have gone for GCS bucket here, mentioned the project number and performed auth using the service account json key&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fj85khibhho82q9lgyn6v.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fj85khibhho82q9lgyn6v.png" alt=" " width="800" height="175"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Foom20jp5u9ksb4jug4n2.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Foom20jp5u9ksb4jug4n2.png" alt=" " width="800" height="175"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;Also if you're concerned and specific about object acl and classes, you can pick the appropriate one from the options &lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fz5ukk8rw8jhp91ppzyox.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fz5ukk8rw8jhp91ppzyox.png" alt=" " width="800" height="499"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fuggve0oui3s5ti664s4n.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fuggve0oui3s5ti664s4n.png" alt=" " width="800" height="964"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;once you're done with it, you can check if the mounting has been successful by using the ls command along with the remote name&lt;br&gt;
&lt;/p&gt;
&lt;/blockquote&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;rclone ls filescom:
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;blockquote&gt;
&lt;p&gt;And to copy the files the usual syntax is&lt;br&gt;
&lt;/p&gt;
&lt;/blockquote&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;rclone copy &amp;lt;source&amp;gt; &amp;lt;destination&amp;gt; [flags]
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;blockquote&gt;
&lt;p&gt;we got bunch of flags to show the progress --progress, mention the parallel transfer with number --transfers [number], to perform a simulation use --dry-run, to exclude or include any files we can use --exclude or --include&lt;br&gt;
&lt;/p&gt;
&lt;/blockquote&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;rclone copy filescom:/hawk gcs:vault-archive/-P --transfers=8 --checkers=10 --buffer-size=64M --fast-list --retries=5 --low-level-retries=10 --timeout=5m --contimeout=30s --retries-sleep=10s  --log-file=/home/mohamed-roshan-k/rclone_transfer.log --log-level=INFO

&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;blockquote&gt;
&lt;p&gt;-p = progress bar&lt;br&gt;
--checker = checking if the file already exists in the destination&lt;br&gt;
--buffer-size = mentions the size per file that's transferred to the buffer&lt;br&gt;
--retries = number of times it should retry the transfer if it fails&lt;br&gt;
--low-level-retries = similar like --retries but for network and file level error&lt;br&gt;
--timeout = aborts the task if its stuck more than the mentioned time&lt;br&gt;
--contimeout = connection timeout&lt;br&gt;
--retries-sleep = interval between each retry&lt;br&gt;
--log-file = path to the logs&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;Some screenshots on the time taken for transfer. &lt;br&gt;
Do note, the process can be made faster if we increase&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Transfer = --transfers&lt;/li&gt;
&lt;li&gt;Checkers = --checkers&lt;/li&gt;
&lt;li&gt;Buffer size = --buffer-size&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;If your VM has the specs to handle the increased load (CPU, RAM, and network), you’ll see a noticeable improvement in performance (pretty obvious but yea)&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fig7uymrg288vkcrqonye.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fig7uymrg288vkcrqonye.png" alt=" " width="800" height="94"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fozievqs9x6vi7iiznmh4.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fozievqs9x6vi7iiznmh4.png" alt=" " width="800" height="102"&gt;&lt;/a&gt;&lt;/p&gt;

</description>
      <category>gcp</category>
      <category>cloud</category>
      <category>cli</category>
      <category>linux</category>
    </item>
  </channel>
</rss>
