Forem: SaaSHub

How to steal a website and how to prevent it

Stan Bright — Tue, 13 Dec 2022 03:23:56 +0000

OK, this is going to be a quick showcase of how someone proxy-mirrored my website in a way that is very difficult to detect or prevent it. Moreover, I will share some of the helpful advice that was received from the Hacker News and /r/sysadmin communities.

Here it is the case

I noticed yesterday that one of my websites, SaaSHub, is not present on bing.com; however, its content is being served on a different and very suspicious domain name - sukuns.us.to. And, of course, I started digging my nginx logs - grepping for that domain name. All I could find was that domain name appearing as the referrer when accessing some images. Then, suspecting what might be going on, I started accessing some unpopular pages (on their domain name) and monitoring the logs for them. All requests to my server were coming from different IPs located in the US but with my browser's User-Agent. At that point, I was pretty sure what was happening - someone was proxy-mirroring SaaSHub and serving it on-demand on their domain name.

Here it is a simplified diagram of the case

I started thinking, brainstorming and searching for some generic advice on what I could do to prevent this...

Option 1 - block their IP. Unfortunately, that's not very effective in general, as they could easily change their IP address. What is more, in this particular case, they are using a very wide range of IPs coming from a variety of different ASNs (some of the cases I caught - AS55081, 35913, 36352, 21769, 52393, 394814, 55286). What is more, at first glance, collecting those IPs and ASNs isn't a trivial/quick task.

Option 2 - block them by user-agent - Again, not an option as they are copying the user-agent of the user making the request on their domain-name.

Option 3 - add a small piece of JavaScript that checks for the domain name, and if not www.saashub.com - redirect to it. This isn't a viable option in this case as the perpetrator is stripping all <javascript> tags and files.

Option 4 - use absolute URLs everywhere. Well, they are rewriting all mentions of www.saashub.com in links to their own domain name. So that doesn't seem like an option, too.

At that point, I felt helpless and asked for help on Hacker News & /r/sysadmin

Possible solutions and advice

i.e. what can you do if this happens to you... based on the suggestions and advice that I received on HN & /r/sysadmin.

1) File a DMCA to their host. I've sent a request to their free domain name provider so far.

2) Covert the whole website to JavaScript. Yup, that would work, but then no one without JS would be able to open SaaSHub, I'm guessing SEO will suffer too. So, this is not a path I'd like to take.

3) Block all images being accessed with a referrer that's not the expected domain name - I haven't implemented this one yet, but this could actually work in a generic way. I'm keeping this one in my toolbox of viable options.

4) Implement a honey-trap URL so that all their IPs could be exposed and blocked. This could work as well. "All" you have to do is add an endpoint that is not accessible by normal users - e.g. /honeytrap?some-random-param=rand-number (Note: you need the randomness as the culprit is loading each page once only and caching it after that). Then, add a cron script that will call that end-point on the proxy-mirror domain name and expose all their IPs. That way, we can block them in a relatively automated manner (as long as we know their domain name).

5) Add some hidden and random text to your pages. Why? So that you can search for that text later on and find out all the domain names that mirror your content.

Moral of the story

We are not absolutely helpless! It is unfortunate that we have to deal with it, but, as they say - it is what it is. At least we have a small toolbox of viable options on how to react and what we can try out.

Also, I guess there could be other things and means available to prevent and mitigate similar vicious actions. I'd be happy if you shared your experience and solutions in the comments here. You can find more advice within the discussions linked above, too. Nevertheless, whatever we (you and I) decide to do in similar cases, it's always a difficult and time-consuming battle. And I believe that bringing some awareness to this might help others resolve it quicker than I.

As of now, I think that the report to their free DNS provider has worked out, as their domain-name is not responding to an actual IP anymore. I will have to work on #5 from above, though, so that I can know when they move to a new domain-name.

p.s. if someone working at Microsoft or Bing is reading this, please could you help get SaaSHub indexed instead of the fake mirror sukuns.us.to. Thanks 🙇!

How to speed up your Ruby/Rails app by ~5% with zero efforts

Stan Bright — Mon, 04 May 2020 06:07:33 +0000

This is going to be a quick one. Yet I think it might be helpful to almost everyone using ruby. A few days ago, I learned about a gem by Shopify - symbol-fstring. Thank you @strzibnyj.

It does something very simple - very well: Access symbols internal strings without duplicating them. Why is that important?

In Ruby many APIs tend to accept symbols, but regularly convert them to string internally. The typical example is ActiveSupport::HashWithIndifferentAccess, but there are plenty more.

The problem with this is that Symbol#to_s creates a new string every time it is invoked, and since it often happens in hotspots, it causes a lot of work for the garbage collector, and cause many identical strings to be kept in memory.

To activate the gem in an app, all you need is adding the gem to your Gemfile and requiring the "patch".

e.g. gem 'symbol-fstring', require: 'fstring/all'

That's all. SaaSHub has been running with it for 3 days now, and there haven't been any unexpected issues. Based on my rough benchmarks, a typical page like "basecamp alternatives" responds about 5% faster and allocates about 5% fewer objects.

I know, 5% isn't that much, yet it's "FREE". Moreover, this gem is maintained by a billion-dollar company, Shopify, popular with its high-quality dev standards. So, I guess it is both safe and worth it giving it a go.

Replacing React with Preact. It was easy and worth it.

Stan Bright — Fri, 17 Jan 2020 10:47:35 +0000

I had been considering preact for quite some time. After all, the sell is easy:

100% compatible with the React ecosystem (kind of)
Much smaller (js bundle size)
Faster (performance)

About two years ago, I went to a local meet-up in Sydney where the presenter was sharing how they were integrating Preact successfully in some parts of Qantas. That was interesting. And he was convincing. Yet, given that everyone is using React, I thought it was a daunting task and never put the time to research it further. Until recently.

I was working on optimizing the page-load speed of SaaSHub, and one of the paths was decreasing the JS bundle size. I played a bit with webpack-bundle-analyzer and source-map-explorer, and it was apparent that 35% of all the libs were taken by React & react-select. Then I remembered preact... and decided to review it again.

It happened that that task was more scary than difficult. After going through the docs, the whole switch to preact consisted of adding it to packages.json, adding the relevant aliasing to webpack's build configs:

environment.config.resolve.alias = {
  "react": "preact/compat",
  "react-dom/test-utils": "preact/test-utils",
  "react-dom": "preact/compat",
  // Must be below test-utils
}

module.exports = environment

and importing 'preact/debug' somewhere in the app:

import 'preact/debug'

That was all. So simple. Everything worked without touching another line of code. Of course, the process could be more complicated for web apps with more sophisticated code.

What are the benefits:

SaaSHub's JS bundle file-size decreased with 20%: from 577k down to 460k
Faster JS (although I haven't benchmarked that, neither have felt it being slow)

In the end, if you are working on optimizing your JS, and you don't have super complicated setup, I'd highly recommend giving preact a go. It might be easier than you think.

p.s. the next step will be replacing react-select with downshift. I've already implemented one small component with it, and it is amazing. Unfortunately, that migration will require a lot more code changes.

Postgres Trigram indexes VS Algolia

Stan Bright — Mon, 19 Aug 2019 09:06:23 +0000

I want to share my experience of using the native "trigram indexes" of Postgres to achieve a robust "typeahead/autocomplete/fuzzy search" functionality. I am using Algolia as an example, as I have noticed that a lot of people are using it precisely for that. Yet, I believe that you could do perfectly fine by using only Postgres in many situations.

What are trigram indexes? They are a special type of index in Postgres that is available by default; however, they have to be explicitly enabled. Here it is an extract from the official docs:

"The pg_trgm module provides functions and operators for determining the similarity of alphanumeric text based on trigram matching, as well as index operator classes that support fast searching for similar strings.", "A trigram is a group of three consecutive characters taken from a string."

For example, the set of trigrams in the string "cat" is " c", " ca", "cat", and "at ". The set of trigrams in the string "foo|bar" is " f", " fo", "foo", "oo ", " b", " ba", "bar", and "ar ".

Now let's follow what's necessary to build our autocomplete engine.

1) Enable the Postgres extension

CREATE EXTENSION IF NOT EXISTS pg_trgm

2) Index the columns you are going to query

Once you have enabled pg_trgm you have to index varchar and text columns to make use of it. e.g. CREATE INDEX index_fuzzy_searches_on_name_trgm ON fuzzy_searches USING gin(name gin_trgm_ops). Notice USING gin(name gin_trgm_ops)

Now we have very fast LIKE queries with leading and trailing wild cards (%).

3) Prepare the query

The next step is splitting the user query by spaces and adding a separate WHERE clause per name. For example, if a user searches for "sales cloud". We'd end up with a query like SELECT * FROM mytable WHERE name LIKE '%sales%' AND name LIKE '%cloud%'

A simplified implementation in Ruby on Rails could look like (in this case ServiceSearch would be MATERIALIZED VIEW that I will explain down the post):

relation = ServiceSearch.popular_first.limit(limit).includes(:service)
query.split("\s").each do |word|
  relation = relation.where("content LIKE ?", "%#{word}%")
end
relation.map(&:service)

4) Search for different types of records simultaneously

One of the use cases of using a dedicated full-text search engine is querying all your records simultaneously. That could be achieved through a MATERIALIZED VIEW. Moreover, you can easily give priority to some records. For example, showing Products before Categories. I am managing my views by the awesome scenic gem, but you don't need to.

Here it is the body of an exemplary materialized view:

SELECT
  id AS searchable_id,
  'Service' AS searchable_type,
  1 AS priority,
  post_services_count AS weight,
  LOWER(name) AS name
FROM services
WHERE state IN ('approved', 'closed')

UNION

SELECT
  categories.id AS searchable_id,
  'Category' AS searchable_type,
  2 AS priority,
  COUNT(*) AS weight,
  LOWER(name) AS name
FROM categories
JOIN categorizations cats ON cats.category_id = categories.id
WHERE state = 'approved'
GROUP BY categories.id, categories.name

By having a view like that one, we can query multiple records with something as simple as:

SELECT * FROM fuzzy_searches WHERE name LIKE '%test%' ORDER BY priority, weight DESC

Real-world examples

One of the top competitors of SaaSHub - alternativeto.net is using Algolia for their search autocomplete. I'd suggest testing their search and SaaSHub's. You won't find many differences in the UX or speed.

In the end, if you need a simple autocomplete functionality for your project, and you are using Postgres, my suggestion is to consider its trigram indexes + materialized views. Reaching out to external services may not be necessary and could be an overkill. However, I'm sure that there will be many cases in which it will make sense to pay for a professional SaaS product.

p.s. you can find more Algolia alternatives on SaaSHub 🙈.