I built a self-hosted news digest that re-ranks itself based on what you actually read

Tiago Pereira — Sun, 17 May 2026 11:09:28 +0000

I use RSS readers constantly. The problem I kept running into was that they are
stateless. Every day the same firehose, in the same order, with no memory of
what I cared about yesterday. I wanted something that actually got better over
time, ran on my own machine, and didn't require handing my reading habits to a
third party.

So I built CondenseIt.

What it does

CondenseIt is a self-hosted daily news digest. You point it at your sources, it
runs on a schedule, summarises everything using a local LLM, and serves the
results as a web app you can read in the browser.

Sources it supports out of the box:

RSS and Atom feeds
YouTube channels (via transcripts from the public channel RSS)
Reddit subreddits (transparently served via Lemmy.world RSS so it works on VPS IPs where Reddit is blocked; the "Reddit" badge still shows)
Hacker News (top, best, new, ask, show)
GitHub release feeds
Google News searches (with operator support: site:, when:, intitle:)
Website change detection (diffs the page content and treats meaningful changes as new items)
Podcast feeds (new episodes from any podcast RSS, with iTunes search built in to find the feed URL)

All of that is configured from an admin panel. No YAML editing required after
initial setup. Per-source keyword filters (allowlist, hide, and highlight rules)
are also set per-source from the admin panel.

The LLM part

Summaries run through Ollama locally, Metal-accelerated on Apple Silicon. No
discrete GPU needed. I run it on a Mac Mini and it handles a daily digest of
50-80 articles without issues. If you prefer cloud inference, OpenRouter is
supported too, with budget limits so you don't accidentally run up a large bill.
Summarization is now parallelized so the digest runs noticeably faster. OpenRouter
calls also retry automatically on rate-limit responses.

The LLM is used for summarisation. Ranking is a separate system built on top of
classical signals, with optional AI layers you can turn on incrementally.

How the re-ranking works

Every article gets a score composed of classical signals:

Keyword overlap with your liked/disliked term profile
Bigram phrase matching
TF-IDF cosine similarity against your content history
Category and source averages from your past ratings
Three implicit engagement signals: reading an article, saving it for later, and dismissing it
Synonym boost (configurable synonym groups propagate weight across related terms)

Explicit star ratings (1-5) drive the biggest updates to the profile. Implicit
signals are weighted at 0.5 by default so they contribute but don't dominate.
All rating contributions decay exponentially (default half-life: 30 days) so
stale preferences fade rather than accumulating forever.

There are also three optional AI layers, each independently controlled and off
by default:

Semantic embeddings - article text and your liked/disliked articles are
encoded as vectors. Each candidate is scored by cosine similarity to the
centroid of your liked embeddings minus disliked embeddings. Embeddings are
generated once and cached in SQLite (keyed by URL + content hash), so
subsequent runs are fast.
Topic/entity enrichment - the same LLM call that summarizes an article
also extracts topics, entities, and a novelty score (1-5) at no extra cost.
These are used to build a topic preference profile from your ratings. Topics
and a "novel" badge appear on each card.
LLM reranker - after classical scoring, a compact profile narrative is
built from your top liked/disliked terms, categories, and sources. The LLM
scores the top-K candidates by relevance and returns a brief reason. The
relevance score is blended with the classical score (default blend: 0.3) and
the reason appears in the "Why ranked here?" panel.

There is also a cold-start bootstrap: if you have no ratings yet, you can visit
Admin > Preferences and describe your interests in plain text. The LLM derives
initial keywords, synonyms, and a profile summary that seed the engine before
any ratings exist.

The part I found most useful to add was a transparency panel. Every article card
has a collapsible "Why ranked here?" section that shows each signal (classical
and AI) as a proportional bar. It made tuning the weights much easier and
surfaced some surprising things about my own reading habits.

A newer addition is semantic deduplication: articles covering the same story are
collapsed before ranking, so you don't see five versions of the same news item.

Saves vs. star ratings

There are two kinds of saves now:

Save for later - implicit strong positive; contributes to your preference profile like a high star rating.
Starred (permanent save) - has no ranking effect. Starred articles live on a separate page and survive all future digest runs. Useful for bookmarking things you want to return to.

Stack

Python 3.11, FastAPI, SQLite
React, TypeScript, Vite
Ollama or OpenRouter for inference
MIT license

Local setup is uv sync then condenseit serve. There's also a deploy script
for running it on a VPS with nginx and a systemd service. The deploy script
now supports multiple instances with an interactive picker if you have more
than one server.

The web app is installable as a PWA on iOS and Android (there's a guide in the
docs).

What I'm looking for

The project is open source and I'm actively looking for contributors. Areas
where help would be most useful: new collector types, frontend and accessibility
improvements, test coverage, and Docker packaging. Good first issues are labeled
in the repo.

Repo: https://github.com/wildlifechorus/condenseit

Happy to answer questions about any of the implementation decisions in the
comments.