Forem: 3obby

peer review

3obby — Tue, 03 Sep 2024 07:27:53 +0000

I've begun writing tests in Hardhat, and I'm second-guessing some of my design choices.

I feel most of the design work is done, and I'm reaching out to my good friend Carlos, a brilliant Ethereum security engineer to see how he would approach the design.

Here's what I'm sharing:

Stonewall is a decentralized text-based posting platform.
Data is permanant and undeditable.

This looks like reddit/twitter/etc, basically:
Topics (AMERICA)
Posts (Historical record- first-hand account of an event)
Comments (May have comments themselves, no limit to depth)

posts and comments are ordered beneath their parent by value-
authors and readers send value to posts and comments they like.
the order may change over time as readers donate or new posts/comments are created.

New topics like 'America' are created when users create them by posting about them.
Posts are stored as keccak hashes:
    keccak256(abi.encode("AMERICA")) -> 0xd2f...b0a6

Topics may have any number of posts
Posts may have any number of comments
Comments may have any number of sub-comments, of infinite/arbitrary depth

additionally, topics and posts each have their own 'relevance list'
the relevance list offers readers similar topics/posts
the relevance list is also user-generated and ranked by user-donated value

the MVP allows users to:
-create posts (which create a new topic, if it doesn't exist)
-create comments
-send value to posts and comments they like (maybe triggering the data to re-order)
-add topics to the relevance list for a topic/post
-see the ranked posts and comments beneath a topic
-see the relevance list of similar topics/posts

Challenges:
-storing comments of arbitrary nested depth
-re-ordering posts/comments efficiently
-ensuring content is stored statically, such that links never break
-ensuring content doesn't collide in storage
-designing a simple system to link topic/posts to relevance lists

my solutions to the challenges feel naive; to ensure unique data storage slot generation, I've decided store the structure of posts/comments in a uint256 2d array on each parent element- each parent has its own defined depth and has a list of children which determine width.

for value-based ordering, keep an ordered list of pointers in the parent element. re-ordering efficiently... a basic insertion algo will work, but it feels sloppy to not consider gas limits. It could be possible to allow users to determine where they'd like the data to be indexed, then simply calculate an appropriate cost and ensure there's no problem. Finally, the static linking would enable each parent to store a similarly-ordered list.

Let's see what Carlos comes up with!

front-end considerations

3obby — Thu, 29 Aug 2024 14:45:16 +0000

threw together a stupid simple front-end with the help of gptengineer

Realized a few things today:

topics need to be created/stored in a mapping upon the first post such that the front-end can promote trending topics
gptengineer is for rapid prototyping (and probably not much else)
probably other discoverability that people would find interesting are 'top comments', 'top of all time', 'most valued authors'
It's possible to split donations between stonewall and authors, I think this is a good idea
It's possible to allow people to 'post' for free without data permanence, or allow data from other sites to be cross-posted to stonewall and made permanent.
trending topics could 'decay' over time

designing the skeleton

3obby — Wed, 28 Aug 2024 16:53:19 +0000

spent the day considering stonewall's architecture in bukit bintang's 🔥 breakfast spot. I've started building basic examples and tests in solidity in a hardhat project with js and have a bit more clarity.

the data structure of stonewall looks like reddit/twitter/etc:

topics
posts
comments (recursive)

a 'topic' at the top level has a list of any number of posts, which has a list of any number of comments, which themselves may have a list of any number of comments, and so on. All lists are ordered by descending value, and must be re-ordered when the value changes.

for example:

>politician name (topic)
    politician did bad thing!
        >and here's more proof...
        >no they didn't
            >ur an idiot
            >idiot dummy
    politician has fake hair?
        >lol watch this vid...
        >finally some real reporting...
        >yeah obv

a few technical bits of stonewall:

users' posts and comments must be stored in an array of arbitrary length, probably searchable in both directions, so a double linked list seems appropriate
users' posts and comments must be re-organized upon receiving value
comments may be nested arbitrarily deeply
the text length of users' posts and comments submissions are limited by the EVM, but it is possible to allow authors to append more data

posts and comments must be referenceable using unique static IDs both for the EVM and external sharing of content. For example, a link might look like:

https://stonewall.com/post/
https://stonewall.com/comment/

in future planning, topics/posts will be upgraded to support some type of community-powered binding to improve discoverability

Data Structure

Each topic 'already exists' and is discoverable as a hash of an upper-cased human-searchable string (e.g., "HARVARD UNIVERSITY"=0x4841525641524420554e4956455253495459). At this storage slot, there may or may not be a Post (aka an instance of the 'PostComment' struct- posts and comments are about the same)

it's important to know how many top-level posts are here...
Mapping Topic (b32) -> numPosts (used to generate unique PostComment ID for Topics)

PostComments always exist in a double-linked list of posts ordered by value.

Struct PostComment is stored at the hash of (topic(numPosts) for posts, parent(numComments) for comments)

{
author
createdAt
numPages (number of continued pages, else 0)
nextPage (location of next page, else null)
prevPost (location of adjacent higher-value post, otherwise null)
nextPost (location of adjacent lower-value post, otherwise null)
numComments (used to generate unique storage for children)
valueSum
}

PostComments are stored in a double-linked list of PostComments allowing for unlimited size and easy modification due to value change (find new location, update old neighbors' prevPost/nextPost, update own + new neighbors prevPost/nextPost)
Unique permanent storage slot identified by a static ID generated via keccak of parent + index
Allows authors to append additional data, but not delete/modify

dev thoughts for later
///hashing with keccak... encode vs encodepacked, keccak recursive depth limit
///ordering by value optimization... keep another array of ordered values->ids per parent?

This design will need to be modified as I learn more.

In the meantime, I've had some other thoughts about the project as a whole... loving the idea of a simple monospace retro design, can't wait to throw together a demo. in the end, the device to target is the phone, and most battery burn comes from the screen, so white on black minimalism also rocks for performance. Not sure about the final user experience- the most simple user experience I can think of for now is hitting a dex contract with your message and quoting+converting any erc-20 token on the fly.

One little thing that irks me about so many social media sites like X is the fact that comments are hidden, requiring that you click on them. why?? just show the full chain

A devious way to launch this would be to start posting shit about random topics, then get others to piggy-back the headlines. If anyone reads this retroactively, know I'm not dumb enough to actually do this.

Next, I'll either formally define the set of functions or look into the limitations of nested mapping via recursive hashing... it'd be a shame to only get like 12 layers of comment depth.

building free speech forever

3obby — Mon, 26 Aug 2024 18:26:15 +0000

If you could tell the world anything, what would you say?
How much would you pay to share that information?

In the wake of the arrest of Pavel Durov, I'm reminded there is an ongoing fight for freedom of speech.

Today, I begin designing 'Stonewall Data Permanence', a tool to give normal people the ability to publish and comment on any topic, permanently, without moderation or censorship, with their content made easily discoverable by others.

Keep an open mind while considering the resulting behavior of the few core design decisions:

-anyone can post and comment anything about any topic, permanently and optionally anonymously
-users pay per-character to post and comment
-posts and comments may receive funding from the author or readers
-posts and comments are ranked by monetary value

This design accomplishes a few things:

-Misinformation is magnitudes more expensive to promote- consider that factual competition is indexed in both adjacent posts & their comments
-bot/spam protection (economically unfeasible)
-transparency of funding enables users to optionally filter content based on funders' behavior (bot-like patterns)
-promoting information correlates directly to spend
-the world collaborates to prioritize content

Assuming this recipe provides novel value, the unknowns are tech stack and indexing. For the tech stack, I want Bitcoin's data permanence with Ethereum's smart contracts. I'm considering Ethereum, Rootstock, and Stacks as the main contenders for trust-minimized code execution.

A front-end will need to be bulletproof with a reverse proxy on SporeStack, eventually.

For indexing, maybe I'll first implement a basic topic lookup with a community-based 'similar topics' list within each topic... maybe some kind of vectorization that isn't gamifiable... perhaps community-driven.

I'll keep building