A feed that doesn't need a topic

· 4 min read

indiedesk is a small IndieWeb experiment where posts from different websites can share a feed without hashtags or a fixed topic. Just a UUID, a Webmention, and a little corner of the web where things can come together without a social network.

Contents

Go to https://desk.pbl.md and you'll find something a little unusual: a feed that doesn't tell you what it's about. There are no categories to browse or tags you have to agree on. Instead, people publish posts on their own websites and send a Webmention to the relay. Each post goes into a collection identified by a UUID, and anyone who knows that UUID can follow the collection through Atom or JF2.

The collection is whatever the people using it make of it. The idea comes from @moja.blue, who called it a mezzanine: a small space for sharing things without putting them on a giant public timeline. Nawashiro adapted the idea for the IndieWeb and wrote about it in Japanese. The original program was called indie-mezzanine. This English-language version is indiedesk.

Putting a post in a collection #

Start with a public page on your own site, served over HTTP or HTTPS. It needs exactly one microformats2 h-entry. That part is strict. If the page has two, indiedesk won't choose one for you. If it has none, there's nothing to collect.

Next, choose a collection ID. It's simply a UUID, either one someone shared with you or one you generated yourself. For example, bbe44bfe-3a9c-410e-ad37-f1bf7402bce4. Add this to your page:

<link rel="collection" href="urn:uuid:bbe44bfe-3a9c-410e-ad37-f1bf7402bce4">

The page also needs a link to https://desk.pbl.md/ so the relay can tell the mention was meant for this host. Once the page is actually online, send a Webmention with a POST request to https://desk.pbl.md/webmention. Set source to the URL of your post and target to https://desk.pbl.md/.

Be particular about the target. The relay accepts its root URL with or without the trailing slash. A different path, a query string, or a username in that URL will be rejected.

An HTTP 202 response means the Webmention was accepted for processing, not that the post is already in the feed. The relay still has to fetch the page and validate its HTML. If everything checks out, it saves the extracted microformats2 data, not the page's HTML.

That first successful fetch is the version that stays. Edit the original post next week and the feed won't change. Submit the same source again and you'll get an already-stored response. If two Webmentions for the same page arrive while it is still in the queue, they become one job.

Following a collection #

The JF2 response includes Access-Control-Allow-Origin: *, so a page on another site can read it directly. If a post has no author name, the entry says Unknown author. The feed's own author is listed as indiedesk.

Each entry uses its original source URL as its ID, in the form it was submitted. That matters if the same post reaches two relays following the same rules. You can merge their JF2 feeds without creating two separate entries for the same post.

There isn't much of a stack to manage. One Go process handles the HTTP requests, fetches posts and serves the feeds. A SQLite file on a volume holds both the posts and the Webmention queue. No Redis, no second database server. A couple of workers pick up queued jobs, fetch the source page and record the result.

Fetching URLs supplied by other people needs some care. indiedesk uses Doyensec's safeurl library to keep those requests away from private networks. Only ports 80 and 443 are allowed. Responses have to be HTML and stay within a size limit. Redirects are counted, and each hop goes through the checks again.

The landing page is a Markdown file rendered when the process starts. It also declares a single h-entry, which is a nice way for the project to follow its own rules. Cloudflare Tunnel exposes the container, with PUBLIC_URL set to https://desk.pbl.md. Every Webmention target is checked against that root URL.

Getting the English right #

The guide, error messages, code comments and specifications all started in Japanese. Translating them wasn't just a matter of making the sentences sound natural. The behavior they described had to stay the same: one post per page, duplicate mentions combined into a single job, and microformats2 data stored instead of the original HTML.

Some of the responses shown to clients sounded oddly formal in a literal translation. They're shorter now. bad form is enough when the request body is wrong. The wording is less fussy, but the validation hasn't changed.

Cloudflare can be the awkward part. Most of the useful traffic here doesn't come from someone browsing the home page. Webmentions are POST requests sent by other servers. Feed readers make GET requests, usually without cookies. Put a browser challenge or the wrong WAF rule in front of those requests and Cloudflare stops them before Go ever sees them. GET and POST requests to desk.pbl.md need to bypass those checks.

There's also a filesystem detail that's easy to miss. The data directory must exist before the containers start. On Linux, it needs to belong to UID 65532 and have mode 0750, because that's the user inside the image. This Compose setup expects the directory to be created ahead of time. On Windows, that numeric UID doesn't carry over cleanly through a bind mount. This deployment really wants a Linux host.

The validator is intentionally picky. A source URL pointing to localhost won't pass. Neither will a page with zero or multiple h-entry elements, a collection ID that isn't a UUID, or a Webmention aimed at another host.

And sooner or later, somebody will fix a typo on their original page and wonder why the feed still shows the old version. That's expected. indiedesk keeps the first valid copy and leaves it alone. A service that keeps checking the source and updating its copy would be a mirror. This is a relay.

Links and notes #

@moja.blue: https://tangled.org/moja.blue

Nawashiro, the original mezzanine note: https://nawashiro.dev/posts/20261005-mezzanine

Webmention: https://indieweb.org/Webmention

JF2 specification: https://jf2.spec.indieweb.org/

indiedesk relay: https://desk.pbl.md

1 like NawashiroNawashiro
1 repost NawashiroNawashiro
Reply by email

Your message goes straight to Pablo Murad. Your address is only used to write back.

Comments

Leave a comment

Comments are read before they show up here. No email address needed.