# Markdown File Rendering ## Current State (writeonce-articles-s3) Each article is a directory containing a JSON metadata file and one or more `.md` files: ``` auto-scale-gitlab-runner-using-aws-spot-instance/ docker-machine-test-with-t2.md gitlab-runner-config.md stop-test-gitlab-docker-machine.md gitlab-runner-with-kubernetes-executor/ gitlab-runner-with-kubernetes-executor.json deploy.md permission.md role-binding.md role-defination.md gitlab-runnergitlab-runner-deploy.md ``` The JSON metadata currently defines the full article structure — sections, headings, paragraphs, and code snippet references. Markdown files are limited to code blocks referenced via the `codes[].snippet` field. ## Problem The JSON metadata carries too much content. Headings, paragraphs, prose — all of this is duplicated as JSON strings inside `content.content.sections`. The markdown files only hold code snippets, referenced by `sectionIndex` and `paragraphIndex`. This is backwards. The markdown file should be the content. The JSON should be minimal metadata. ## Target: Markdown-First Content Model **The markdown file is the article.** All prose, headings, code blocks, and inline formatting live in the `.md` file. The JSON metadata file holds only what markdown cannot express: system fields, tags, publication state, and author. ### Minimal JSON Metadata ```json { "sys_title": "gitlab-runner-with-kubernetes-executor", "title": "Gitlab Runner with Kubernetes Executor", "published": true, "author": "Shoney Arickathil", "tags": ["kubernetes", "gitlab", "ci-cd"], "published_on": 1740950884 } ``` No `content.content.sections`. No `content.content.codes`. No `paragraphs[]` arrays. No `sectionIndex`/`paragraphIndex` mapping. ### Markdown File = Full Article Content ````markdown # Introduction Deploying a Gitlab runner using kubernetes is a great option to overcome the limitations of other gitlab runner executor such as docker and docker machine. ## Running Gitlab Runner in gitlab namespace Create the namespace and apply the deployment: ```yaml apiVersion: apps/v1 kind: Deployment metadata: name: gitlab-runner namespace: gitlab ``` ```` ## Permissions The runner needs RBAC permissions to create pods: ```yaml apiVersion: rbac.authorization.k8s.io/v1 kind: Role metadata: name: gitlab-runner ``` Everything is in the markdown — headings, paragraphs, code blocks with language hints, links, images. The rendering pipeline parses the markdown directly. ### Directory Structure ``` content/ gitlab-runner-with-kubernetes-executor/ gitlab-runner-with-kubernetes-executor.json # minimal metadata gitlab-runner-with-kubernetes-executor.md # full article content linux-misc/ linux-misc.json linux-misc.md ``` One JSON for metadata. One markdown for content. No scattered `.md` files per code snippet. ## What Changes | Before | After | | -------------------------------------------------------------- | ----------------------------------------------------------------------- | | JSON holds sections, headings, paragraphs as structured arrays | JSON holds only sys_title, title, published, author, tags, published_on | | Markdown files hold only code snippets | Markdown file holds the entire article | | `codes[].snippet` maps filename to sectionIndex/paragraphIndex | No mapping needed — headings and code blocks are inline in markdown | | Renderer reads JSON structure, injects code from .md files | Renderer parses markdown directly into HTML | | Multiple .md files per article (one per code snippet) | One .md file per article | ## Impact on the Data Layer ### wo-model The `Article` struct simplifies: ```rust pub struct Article { pub sys_title: String, pub title: String, pub published: bool, pub author: String, pub tags: Vec, pub published_on: Option, } ``` The nested `ArticleContent` / `ArticleBody` / `Section` / `CodeSnippet` hierarchy is no longer needed. Article content comes from parsing the `.md` file at render time, not from the JSON. ### wo-md Currently handles only inline markdown (`**bold**`, `` `code` ``, links). Needs to become a full markdown-to-HTML renderer: - Block elements: headings (`#`, `##`), paragraphs, code fences (` `lang ```), lists, blockquotes - Inline elements: bold, italic, code, links, images - Code fence language extraction for `wo-md::highlight()` - The renderer reads `{sys_title}/{sys_title}.md`, parses it, and returns HTML ### wo-htmlx The `article.htmlx` template simplifies. Instead of iterating `{{#each article.content.content.sections}}`, it renders the pre-parsed markdown HTML: ```html

{{article.title}}

by {{article.author}} · {{article.tags}}

{{article.content_html}}
``` Where `content_html` is the full HTML output from the markdown renderer. ### wo-store `ContentLoader` reads the `.json` for metadata and the `.md` for content. The `.seg` file stores both. At query time, the markdown is either: - Pre-rendered to HTML during ingestion (stored in .seg alongside metadata) - Rendered on-demand at request time (read .md from disk) Pre-rendering is preferred — it avoids parsing markdown on every HTTP request. ## Migration Path 1. Update `wo-model` with the simplified `Article` struct 2. Extend `wo-md` to handle full markdown (block-level parsing, code fences) 3. Update `ContentLoader` to read `.json` + `.md` pairs 4. Update `wo-store` to store pre-rendered HTML in the .seg file 5. Simplify `article.htmlx` template 6. Migrate existing articles: extract prose from JSON into `.md` files Existing articles with the old JSON format can coexist during migration — `ContentLoader` checks for a `.md` file and falls back to the JSON structure if none exists. ## Blog Subscription — Live Content Reload When a user visits `http://localhost:3000/blog/sample-rust-patterns`, the content should stay live. Any edit to `sample-content/sample-rust-patterns/sample-rust-patterns.md` must auto-reflect in the browser without a page refresh. ### How It Works ``` Browser visits /blog/sample-rust-patterns │ ▼ 1. Server renders article HTML from .seg (pre-rendered from .md) 2. Server writes HTML response to socket fd 3. Server registers socket fd in subscription table: register!(sub_manager, socket_fd, ByTitle("sample-rust-patterns")) 4. Connection transitions to Subscribed state (stays open) │ │ (user edits sample-rust-patterns.md) │ ▼ 5. inotify fires IN_MODIFY on sample-rust-patterns.md 6. ContentWatcher maps file → sys_title "sample-rust-patterns" 7. Store rebuilds: re-reads .json + .md, re-renders markdown to HTML, updates .seg + indexes 8. SubscriptionManager::notify("sample-rust-patterns", ...) fires 9. For each subscribed fd: write(fd, diff_payload) │ ▼ 10. Browser receives payload on the open connection 11. Client-side script applies the update to the DOM ``` ### What Needs to Work | Component | Requirement | |-----------|-------------| | **inotify** (wo-watch) | Already watches `content/` directory. `.md` file changes must trigger `ContentChange::Modified(sys_title)` | | **Store rebuild** (wo-store) | On `.md` change: re-read file, re-render markdown to HTML, update `.seg` and indexes | | **Subscription table** (wo-sub) | Route handler registers the browser's socket fd via `register!` after sending initial HTML | | **Notification** (wo-sub) | On content change, write updated `content_html` to all subscribed fds as JSON payload | | **Event loop** (wo-rt) | After writing initial response, transition connection to `Subscribed` state. Keep fd on epoll for hangup detection. | | **Client script** | Injected in the HTML. Reads payloads from the open connection. Replaces article content in the DOM. | ### Client-Side Script Injected by the template renderer into every article page: ```html ``` ### inotify and .md Files The current `ContentWatcher` watches for `.json` file changes. It must also trigger on `.md` file changes: - `IN_MODIFY` on `*.md` → `ContentChange::Modified(sys_title)` - The sys_title is derived from the parent directory name (same as for JSON) - Both `.json` and `.md` changes trigger a store rebuild and subscriber notification