WERK Update: Pasting, persistence & deep dive

tl;dr Pasting and persistence added, Code & NextJS deep dive below.

Dear reader,

Brace yourselves, this isn't your typical, run-of-the-mill update. No, this is the chronicle of WERK's evolution, a testament to my relentless pursuit of perfection (Or just a working product). I've tackled two glaring voids in the WERK experience that, frankly, were driving me to the brink of madness: the act of pasting content into WERK and the art of persisting data.

Behold, a glimpse into the future—our latest features, unveiled in their raw, unrefined glory within the developer's lair:

Copying from Notion/Obsidian to WERK

Ever tried copying content from Notion or Obsidian, only to watch in dismay as WERK reduced it to a monolithic block of text? A travesty, I know. The seamless flow of ideas from brain to screen shouldn't be thwarted by such trivialities.

So, I've devised a solution. It's a stopgap, admittedly, but one that transforms markdown chaos into structured elegance with a mere paste. This update isn't just a patch; it's a lifeline for your creative process, albeit not without its quirks.

Encountered something odd? Got a burning desire for additional copy/paste wizardry? Speak up. Your silence is as useless "as screen door on a submarine" as they say. I can't promise miracles, but I'm all ears—after all, solving your problems (no matter how peculiar) creates a better product. But remember, I'm not a mind reader; detail your use case, and I'll see what dark magic I can conjure.

Saving to localstorage

Ah, the allure of saving to local storage. Yes, you've guessed it—I've finally added it. Initially, I skipped this in the MVP to hasten the launch and measure your interest. But, as a user myself, the incessant loss of data upon a page reload was becoming a personal nightmare. So, I've dealt with it.

Implementing this was no walk in the park, but here we are. Now, WERK allows you to save your data directly to your browser's local storage. Toggle this feature on or off at your whim. Since it's nestled comfortably in your browser, rest assured, your secrets are safe with... well, you. There's no syncing to servers here—your words remain your own, shrouded in privacy. A perfect scenario for the paranoid among us.

But, a word of caution: if you're prone to clearing your local storage or stroll through the net in private browsing (porn) mode, beware. Closing that window is akin to setting your work ablaze—irrecoverable, irretrievable, gone. This isn't just about my convenience; it's about yours. The horror of losing your work upon a mere reload could very well drive you to abandon WERK altogether. And we can't have that, can we? So use the export feature or copy it to some place more persistent please!

A journey begun with no destination

The path to innovation is littered with hurdles, and WERK is no exception. These latest features were born out of sheer frustration—a testament to the fact that necessity truly is the mother of invention. Yes, the absence of these functionalities was a glaring oversight, one that was frankly quite embarrassing. But let's not dwell on the past.

I am crossing bridges as I come to them, and while there are still many rivers to ford, I'm less inclined to cringe at the thought of sharing WERK with the world now. After all, the true measure of a tool's worth is in its use, and losing your work? That's a cardinal sin we can no longer afford.

For the curious minds craving the nitty-gritty details, a technical deep dive awaits below. Feel free to venture further if you dare. Otherwise, consider this your graceful exit.

The quest for improvement is ongoing, and as WERK evolves, so too will its capabilities. Polishing these features is on the agenda, and as our user base solidifies, I'll turn my gaze toward broader horizons—monetization, expansion, and beyond.

But let's not get ahead of ourselves. For now, take solace in the fact that WERK is improving. Remember, life's journey is anything but a straight line.

Under the hood

Congratulations on making it this far. You're either genuinely interested in the nuts and bolts of WERK, or you've got way too much time on your hands. Either way, I appreciate the company. Let's peel back the layers and see what makes these features tick, shall we?

Pasting

The Pasting Conundrum The issue of pasting content from Notion or Obsidian was a curious one. It was an eye-opener to discover that copying from these applications resulted in markdown formatting—a revelation, to say the least. This discovery was the key to unlocking the entire feature.

The magic happens thanks to the browser's Clipboard API, it proved to be the solution. Let's dive into the code:

const copiedData = event.clipboardData.getData(
  event.clipboardData.types[0]
);

This line of code is where the magic happens. The getData function fetches the content in the format we need, typically "text/plain". While "text/html" could be a contender, it's an elusive shadow. So, we play it safe, betting on the reliability of "text/plain". The moment I snagged that data from the clipboard, I toyed with it—a bit of this, a touch of that—before tossing it into the editor. And, oh, it chews through markdown alright, but it's picky, throwing tantrums over a few deal-breakers here and there. The troubles of manual code—endless, with more surprises lurking than you'd care for. Basically it has a few bail-out conditions but here's the kicker: once you've committed to e.preventDefault(), there's no turning back—it's like trying to unscramble an egg. This means my brain had to do somersaults, backflips, the whole circus, to navigate through this mess. The logic's so twisted, it could give a pretzel a run for its money.

This approach, though seemingly straightforward, it akin to walking in the dark. It's a delicate balance, ensuring the pasted content retains its intended formatting without diving too deep into the abyss of "what-ifs."

Saving

At its core, local storage operates on a key-value pair system—elementary, yet with its own set of limitations. The most notable of these is the 5MB storage limit and its inability to function in server-side rendering. For our purposes, these constraints were inconsequential. The real conundrum lay in managing multiple editors' data efficiently.

The initial strategy was to assign a unique key to each editor's data. However, this approach quickly unraveled when faced with the challenge of retrieving and loading all keys from storage. After a day of fruitless endeavors, I reverted to a more reliable structure. Here's where the art of state design comes into play.

Consider this structure:

{
  "byId": {
     "123": {
       id: "123",
       data: [],
       position: 0
     }
     "abc": {
       id: "abc",
       data: [],
       position: 1
     }
  }
}

This design allows for a myriad of interactions:

const editorIds = Object.keys(getEditors)
const data = getEditors.editorIds[0].data

Looping through editors becomes a breeze, and accessing data by ID is straightforward:

getEditors["123"].data

Thinking about data structures properly and thinking this way gives you a lot of freedom when manipulating your data.

Now, for the elephant in the room—the dreaded re-render. Utilizing a single object for state, while liberating in terms of data manipulation, introduces a significant performance bottleneck. Changes to the state necessitate a re-render of all components, including those unaffected by the update. This was a scenario I was determined to avoid.

Enter Jotai and its concept of "optics." This elegant solution circumvents the re-render problem, ensuring that only relevant components are refreshed upon state changes. The simplicity and efficiency of Jotai not only solved our dilemma but also earned it my wholehearted endorsement as a state management tool.

Optics — Jotai, primitive and flexible state management for React

This doc describes Optics-ts extension.

Jotai

A Deeper Dive: Optics vs. The Old Guard

Venturing further into the state management saga, let's juxtapose the innovative approach of optics with the traditional methods—namely, Immer and the ubiquitous spread operator (...). It's essential to understand the advantages and potential pitfalls of each to truly appreciate the evolution of state management within React.

The Optics Revolution

Consider this implementation using Jotai's focusAtom with optics:

  const editorStateAtom = useMemo(
    () => focusAtom(editorsAtom, (optic) => optic.prop(editorId)),
    [editorId]
  );

  const [editorState, setEditorState] = useAtom(editorStateAtom);

To modify the state, it's as straightforward as:

  setEditorState({
    id: name,
    position: 0,
    data: editor.topLevelBlocks,
  });

This method ensures that updates to a specific key don't trigger unnecessary re-renders across the board. It's clean, efficient, and spares us from the convoluted dance of traditional state management.

The Traditionalist Approach

On the other hand, using Immer for the same task might look something like this:

setEditorsAtom((prev) => {
  return {
    ...prev,
    ["dynamicKey"] : {
       ...prev["dynamicKey"],
       data: "New Data",
    }
  }
})

While manageable for shallow updates, this approach quickly becomes cumbersome as you navigate deeper state structures. The necessity of spreading objects at each level is a bug-ridden minefield waiting to explode.

The Trade-off

Optics offers a streamlined alternative, eliminating the pitfalls of manual spread operations. However, it's not without its drawbacks. This abstraction might shield developers from the intricacies of React's state management, potentially hindering a deeper understanding of core principles.

The beauty of React lies in its purity—the requirement to forge your utilities and helpers. This journey towards mastery is what makes React so rewarding. Having battled through the trenches of state management, I can confidently say that for me, optics is the superior choice. However, for those new to the realm of state manipulation, I implore you to wrestle with the fundamentals first. Bypassing this rite of passage robs you of invaluable insights into object manipulation and the critical role of the spread operator.

The Elegance of Elimination: Removing Editors with Finesse

And what of the act of removal, you ask? Ah, it's here that we find simplicity and elegance in the midst of our complex dance of data management. Removing an editor from storage is as straightforward as it gets, yet it requires a touch of finesse to execute seamlessly.

There are several paths one might tread, but let's focus on the most intuitive ones. For the functional aficionados among us, employing a filter function to sieve out the undesired editor is a viable option. However, for those of us who prefer a more direct approach, the spread operator once again proves its worth:

const { [editorId]: _, ...restEditors } = editors;
setEditors(restEditors);

This method is the epitome of brevity and efficiency. By simply excluding the specified key and spreading the remaining editors into a new object, we achieve our goal with minimal fuss. It's these moments of clarity and simplicity that remind us of the beauty inherent in code—when a few well-chosen lines can speak volumes.

Falling at the final hurdle

In what can only be described as a rite of passage for any developer daring to venture into the realm of NextJS, I encountered a formidable adversary—the usual NextJS build errors, brought on by the innocuous addition of local storage. Ah, the irony.

The usual cheap tricks, the kind that seasoned developers keep up their sleeves for such occasions, proved futile. The classic:

const isServer = typeof window === "undefined";

A crude, inelegant solution to an unnecessary problem under normal circumstances, yet utterly ineffective in the face of React Server components. It seems these components possess a certain... audacity, bypassing logic and rendering as they please. My attempts to gatekeep, to shield my component with:

if (isServer) {
  return null;
}

return (
  <Component />
)

...were met with relentless defiance. The component, with its local storage dependencies, crashed and burned, taking my patience along with it.

Then came the revelation—use client. A beacon of hope, promising salvation, only to cast me further into despair with a blank page. A cruel joke, where isServer mocked me with a false negative, leaving the page barren, unresponsive to the so-called "rehydration" process.

The introduction of use client at the top of a component does more than merely suggest a preference for client-side rendering—it enforces a paradigm shift. Without it, React attempts to render components server-side by default, where client-specific objects like window are nonexistent, leading to the inevitable demise of any code relying on them. Quite the paradox on the surface isn't it.

However, use client doesn't simply flip a switch and render everything client-side in a traditional sense. Instead, it signals to NextJS that this component should be considered client-side only, but here's the catch: if it encounters this directive during server-side rendering, it skips the conditional rendering and goes straight into all children components, only to crash and burn.

When the page "rehydrates" on the client side—a process meant to breathe life into server-rendered pages by attaching event listeners and executing any client-side scripts—it respects this initial rendering decision. If the server passed down a blank state with the assumption that use client would kick in later, rehydration doesn't question this. It doesn't attempt to render the component anew; it merely accepts the server's last word as gospel, leaving the page barren, devoid of the expected interactive elements.

But despair not, the sages at Jotai had foreseen such tribulations and bestowed upon us nuggets of advice—the "client only" wrapper—a panacea for the woes of server-side rendering. The documentation, a treasure trove of wisdom, guided me through the storm:

https://jotai.org/docs/utilities/storage#server-side-rendering

https://www.joshwcomeau.com/react/the-perils-of-rehydration/#abstractions

Armed with this newfound knowledge, the build complied. The ordeal was a testament to the labyrinthine complexity of React Server components—a convoluted journey through the bowels of NextJS.

NextJS, in its grand wisdom, takes the sleek, straightforward beauty of React and decides, "You know what this needs? A heap of our own convoluted garbage." It's like someone took a perfectly good dish and decided to smother it in unnecessary, pretentious sauce, transforming a simple meal into a labyrinth of flavors you never asked for.

Turbopack, SSR, SSG, RSC—what are these, the latest trendy acronyms designed to make developers feel inadequate? It's like every month, there's a new buzzword to chase, a new complication to untangle, all while you're bombarded with errors for daring to step an inch outside their narrow path. And don't get me started on "next image." If I wanted a high-maintenance image handler, I'd ask for it. Sometimes, all you yearn for is the humble simplicity of an <img /> tag, no strings attached. But no, that would be too easy, wouldn't it? Yes, I know about Vite but Vite does not have an out the box router. But I don't want to trade one abstraction for another, I just want pure React.

Conclusion

As we draw the curtain on this expedition through the convoluted corridors of NextJS and the nuanced intricacies of state management, I extend my sincerest commendations. You've persevered through the dense thicket of technical jargon, emerging, I hope, with a deeper understanding and perhaps a newfound appreciation for the delicate dance of modern web development.

Until our next foray into the technical unknown, keep pushing boundaries, questioning conventions, and, above all, coding with purpose. The world of web development is richer for your contributions.