Monday , October 26 2020

Building real-time collaboration applications: OT vs CRDT, Hacker News


Team Tiny has begun work on a real-time collaboration solution for TinyMCE. This is the first in a series of blog posts documenting our journey and decisions along the way.

If you dig below the surface of any real-time collaboration solution you’re likely to start hearing about the terms OT (Operational Transformation) and CRDT (Conflict-free Replicated Data Type) . For a team trying tobuilda real-time collaboration solution, the first question to ask is “which of these two acronym mouthfuls should I use”? Followed quickly by “what on earth do these acronym mouthfuls mean”?

The full response to these questions took me about a month to answer. There is a vast collection of reading material available; navigating that rabbit hole was a fun journey. I will include links at the end to the research and reading material I used to form my conclusions.

At a very high level, this is what we’re dealing with:

  1. Remove text after the split point in the node
  2. Insert a new sibling text node after the current node containing the removed text

This might not be exactly how CRDT algorithms represent these changes, but my point is they do not transmit the intent. OT trades complexity for the ability to capture the intent; CRDT has less complexity but can only guarantee all clients end with the same data – however that data might not be the intended structure, or even valid for your schema.

    Credit:Martin Kleppmann,The effect of this trade-off is that while there are some editors available today with support for collaboration using CRDT, every single one of them has compromises in depth of features and user experience.

    If you can show me a rich text editor with support for CRDT, I can show you why I struggle to call it a “rich” text editor. Perhaps we should call them moderately wealthy text editors. Limiting the breadth of HTML capability for ease of collaboration is a popular trend, but not one that we wish to engage with for TinyMCE.

    In the worst case, when applying CRDT mechanisms to a full rich text editor, the user experience can be compromised due to the lack of intent. During my research I was already leaning towards OT, but this video shows the point at which I decided CRDT was not ready for our needs. This is aconvergence.io demoshowing their collaboration server linked to the Froala rich text editor.

    ()

    This is the exact “split node” scenario I described earlier; Applying bold near the start of a text node necessarily has to split the text node into three parts (before, bold, and after). With a CRDT implementation, when the collaborating user’s cursor is in the “after” section, the application of bold recreates the text node and the user has a poor experience.

    Conclusions

    While my example is very simple and could be managed so the cursor does not move, I saw it as a sign of unknown and potentially far greater pain. The risks of finding an unsolvable issue on that path are too great, compared to the very complex but known feasibility of Operational Transformation on a rich text model.

    CRDT is the holy grail of collaboration, It’s an active area of ​​research, and the prospect of peer-to-peer editing with end-to-end encryption is an exciting one. The technology isn’t ready for our needs yet, but I believe in the future some incredible products will be made possible. In the meantime TinyMCE will rely on OT for collaboration, coming later this year.

    Research links (in no particular order):

    Conflict Resolution for Eventual Consistency, Martin Kleppmann**********************************

    **************************** CRDTs and the Quest for Distributed Consistency, Martin KleppmannLessons learned from creating. a rich-text editor with real-time collaboration, CKSource

    Why CRDT did not work out as well for collaborative editing xi- editor, 7910 Hacker News thread (links to github)

    CR DT for Collaborative Editing apps, how good is it?, Iristate

    Understanding and Applying Operational Transformation, Daniel Spiewak 2017

    Creating a Collaborative Editor

    , Pierre Hedkvist

  1. A Conflict-Free Replicated JSON Datatype, Martin Kleppmann and Alastair R. Beresford

  2. Towards a unified theory of Operational Transformation and CRDT, Raph Levien

    Data Laced with History: Causal Trees & Operational CRDTs (****************, Alexei Baboulevitch (Visualization of OT with a central server, Tim Baumann 2012 (Real Differences between OT and CRDT for Co-Editors, 2018 Hacker News thread (links to paper) *

    * This last paper was updated in and split into three parts:

About admin

Leave a Reply

Your email address will not be published. Required fields are marked *