Multi-user collaborative text editing with ShareDB and CodeMirror

Do you desire to add a multi-users collaborative text editing to your application? This article will show you how to do it by using ShareDB and the CodeMirror library.

ShareDB which used to be ShareJS is a real time database backend based on Operational Transformation (OT) of JSON documents, it allow real-time synchronization of documents and many other nifty things, in this article, i will focus on the implementation of a multi-user collaborative text editor by using ShareDB and CodeMirror which is a versatile text editor implemented in JavaScript for browsers.

There is some projects that exist already such as a CodeMirror binding for ShareDB and ShareJS but they are outdated or not working correctly, as there is limited resources on this subject, i am sharing my implementation for CodeMirror, it can also be useful for other editors such as the Ace Editor.

In this article, i assume that there is already a CodeMirror instance which is named “code_editor” on the client side.

The Node.js server is extremely simple, ShareDB will do everything, all alone… we will just make ShareDB listen to a WebSocketJSONStream :

var http = require('http');
var express = require('express');
var ShareDB = require('sharedb');
var ShareDB_logger = require('sharedb-logger');
var WebSocket = require('ws');
var WebSocketJSONStream = require('websocket-json-stream');

var share = new ShareDB();

var sharedb_logger = new ShareDB_logger(share);

var app = express();
var server = http.createServer(app);

var wss = new WebSocket.Server({server: server});
wss.on('connection', function(ws, req) {
    console.log("client connected");
    
    var stream = new WebSocketJSONStream(ws);
    share.listen(stream);
});

wss.on('close', function(ws, req) {
    console.log("client disconnected");
});

server.listen(3000);

Here are the dependencies to add to the “package.json” file to build the server with “npm” :

    "dependencies": {
        "sharedb": "1.0.0-beta.6",
        "ot-text": "1.0.1",
        "websocket-json-stream": "0.0.3",
        "express": "4.14.0",
        "ws": "1.1.1",
        "sharedb-logger": "0.1.4"
    }

sharedb-logger” is useful to monitor all ShareDB messages.

On the client side, you need to connect to the server through WebSocket (ShareDB is transport agnostic but it will require additional code to implement other transports) :

var ws = new WebSocket("ws://127.0.0.1:3000"); // Connect to localhost on port 3000

Then bind ShareDB to the WebSocket :

var sharedb_connection = new ShareDB.Connection(ws);

Now it is time to get the document we are interested in :

sharedb_doc = sharedb_connection.get("my_collection", "my_document");

We are getting a document named “my_document” in the collection “my_collection”, if the connection is established and the document exist on the server, we will get the latest snapshot of that document from the server, if it does not exist already, we will create it and set the document content to the text editor content, if it exist we just assign the document content to our text editor, we do that by using the document function “subscribe“, we will also start to listen to the “op” event to integrate remote document changes to our CodeMirror instance.

    sharedb_doc.subscribe(function(err) {
        if (err) {
            console.log(err); // handle the error
        }
        
        if (!sharedb_doc.data) { // does not exist so we create the document and replace the code editor content by the document content
            sharedb_doc.create(code_editor.getValue());
        } else { // it exist, we set the code editor content to the latest document snapshot
            code_editor.setValue(sharedb_doc.data);
        }

        // we listen to the "op" event which will fire when a change in content (an operation) is applied to the document, "source" argument determinate the origin which can be local or remote (false)
        sharedb_doc.on('op', function(op, source) {
            var i = 0, j = 0,
                from,
                to,
                operation,
                o;
            
            if (source === false) { // we integrate the operation if it come from the server
                for (i = 0; i < op.length; i += 1) {
                    operation = op[i];
                    
                    for (j = 0; j < operation.o.length; j += 1) {
                        o = operation.o[j];
                        
                        if (o["d"]) { // delete operation
                            from = code_editor.posFromIndex(o.p);
                            to = code_editor.posFromIndex(o.p + o.d.length);
                            code_editor.replaceRange("", from, to, "remote");
                        } else if (o["i"]) { // insert operation
                            from = code_editor.posFromIndex(o.p);
                            code_editor.replaceRange(o.i, from, from, "remote");
                        } else {
                            console.log("Unknown type of operation.")
                        }
                    }
                }
            }
        });
        
        sharedb_doc_ready = true; // this is mandatory but we will use this to determine if the document is ready in the "change" event of CodeMirror
    });

You can look here for informations concerning the text operations, this is pretty easy to manipulate because there is basically two types, insert, delete and both come with the position from which it happen, we need to do some conversions for the position because CodeMirror use lines and characters, ShareDB just use characters. The very important things is the last argument that we pass to CodeMirror “replaceRange” function, because all of these operations will fire the “changes” event of CodeMirror which we will listen to submit operations… so if we want to avoid cyclic changes problem, we need to “type” the origin by setting it as “remote” and we will ignore the change with this origin in the CodeMirror “changes” event.

Now it is time to listen to changes from CodeMirror through the “changes” event and push the operation to ShareDB :

    CodeMirror.on(code_editor, 'changes', function (instance, changes) {
        var op,
            change,
            start_pos,
            chars,

            i = 0, j = 0;

        if (!sharedb_doc_ready) { // if the document is not ready, we just ignore all changes, a much better way to handle this would be to call the function again with the same changes at regular intervals until the document is ready (or just cancel everything if the document will never be ready due to errors or something else)
            return;
        }

        op = {
            p: [],
            t: "text0",
            o: []
        };

        // we must do it in order (this avoid issue with same-time op)
        changes.reverse();

        for (i = 0; i < changes.length; i += 1) {
            change = changes[i];
            start_pos = 0;
            j = 0;

            if (change.origin === "remote") { // do not submit back things pushed by remotes... ignore all "remote" origins
                continue;
            }

            while (j < change.from.line) {
                start_pos += code_editor.lineInfo(j).text.length + 1;
                j += 1;
            }

            start_pos += change.from.ch;

            if (change.to.line != change.from.line || change.to.ch != change.from.ch) {
                chars = "";

                for (j = 0; j < change.removed.length; j += 1) { chars += change.removed[j]; if (j !== (change.removed.length - 1)) { chars += "\n"; } } op.o.push({ p: start_pos, d: chars }); } if (change.text) { op.o.push({ p: start_pos, i: change.text.join('\n') }); } } if (op.o.length > 0) {
            sharedb_doc.submitOp(op);
        }
    });

Here we are looping through all changes that happened, determinate what kind of change it is and the start position (remember: CodeMirror use lines and characters so we have to do some computations to transfom lines&characters to a character based position), we then submit the operation to ShareDB.

This is it, you should have a multi-user collaborative text editor working fully with CodeMirror…

For an easier way to add collaborative stuff to your application , you can also integrate TogetherJS which a library which will almost “automatically” allow it for many things and provide some nice widgets such as a chatbox, audio chat… my preference goes for ShareDB as it seem easier to control and manage while remaining fairly easy to use when doing advanced stuff and it does not have any fancy stuff.

09/08/2017: Added `changes.reverse()` to the CodeMirror changes function so that changes are processed in order, this avoid issues with same-time operations.

1 Star2 Stars3 Stars4 Stars5 Stars (No Ratings Yet)
Loading...


5 thoughts on “Multi-user collaborative text editing with ShareDB and CodeMirror”

  1. Thank you for the details! I’m the author of Joukkue, a collaborative creative coding tool based on nodejs. I made a branch last year called realtime, which uses sharejs and codemirror and I hope I can use your post to update it to sharedb.

    Since you’re also interested in art, demoscene, coding… might you be interested also in checking out Joukkue? It uses p5.js, but it could be used with other libraries for graphics and sound. It’s fun to use with some friends for real time coding graphics. I’ve used it in a performance once.

    It would be great to get some help with it 🙂 Cheers!

  2. Hello,

    feel free to ask if you have any questions regarding sharedb, (or anything you think i could help) this article was made after i implemented sharedb in my live coding synthesizer, it should work as expected 🙂

    Joukkue seem pretty nice, i might try it if i have more time, i especially like the idea of layers, you also have some nice projects on your website.

    Cheers!

  3. Hi Julien,

    I’m the author of the sharedb-codemirror library linked here, which was an adaptation of the earlier share-codemirror.

    Do you mind filing an issue on the GitHub repo for what wasn’t working? The hope is of course that people don’t have to re-implement the integration in their own projects, so any suggestions or feedback would be really useful. PRs are welcome too!

    Thanks!

    1. Hello Evan,

      i will see if i can do it this week, it was some kind of errors as i remember but as i was on a rush, i did not have the time to fill an issue, it would be useful to have a working implementation since many peoples seem to look for one.

Leave a Reply

Your email address will not be published. Required fields are marked *

*