Why is not “majority” the default learn concern in MongoDB?

TL;DR: If you happen to’re used to conventional SQL databases and synchronous request–response flows—the place you learn your writes in the identical transaction or session—use the “majority” learn concern in MongoDB and you’ll have the very best isolation and sturdiness you possibly can count on from a database. It’s not the default, nevertheless it’s protected to alter it on your connection. The default is optimized for event-driven, microservice architectures with asynchronous communication, the place decrease latency is most well-liked even when it means generally studying a state that will later be rolled again.

PostgreSQL customers usually count on writes to develop into seen to different classes solely after they’re acknowledged, both through auto-commit DML or an specific COMMIT. Against this, in MongoDB, it’s essential to allow the “majority” learn concern to attain comparable ACID ensures, and this isn’t the default. It might appear stunning that MongoDB gives the strongest consistency possibility—full ACID semantics in a distributed database—but doesn’t allow it by default, regardless of seemingly no vital efficiency impression. This caught my consideration and made me wish to perceive the reasoning behind it. NoSQL and SQL now tackle comparable use instances, however their origins are essentially completely different. Let’s discover that.

Non-blocking learn and write considerations

Within the SQL normal, isolation ranges have been first outlined by the anomalies (phenomena) that may happen when concurrent classes learn and write the identical information. However these definitions have been tied to a particular lock-based implementation somewhat than an summary mannequin: they assumed that reads and writes use locks and that energetic transactions share and modify a single present database state.

In actuality, many databases selected completely different designs for scalability:

• Non-blocking reads with MVCC (e.g., PostgreSQL or MongoDB) present anomalies not lined by the usual—”write skew,” as an illustration—and help isolation ranges like Snapshot Isolation (SI), which differs from the SQL definitions, regardless that PostgreSQL makes use of the identify Repeatable Learn to match the SQL normal.
• Non-blocking writes (e.g., in MongoDB) detect write conflicts instantly and lift a retryable exception as a substitute of ready for lock acquisition, often known as optimistic concurrency management.

To know isolation and sturdiness in MongoDB, we should first think about learn and write considerations independently, particularly in a replicated, distributed setup the place reads and writes can hit completely different servers. Then we are able to study how they work together after we learn after writing.

Isolation and sturdiness

First, let’s distinguish isolation and sturdiness — the I and D in ACID:

• Isolation defines how reads and writes from completely different classes are seen to 1 one other. To protect atomicity, it should cover intermediate states of uncommitted writes till the transaction completes and must also stop stale reads that miss beforehand dedicated writes.
• Sturdiness ensures that after information is written, it stays persistent and isn’t misplaced after a failure. Equally, to stop soiled reads which may later be rolled again throughout failure restoration, information that has already been learn must also be assured to stay persistent.

Initially, these definitions assumed a single-node database. In fashionable methods, sturdiness should additionally deal with community and information middle failures, so information is persevered throughout a number of nodes somewhat than simply on a neighborhood disk.

A commit, whether or not in an specific transaction or implicit in a write operation, usually proceeds as follows:

1. Commit is initiated.
2. The write-ahead log is flushed to native disk (native sturdiness).
3. The write-ahead log is flushed to the distant disk (world sturdiness).
4. Modifications develop into seen (finish of isolation) to different classes.
5. The commit is acknowledged within the session.

Sturdiness and isolation every contain a number of operations, and their order can fluctuate. The sequence above matches PostgreSQL with synchronous_commit = on, or MongoDB with w:majority and a majority learn concern in different classes.

Different configurations are attainable. For instance, Oracle Database makes use of a unique order for sturdiness and isolation, making modifications seen earlier than the redo log is flushed (besides when paranoid_concurrency_mode is ready), and remains to be thought of as strongly constant. With PostgreSQL synchronous_commit = native or MongoDB w:1, acknowledgment happens earlier than world sturdiness. With MongoDB’s native learn concern, information turns into seen earlier than it’s sturdy.

Why isn’t the above sequence—which appears to supply the strongest isolation and sturdiness—the default in MongoDB?

Learn after a write with asynchronous calls

There’s one other anomaly not described by the SQL normal, which assumes that learn and write locks on a single database state are mutually unique. With MVCC, a transaction as a substitute works with two states:

• Learn time is the beginning of the transaction (or the beginning of the assertion in Learn Dedicated transactions). All reads use a snapshot from this time.
• Write time is the top of the transaction, since all writes should seem to happen atomically at commit.

As a result of the learn time is sooner than the write time, one other anomaly can happen:

1. Microservice A writes an occasion, assumes will probably be persevered and visual, and notifies microservice B.
2. Microservice B receives the notification and reads the occasion, assuming it’s seen.
3. Microservice A receives the write acknowledgment a couple of milliseconds later, particularly if world sturdiness should be confirmed.

In a non-MVCC database with blocking reads, this preserves causality as a result of, in step 2, microservice B requires a share lock and waits on an unique lock acquired by A and launched at step 3, so B sees the write solely after it acquires the share lock, after step 3. Non-MVCC is uncommon (e.g., DB2 or SQL Server with out RCSI isolation degree), however SQL isolation ranges have been outlined based mostly on it, and did not point out causality.

Remember the fact that on this instance, the appliance doesn’t look forward to the write acknowledgment earlier than telling the opposite service to learn, but it nonetheless expects the write to be full when the learn happens. Learn-after-write causality was assured with learn locks within the non-MVCC database.

Nonetheless, in an MVCC database, as in most fashionable methods, microservice B might learn a state from earlier than a write is seen, inflicting a read-after-write anomaly. If the write is acknowledged solely domestically—for instance, PostgreSQL with synchronous_commit = native or MongoDB with w:1—it’s going to possible be seen by the point B receives the notification, as a result of the write normally completes sooner than the notification is delivered.

Against this, PostgreSQL with synchronous_commit = on, or MongoDB with majority learn concern, might not see the write but if it has not been replicated to a majority. Thus, when utilizing w:1, customers ought to choose the native learn concern to keep away from read-after-write anomalies. w:1 shouldn’t be the default. Nonetheless, it may be chosen to cut back latency, on the threat of dropping occasions on failure—one thing event-driven architectures can usually tolerate.

With PostgreSQL synchronous_commit = on or MongoDB w:majority (the default), writes incur additional community latency as a result of they need to look forward to distant acknowledgment. On this case, the state of affairs can nonetheless present a read-after-write anomaly if the bulk has not but acknowledged microservice A’s write when microservice B reads. Utilizing MongoDB native learn concern avoids this anomaly, however dangers studying information which may later be rolled again on failure.

“native” is the default, however use “majority”

The default learn concern is well-suited to event-driven architectures. As event-driven methods have been a major use case for NoSQL databases like MongoDB, retaining this default is smart, not less than for backward compatibility. Builders might use asynchronous database calls and sometimes count on reads to return the newest modifications, even when these modifications haven’t but been acknowledged within the thread that carried out the write operation.

Right this moment, MongoDB can also be used with conventional architectures, the place it’s affordable to choose sturdiness over quick visibility and use the “majority” learn concern. This provides no efficiency penalty, since you already paid the synchronization latency when ready for the write acknowledgment. “Majority” learn concern units the learn time to the final commit time, whereas retaining reads native. It will possibly wait in uncommon instances, reminiscent of throughout occasion startup or rollback, till it could actually acquire a dedicated timestamp snapshot, or when secondaries are unavailable or lagging. However typically, there is not any efficiency impression.

Not like SQL databases—which should assure consistency for any DML executed by any person, together with non-programmers on the command line—MongoDB shifts extra duty to builders. As an alternative of counting on a one-size-fits-all default, builders should configure their session or connection by selecting:

• the write concern (for instance, w:majority for sturdiness over community or information middle failures),
• the learn concern (reminiscent of majority, or snapshot for stronger consistency in multi-shard transactions), and
• the learn choice (to scale reads throughout replicas when some staleness is suitable).

This configuration lets MongoDB adapt to completely different consistency and efficiency expectations, with specific settings:

// greatest consistency:
mongodb+srv://mongo.internet/take a look at?w=majority&readConcernLevel=majority&readPreference=major

// quick visibility:
mongodb+srv://mongo.internet/take a look at?w=majority&readConcernLevel=native&readPreference=major

// quick write:
mongodb+srv://mongo.internet/take a look at?w=1&readConcernLevel=native&readPreference=major

Observe that readPreference=major is the default as a result of secondary reads might miss latest modifications, as not all secondaries have to acknowledge a w:majority write. I wish to set it explicitly within the microservice connection to make the anticipated consistency apparent.

Last advice: don’t use majority learn concern with w:1 writes, since it’s possible you’ll not be capable of learn your personal acknowledged writes till they attain a majority of replicas. This can be one more reason to maintain native because the default learn concern, as some writes might use w:1 as a substitute of w:majority.