Release Notes for MongoDB 8.0

8.0.1 - Oct 9, 2024

Issues fixed:

• SERVER-76883 Reduce chattiness of "Role does not exist" logs for externally sourced users

• SERVER-82221 listCollections and listIndexes should include commit-pending namespaces

• SERVER-94635 Make session refresh parameters configurable

• SERVER-95244 Upsert statements which result in an insert may fail with tassert 9146500 when client connects directly to shard

• WT-13409 One ret in __txn_checkpoint is not handled

• All Jira issues closed in 8.0.1

• 8.0.1 Changelog

8.0.0 - Oct 2, 2024

The rest of this page describes changes and new features introduced in MongoDB 8.0.

Database Profiler

When you specify a filter for the database profiler, you can log operations based on the new workingMillis metric. You can log operations based on both workingMillis and durationMillis and set each metric to a different threshold.

BinData Conversion

Starting in MongoDB 8.0, you can use the $convert operator to perform the following conversions:

• String values to binData values

• binData values to string values

MongoDB 8.0 also includes a new helper expression, $toUUID, which provides simplified syntax for converting strings to UUID values.

$queryStats

Starting in MongoDB 7.1, the $queryStats stage returns statistics for recorded queries.

$rank and $denseRank Behavior

Starting in MongoDB 8.0, null and missing field values in $denseRank and $rank sortBy operations are treated the same when calculating rankings. This change makes the behavior of denseRank and rank consistent with $sort.

Queryable Encryption Range Queries

Starting in MongoDB 8.0, Queryable Encryption supports range queries on encrypted fields using the $lt, $lte, $gt, and $gte operators. For details, see Supported Operations for Queryable Encryption. To enable range queries on encrypted fields, see Create an Encryption Schema.

OCSF Schema for Log Messages

Starting in MongoDB 8.0, you can specify the OCSF schema for audit log messages. The OCSF schema provides logs in a standardized format compatible with log processors.

To set the schema used for log messages, use the auditLog.schema configuration file option.

For example log messages in OCSF format, see OCSF Schema Audit Messages.

Ingress Queue

MongoDB 8.0 introduces a new queue for ingress admission control. Operations that wait for entry into the database from the network enter the ingress queue. By default, the queue is unrestricted, meaning that MongoDB allows all operations to execute through this stage without any queueing. Setting the queue maximum to a specific value allows you to queue operations at this stage if the number of oncurrent operations reaches the specified limit.

Moving a Collection

Starting in MongoDB 8.0, you can move an unsharded collection to a different shard using the moveCollection command.

For details, see Moveable Collections. To get started, see Move a Collection.

Unsharding a Collection

Starting in MongoDB 8.0, you can unshard existing sharded collections with the unshardCollection command or the sh.unshardCollection() method. This operation moves all documents in the collection onto a specified shard or the shard with the least amount of data.

For details, see Unsharded Collections. To get started, see Unshard a Collection.

Config Shards

Starting in MongoDB 8.0, you can configure a config server to store your application data in addition to the usual sharded cluster metadata. A mongod node that provides both config server and shard server functionality is called a config shard. A mongod node that runs as a standalone --configsvr without shard server functionality is called a dedicated config server.

To configure a dedicated config server to run as a config shard, run the transitionFromDedicatedConfigServer command.

To configure a config shard to run as a dedicated config server, run the transitionToDedicatedConfigServer command.

For details, see Config Shard. To get started, see Start a Sharded Cluster with a Config Shard. To convert a replica set to a sharded cluster with a config shard, see Convert a Replica Set to a Sharded Cluster with an Embedded Config Server.

New Database Commands

• abortMoveCollection

• abortUnshardCollection

• moveCollection

• transitionFromDedicatedConfigServer

• transitionToDedicatedConfigServer

• unshardCollection

New mongosh Methods

• sh.abortMoveCollection()

• sh.abortUnshardCollection()

• sh.moveCollection()

• sh.shardAndDistributeCollection()

• sh.unshardCollection()

serverStatus Metrics

Starting in MongoDB 8.0, serverStatus includes the following new shardingStatistics fields in its output:

• shardingStatistics.countTransitionToDedicatedConfigServerStarted

• shardingStatistics.countTransitionToDedicatedConfigServerCompleted

• shardingStatistics.countTransitionFromDedicatedConfigServerCompleted

• shardingStatistics.configServerInShardCache

MongoDB 7.1 includes the following new sharding statistics for chunk migrations:

• shardingStatistics.countDonorMoveChunkCommitted

• shardingStatistics.countDonorMoveChunkAborted

• shardingStatistics.totalDonorMoveChunkTimeMillis

• shardingStatistics.countBytesClonedOnRecipient

• shardingStatistics.countDocsClonedOnCatchUpOnRecipient

• shardingStatistics.countBytesClonedOnCatchUpOnRecipient

Default Chunks Per Shard

Starting in MongoDB 7.2, when you shard an empty collection with a hashed shard key, the operation creates one chunk per shard by default. Previously, the operation created two chunks by default.

directShardOperations Role

Starting in MongoDB 8.0, you can use the directShardOperations role to perform maintenance operations that require you to execute commands directly against a shard.

Exhaust Cursors Enabled for Sharded Clusters

Starting in MongoDB 7.1, mongos supports exhaust cursors when a client's getMore request sets the exhaustAllowed flag. This can improve query performance on sharded clusters when the client receives multiple replies from the database server for a single request.

mongos Port Range

Starting in MongoDB 7.1, mongos accepts --port values from [0, 65535]. For more information, see --port.

Query with Partial Shard Key

Starting in MongoDB 7.1, findAndModify and deleteOne() can use a partial shard key to query on a sharded collection.

Resharding Improvements

MongoDB 7.2 introduces significant performance improvements in reshard collection operations, substantially reducing the amount of time the operation takes to run.

Additionally, if your application and cluster meet the necessary requirements and limitations, you can reshard the collection on the same key using the reshardCollection command to redistribute your collection, which is much faster than alternative range migration procedure.

The following options are added to the command:

For examples, see Redistribute Data to New Shards.

Self-Managed Backups of Sharded Clusters

Starting in MongoDB 7.1, the fsync and fsyncUnlock commands can perform fsync operations on sharded clusters.

When run on mongos with the lock field set to true, the fsync command flushes writes from the storage layer to disk and locks each shard, preventing additional writes. The fsyncUnlock command can then be used to unlock the cluster.

This feature enables self-managed backups of sharded clusters using mongodump.

UpdateOne upsert Behavior on Sharded Collections

Starting in MongoDB 7.1, when using updateOne() with upsert: true on a sharded collection, you do not need to include the full shard key in the filter.

$lookup Stage in Transactions with Sharded Collections

Starting in MongoDB 8.0, you can use the $lookup stage within a transaction while targeting a sharded collection.

Majority Write Concern

Starting in MongoDB 8.0, write operations that use the "majority" write concern return an acknowledgment when the majority of replica set members have written the oplog entry for the change. This improves the performance of "majority" writes. In previous releases, these operations would wait and return an acknowledgment after the majority of replica set members applied the change.

If your application reads from a secondary immediately after receiving an acknowledgment from a { w: "majority" } write operation (without a causally consistent session), the query may return results that don't include changes from the write.

New replSetGetStatus Metrics

Starting in MongoDB 8.0, the following metrics are available when using the replSetGetStatus command:

• replSetGetStatus.electionCandidateMetrics.lastSeenWrittenOpTimeAtElection

• replSetGetStatus.electionParticipantMetrics.lastWrittenOpTimeAtElection

• replSetGetStatus.electionParticipantMetrics.maxWrittenOpTimeInSet

• replSetGetStatus.members[n].optimeWritten

• replSetGetStatus.members[n].optimeWrittenDate

• replSetGetStatus.members[n].lastWrittenWallTime

• replSetGetStatus.optimeWritten

• replSetGetStatus.optimeWrittenDate

• replSetGetStatus.optimes.lastWrittenWallTime

• replSetGetStatus.optimes.writtenOpTime

Oplog Buffers

Starting in MongoDB 8.0, secondaries write and apply oplog entries for each batch in parallel. A writer thread reads new entries from the primary and writes them to the local oplog. An applier thread asynchronously applies these changes to the local database. This changes increases replication throughput for secondaries.

This introduces a breaking change for metrics.repl.buffer, as it now provides metrics on two buffers instead of one.

MongoDB 8.0 deprecates the following server status metrics:

• metrics.repl.buffer.count

• metrics.repl.buffer.maxSizeBytes

• metrics.repl.buffer.sizeBytes

MongoDB 8.0 adds the following server status metrics:

• metrics.repl.buffer.apply

• metrics.repl.buffer.apply.count

• metrics.repl.buffer.apply.maxCount

• metrics.repl.buffer.apply.maxSizeBytes

• metrics.repl.buffer.apply.sizeBytes

• metrics.repl.buffer.write

• metrics.repl.buffer.write.count

• metrics.repl.buffer.write.maxSizeBytes

• metrics.repl.buffer.write.sizeBytes

• metrics.repl.write

• metrics.repl.write.batchSize

• metrics.repl.write.batches

• metrics.repl.write.batches.num

• metrics.repl.write.batches.totalMillis

serverStatus Metrics

Starting in MongoDB 8.0, the following new serverStatus metrics report information about tcmalloc usage:

• tcmalloc.usingPerCPUCaches

• tcmalloc.generic.peak_memory_usage

• tcmalloc.tcmalloc.cpu_free

• tcmalloc.tcmalloc.total_bytes_held

• tcmalloc.tcmalloc.cpuCache

• tcmalloc.tcmalloc.cpuCache.N.overflows

• tcmalloc.tcmalloc.cpuCache.N.underflows

• tcmalloc.tcmalloc_derived.total_free_bytes

Shutdown Performance

Starting in MongoDB 8.0, Bulk.insert() and data ingestion workloads may perform better. However, shutting down MongoDB immediately after running these workloads might take longer because of the extra data being flushed to disk.

Store Application Data on Config Shards

Starting in MongoDB 8.0, you can configure a config server to store application data in addition to the usual sharded cluster metadata. The config server is then known as a config shard. For details, see Config Shards.

Compaction Improvements

Starting in MongoDB 7.3, the compact command includes a new freeSpaceTargetMB option to specify the minimum amount of storage space, in megabytes, that must be recoverable for compaction to proceed.

Concurrent DDL Operations

Starting in MongoDB 7.1, when you run multiple DDL operations that target different collections from the same database, MongoDB runs those operations concurrently.

This change adds two new types to the serverStatus locks field and currentOp.locks output:

• DDLDatabase

• DDLCollection

Database Validation on mongos Aggregation Queries

Starting in MongoDB 7.2, aggregation pipeline queries that attempt to use non-existent databases on mongos deployments return validation errors.

In previous versions, these aggregation queries return empty cursors.

DDL Operations

In MongoDB 8.0, if you add or remove a shard while your cluster executes a DDL operation (operation that modifies a collection such as reshardCollection), any operation that adds or removes a shard only executes after the concurrent DDL operation finishes.

Error Codes for Exceeding Pipeline Size Limit

Starting in MongoDB 7.1, an aggregation command will throw an error when a pipeline exceeds the pipeline stage limit. For more details, see Number of Stages Restrictions.

getField Field Supports All Strings

Starting in MongoDB 7.2, you can specify any valid expression that resolves to a string for the field input of the $getField operator. In earlier versions, field accepts only string constants.

Improved Index Builds

Starting in MongoDB 7.1, index builds are improved with faster error reporting and increased failure resilience. You can also set the minimum available disk space required for index builds using the new indexBuildMinAvailableDiskSpaceMB parameter, which stops index builds if disk space is too low.

The following new index build metrics were added:

• indexBuilds.killedDueToInsufficientDiskSpace

• indexBuilds.failedDueToDataCorruption

For full details, see Index Builds.

New Parameters

New Bulk Write Command

Starting in MongoDB 8.0, you can use the new bulkWrite command to perform many insert, update, and delete operations on multiple collections in one request. The existing db.collection.bulkWrite() method only allows you to modify one collection in one request.

New Query Shape and Query Settings

MongoDB 8.0 introduces a new query shape. The pre-existing query shape is renamed as the plan cache query shape.

Starting in MongoDB 8.0, you can add query settings for the new query shape. The query optimizer uses the query settings as an additional input during query planning, which affects the plan selected to run a query that has a matching query shape.

Query settings have improved functionality compared to index filters. Index filters are also deprecated starting in MongoDB 8.0, and you should avoid using them.

• To add query settings, use the setQuerySettings command.

• To delete query settings, use the removeQuerySettings command.

• To retrieve query settings, use a $querySettings stage in an aggregation pipeline.

• To block a query shape, see operation rejection filters.

Starting in MongoDB 8.0, queryShapeHash is included in the following output:

• explain command in the explain.queryShapeHash field

• database profiler in the system.profile.queryShapeHash field

• $currentOp aggregation pipeline stage in the $currentOp.queryShapeHash field

• slow query log

Starting in MongoDB 8.0, the pre-existing queryHash field is renamed to planCacheShapeHash. If you're using an earlier MongoDB version, you'll see queryHash instead of planCacheShapeHash.

Parameter Filtering

Starting in MongoDB 8.0, the getParameter command accepts a setAt field. You can use this field to filter the allParameters: true return document to show only those parameters that you can set at startup or runtime.

Read Concern on Capped Collections

Starting in MongoDB 8.0, you can use read concern "snapshot" on capped collections.

serverStatus Output Change

Starting in MongoDB 7.1, the serverStatus command output includes the following new metrics:

• changeStreamPreImages.expireAfterSeconds

• metrics.changeStreams.showExpandedEvents

• queryStats

Starting in MongoDB 7.2, the serverStatus command output includes the following new metrics:

• metrics.operation.killedDueToMaxTimeMSExpired

• readPreferenceCounters

Starting in MongoDB 7.3, the serverStatus command output includes the following new metrics:

• metrics.repl.stateTransition.totalOperationsRunning

• metrics.repl.stateTransition.totalOperationsKilled

• metrics.repl.waiters.replication

• metrics.repl.waiters.opTime

• planCache.classic.skipped

• planCache.sbe.skipped

Starting in MongoDB 8.0, the serverStatus command output includes the following new metrics:

• opWorkingTime

• opWorkingTime.commands

• opWorkingTime.reads

• opWorkingTime.writes

• opWorkingTime.transactions

Sort Support for updateOne

Starting in MongoDB 8.0, the updateOne() method supports a sort option. This controls how MongoDB orders the document before it selects the document to receive the update.

Previous releases use the findAndModify() and findOneAndUpdate() methods to update the first document in a user-specified sort order. Support for retryable writes requires these methods to copy the entire document to a special side collection for each node, which is a more expensive operation than the updateOne() method with the new sort option.

Specify Query Index Hints for distinct Commands

Starting in MongoDB 7.1, the hint field is available in the distinct command, allowing you to specify the query's index.

TTL Indexes

Starting in MongoDB 7.1, you can create TTL indexes on capped collections.

Query Planning and Execution

Batch Multi-Document Insert Operations

Starting in MongoDB 8.0, insert operations for multi-document transactions no longer produce individual oplog entries. Instead, multi-document inserts are batched as a single entry. The change stream insert event for each document has the same clusterTime.

This change increases performance of multi-document insert operations and eliminates possible replication lag caused by multiple oplog writes.

OIDC Identity Providers Can Share an Issuer

Starting in MongoDB 8.0, when multiple identity providers (IDP) are defined, the oidcIdentityProviders parameter accepts duplicate issuer values as long as the audience value is unique for each issuer. This is also available in versions 7.3 and 7.0.

Namespaces in Subpipelines

Starting in MongoDB 8.0, namespaces in subpipelines inside $lookup and $unionWith are validated to ensure the correct use of from and coll fields.

For details, see $lookup subpipelines and $unionWith subpipelines.

Query Planner Optimization Time

The explain() method now returns information on query planner optimization time. The queryPlanner.optimizationTimeMillis status shows the time in milliseconds that the query planner spent on optimizations.