Provisioning MongoDB with Ops Manager, end to end

Atlas is the right answer for most teams. This post is for the teams it is not the right answer for — where the data has to stay inside a boundary you control, and MongoDB Enterprise with Ops Manager becomes the deployment model.

Ops Manager is where most self-managed MongoDB projects go wrong, because it looks like one product and is actually four: an application, a database for that application, a place to stream oplogs, and a place to keep snapshots. Get the topology right up front and it runs quietly for years. Get it wrong and you will discover the problem during your first restore, which is the worst possible moment.

What Ops Manager actually is

Ops Manager is a control plane. It does not sit in the data path of your application. Instead, an Automation Agent runs next to each mongod and continuously reconciles the local process against a goal state published by the Ops Manager server. You describe the deployment you want; the agents make it true.

That indirection is the whole value proposition. A version upgrade across a 30-node sharded cluster becomes a change to one document rather than 30 careful SSH sessions, and the agents roll it through in the correct order without dropping a primary.

Four components, and they must be sized independently:

Ops Manager application

Stateless JVM web app. Scale horizontally behind a load balancer.

Application Database

A dedicated MongoDB replica set holding all Ops Manager state. Not shared with your data.

Oplog Store

A MongoDB replica set receiving a continuous oplog stream from every backed-up deployment.

Snapshot Store

S3-compatible object storage, or a MongoDB blockstore. This is where the bytes live.

Sizing before you install

Three numbers determine everything. Work them out before you provision anything.

Total hosts under management. Ops Manager holds roughly 1–2 GB of Application Database storage per monitored host per year, dominated by monitoring rollups. Fifty hosts is a comfortable 200 GB volume with room to grow; five hundred is a different conversation.

Daily oplog churn. Sum the oplog write rate across every deployment you intend to back up. The Oplog Store must absorb all of it, continuously, and it only retains data between snapshots — so size it for your snapshot interval plus a healthy safety margin, not for retention.

Snapshot footprint. Data size × snapshot count × change rate. MongoDB's blockstore deduplicates aggressively, so this is usually far smaller than the naive multiplication suggests, but "usually" is not a capacity plan. Measure the first month.

A workable starting point for a fifty-host estate:

Deploying Ops Manager with the Kubernetes operator

You can install Ops Manager from an RPM onto a VM, and for a long time that was the only way. If you already run Kubernetes, the operator is a better deal: the Application Database, the application itself and every managed deployment become declarative resources that reconcile.

Install the operator

helm repo add mongodb https://mongodb.github.io/helm-charts
helm repo update
 
kubectl create namespace mongodb
 
helm install mongodb-kubernetes mongodb/mongodb-kubernetes \
  --namespace mongodb \
  --set operator.watchNamespace="mongodb,data-prod" \
  --version <pin-me>

Two flags matter more than the rest. operator.watchNamespace decides the operator's blast radius — list namespaces explicitly rather than watching everything, so a mistake in a development namespace cannot reconcile something in production. And pin the chart version: an unpinned operator upgrade will happily try to move Ops Manager with it.

Confirm the CRDs landed:

kubectl get crd | grep mongodb.com
# mongodb.mongodb.com
# mongodbusers.mongodb.com
# opsmanagers.mongodb.com

Credentials

Ops Manager needs a first admin user. The operator reads it from a Secret and creates the account on first boot.

kubectl -n mongodb create secret generic om-admin \
  --from-literal=Username="admin@example.com" \
  --from-literal=Password="$(openssl rand -base64 24)" \
  --from-literal=FirstName="Ops" \
  --from-literal=LastName="Admin"

The Ops Manager resource

This one manifest describes the application, the Application Database beneath it, and both backup stores.

apiVersion: mongodb.com/v1
kind: MongoDBOpsManager
metadata:
  name: ops-manager
  namespace: mongodb
spec:
  version: 8.0.0
  replicas: 2
  adminCredentials: om-admin
 
  applicationDatabase:
    members: 3
    version: 8.0.0-ent
    podSpec:
      persistence:
        single:
          storage: 250G
          storageClass: fast-ssd
 
  configuration:
    mms.fromEmailAddr: ops-manager@example.com
    mms.ignoreInitialUiSetup: "true"
 
  backup:
    enabled: true
    opLogStores:
      - name: oplog-primary
        mongodbResourceRef:
          name: oplog-store
    s3Stores:
      - name: snapshot-primary
        s3SecretRef:
          name: s3-credentials
        pathStyleAccessEnabled: true
        s3BucketEndpoint: https://objects.internal.example.com
        s3BucketName: mongodb-snapshots
        mongodbResourceRef:
          name: oplog-store
 
  externalConnectivity:
    type: LoadBalancer
    port: 8080

A few decisions are buried in there.

replicas: 2 gives you a rolling upgrade path for the application tier. It is stateless, so this costs you nothing but a second pod.

mms.ignoreInitialUiSetup: "true" skips the browser-based first-run wizard. You want this: the wizard writes configuration that your manifest does not know about, and the next reconcile will fight it.

The s3Stores entry still carries a mongodbResourceRef. That is not a mistake — even with snapshots in object storage, Ops Manager keeps snapshot metadata in a MongoDB deployment, and here we are reusing the oplog store replica set for it.

pathStyleAccessEnabled: true is required for most on-premises S3 implementations, including Nutanix Objects and MinIO, which do not support virtual-host-style bucket addressing.

The oplog store

The oplog store is just another MongoDB resource that the operator manages. It must exist before Ops Manager reconciles backup, so apply it first.

apiVersion: mongodb.com/v1
kind: MongoDB
metadata:
  name: oplog-store
  namespace: mongodb
spec:
  members: 3
  version: 8.0.0-ent
  type: ReplicaSet
  opsManager:
    configMapRef:
      name: ops-manager-connection
  credentials: om-api-credentials
  security:
    authentication:
      enabled: true
      modes: ["SCRAM"]
  podSpec:
    persistence:
      single:
        storage: 500G
        storageClass: fast-ssd

There is a chicken-and-egg problem here that catches everyone: the oplog store is managed by Ops Manager, but Ops Manager needs it to enable backup. The resolution is ordering. Bring up Ops Manager with backup.enabled: false, create the oplog store as a managed deployment, then flip backup on. The operator handles the rest.

Provisioning your first deployment

With the control plane running, provisioning a replica set is a project reference, an API key and a manifest.

Wire up the project

Ops Manager groups deployments into projects. Create a ConfigMap describing the target project and a Secret holding programmatic API credentials generated in the Ops Manager UI under Organization → Access Manager.

kubectl -n data-prod create configmap orders-project \
  --from-literal="baseUrl=http://ops-manager-svc.mongodb.svc.cluster.local:8080" \
  --from-literal="orgId=<org-id>" \
  --from-literal="projectName=orders-prod"
 
kubectl -n data-prod create secret generic om-api-credentials \
  --from-literal="publicApiKey=<public-key>" \
  --from-literal="privateApiKey=<private-key>"

Give the API key the Project Creator role at organisation scope. If the project named in the ConfigMap does not exist, the operator creates it — which means a typo silently produces an empty new project rather than an error, and you spend twenty minutes wondering why your deployment never appears.

Declare the replica set

apiVersion: mongodb.com/v1
kind: MongoDB
metadata:
  name: orders
  namespace: data-prod
spec:
  members: 3
  version: 8.0.0-ent
  type: ReplicaSet
 
  opsManager:
    configMapRef:
      name: orders-project
  credentials: om-api-credentials
 
  security:
    tls:
      enabled: true
      ca: internal-ca
    authentication:
      enabled: true
      modes: ["SCRAM", "X509"]
      internalCluster: X509
 
  backup:
    mode: enabled
 
  additionalMongodConfig:
    storage:
      wiredTiger:
        engineConfig:
          cacheSizeGB: 8
    operationProfiling:
      mode: slowOp
      slowOpThresholdMs: 100
 
  podSpec:
    persistence:
      multiple:
        data:
          storage: 500G
          storageClass: fast-ssd
        journal:
          storage: 20G
        logs:
          storage: 20G

internalCluster: X509 switches member-to-member authentication from a shared keyfile to certificates. It is more setup and it is worth it — a leaked keyfile is a permanent problem, whereas a certificate expires on its own.

cacheSizeGB deserves a deliberate value in containers. WiredTiger defaults to half of detected RAM, and depending on the runtime and kernel that detection may see the node rather than the cgroup limit. Set it to roughly half the container memory limit and stop guessing.

operationProfiling at a 100 ms threshold costs almost nothing and means that when someone asks why the API got slow last Tuesday, there is an answer in the profiler collection rather than a theory.

Watch it converge

kubectl -n data-prod get mdb orders -w
# NAME     TYPE         STATE       VERSION
# orders   ReplicaSet   Pending     8.0.0-ent
# orders   ReplicaSet   Running     8.0.0-ent

If it stays Pending, the answer is almost always in the operator log rather than the resource status:

kubectl -n mongodb logs -l app.kubernetes.io/name=mongodb-kubernetes-operator -f

Day two is the actual job

Provisioning is the easy half. Three things separate a deployment that survives from one that merely exists.

Upgrades are a field edit

Change spec.version, apply, and the agents roll the replica set one member at a time, stepping down the primary in a controlled fashion. There is no downtime window to schedule and no runbook to follow.

That is exactly why upgrades need a policy. Without one, "it is only a field" becomes a Friday afternoon habit. Pin the version in Git, promote it through environments, and let the same review that governs application code govern the database.

Restores must be rehearsed

A backup you have never restored is a hypothesis. Ops Manager will happily report green for months while a misconfigured snapshot store quietly produces unusable snapshots.

Run this quarterly, as a job, not as an intention:

1. Pick a random production snapshot from the last week.
2. Restore it into an isolated namespace via the Restore flow or POST /groups/{id}/restoreJobs.
3. Run a document count and checksum against a known collection.
4. Record the wall-clock time from request to queryable.

That last number is your real recovery time objective. Everything else is the number you wrote in a document.

Alerts should map to owners

Ops Manager ships a large default alert set, and the default outcome is that all of it routes to one inbox and gets ignored within a month. Prune it to the alerts that imply an action — replication lag over threshold, a member in RECOVERING, oplog window shorter than your snapshot interval, backup job failure — and route each one to the team that can actually fix it.

Where this leaves you

You now have a control plane that turns a 30-node upgrade into a pull request, backups whose restore path has been exercised, and deployments described in Git rather than in someone's memory of a UI session.

The part worth internalising is the separation. Ops Manager's Application Database, the oplog store and the snapshot store are three different failure domains with three different sizing models, and treating them as one is the decision that makes self-managed MongoDB painful. Keep them apart and the rest of it is, genuinely, boring.