Debugging MySQL Memory Alerts: When innodb_buffer_pool_size Isn't the Whole Story

Image
Posted on MySQL Ninjas | August 2026 Every DBA has been there. An alert fires. You SSH into the box, run free -h , and see MySQL consuming far more RAM than you configured. You double-check innodb_buffer_pool_size . It's set correctly. So where is the memory going? This is the story of debugging exactly that — on a Google Cloud c4d-standard-4 instance with 14.7 GB RAM, MySQL configured with a 7168 MB buffer pool, and mysqld RSS sitting at 10.4 GB . That's a 3.2 GB gap nobody could explain. The Setup We run a large MySQL fleet on GCP — over 200 DR replica nodes across three datacenters. As part of a buffer pool tuning project (fitting the right pool size to the right machine class), we set innodb_buffer_pool_size = 7168 MB on our c4d-standard-4 nodes (16 GB RAM, 14.7 GB usable). Shortly after, memory alerts started firing. When I looked at what was actually happening: $ ps aux | grep mysqld | awk '{print $6/1024 " MB"}' 10490 MB ...
Post a Comment

MongoDB InPlace Version Upgrade 3.4x to 3.6x



Today I am putting simple and straight forward steps to in-place upgrade / downgrade mongoDB single as well as replica set cluster.

Note: Steps are for CentOS kindly modify for other OS.
Warning: IMPORTANT Always backup all of your data before upgrading MongoDB and test steps in nonprod env first.

1. Checking compatibility version

db.system.version.find();
2. setting compatiblity version

db.adminCommand( { setFeatureCompatibilityVersion: "3.4" } )
3. change dir

cd /tmp/
4. download pkgs (link)
5. cd home

cd
6. stopping mongo

systemctl stop mongod
7. backing up conf

cp /etc/mongod.conf /tmp/
8. removing current Binary

yum remove mongod*
9. cd download dir

cd /tmp/
10. installing newly downloaded mongo server

yum install mongodb-org-server-3.6.14-1.el7.x86_64.rpm
11. installing newly downloaded mongo shell 

yum install mongodb-org-shell-3.6.14-1.el7.x86_64.rpm
12. copying back mongo conf 

cp mongod.conf /etc/
13. starting mongo 

systemctl start mongod
To upgrade a replica set, upgrade each member individually, starting with the secondaries and finishing with the primary. Plan the upgrade during a predefined maintenance window.

Upgrade Secondaries

Upgrade each secondary separately as follows:
  1. Upgrade the secondary’s mongod binary by following the above instructions.
  2. After upgrading a secondary, wait for the secondary to recover to the SECONDARY state before upgrading the next instance. To check the member’s state, issue rs.status() in the mongo shell.
  3. The secondary may briefly go into STARTUP2 or RECOVERING. This is normal. Make sure to wait for the secondary to fully recover to SECONDARY before you continue the upgrade.

Upgrade the Primary

1. Step down the primary to initiate the normal failover procedure. Using one of the following:
  • The rs.stepDown() helper in the mongo shell.
  • The replSetStepDown database command.
During failover, the set cannot accept writes. Typically this takes 10-20 seconds. Plan the upgrade during a predefined maintenance window.
2. Once the primary has stepped down, call the rs.status() method from the mongo shell until you see that another member has assumed the PRIMARY state.
3. Shut down the original primary and upgrade its instance by following the above instructions.

Ref:

Comments

  1. dbaApril 18, 2022 at 8:42 PM

    This comment has been removed by the author.

    ReplyDelete
  2. dbaApril 18, 2022 at 8:43 PM

    Hi Thanks for the article.

    Could you please answer the below with regards to Windows Environment.

    a)Shut down the mongod instance on Server and replace the 4.0 binary with the 4.2 binary.
    b) Restart the node.

    Replace the binary means, run the new version msi installer which will overwrite old binary or Manually remove 4.0\bin and replace it with 4.2\bin

    Thanks

    ReplyDelete
  3. Cyberz PcMay 9, 2022 at 8:52 PM

    I entre that publish and were given it great and informative.Please proportion extra considering that. Win 7 Ultimate Product Key

    ReplyDelete
  4. Cyberz PcMay 10, 2022 at 8:15 AM

    this is such a satisfying aid that you are supplying and you pay for it away for pardon.I liked your engagement intensely thanks Happy Brothers Day Quotes

    ReplyDelete
  5. zscrackedJuly 8, 2022 at 5:11 PM

    Excellent site blog this is superb its too good.
    Parallels Desktop Crack
    Navicat Premium Crack
    Windows Movies Maker Crack
    Avira Antivirus Crack

    ReplyDelete
  6. Jeffrey M. HunterSeptember 8, 2022 at 2:13 AM

    Well done. Thanks for sharing.

    ReplyDelete

Post a Comment

Popular posts from this blog

Azure VM Application Consistent MySQL DB Disk Snapshots

Image
Backup of Database is the pillar of our system which is necessary and mandatory to provide us data incase of crash, new machine provisioning and many other scenarios listed here .  As part of the backup process, a snapshot is taken, and the data is transferred to the Recovery Services vault with no impact on production workloads. The snapshot provides different levels of consistency, as described below: 1. Application-consistent: App-consistent backups capture memory content and pending I/O operations. App-consistent snapshots use a VSS writer (or pre/post scripts for Linux) to ensure the consistency of the app data before a backup occurs. When you're recovering a VM with an app-consistent snapshot, the VM boots up. There's no data corruption or loss. The apps start in a consistent state. 2. File-system consistent: File-system consistent backups provide consistency by taking a snapshot of all files at the same time. When you're recovering a VM with a file-system cons...
Read more

Debugging MySQL Memory Alerts: When innodb_buffer_pool_size Isn't the Whole Story

Image
Posted on MySQL Ninjas | August 2026 Every DBA has been there. An alert fires. You SSH into the box, run free -h , and see MySQL consuming far more RAM than you configured. You double-check innodb_buffer_pool_size . It's set correctly. So where is the memory going? This is the story of debugging exactly that — on a Google Cloud c4d-standard-4 instance with 14.7 GB RAM, MySQL configured with a 7168 MB buffer pool, and mysqld RSS sitting at 10.4 GB . That's a 3.2 GB gap nobody could explain. The Setup We run a large MySQL fleet on GCP — over 200 DR replica nodes across three datacenters. As part of a buffer pool tuning project (fitting the right pool size to the right machine class), we set innodb_buffer_pool_size = 7168 MB on our c4d-standard-4 nodes (16 GB RAM, 14.7 GB usable). Shortly after, memory alerts started firing. When I looked at what was actually happening: $ ps aux | grep mysqld | awk '{print $6/1024 " MB"}' 10490 MB ...
Read more