Posts

Showing posts from July, 2019

Debugging MySQL Memory Alerts: When innodb_buffer_pool_size Isn't the Whole Story

Image
Posted on MySQL Ninjas | August 2026 Every DBA has been there. An alert fires. You SSH into the box, run free -h , and see MySQL consuming far more RAM than you configured. You double-check innodb_buffer_pool_size . It's set correctly. So where is the memory going? This is the story of debugging exactly that — on a Google Cloud c4d-standard-4 instance with 14.7 GB RAM, MySQL configured with a 7168 MB buffer pool, and mysqld RSS sitting at 10.4 GB . That's a 3.2 GB gap nobody could explain. The Setup We run a large MySQL fleet on GCP — over 200 DR replica nodes across three datacenters. As part of a buffer pool tuning project (fitting the right pool size to the right machine class), we set innodb_buffer_pool_size = 7168 MB on our c4d-standard-4 nodes (16 GB RAM, 14.7 GB usable). Shortly after, memory alerts started firing. When I looked at what was actually happening: $ ps aux | grep mysqld | awk '{print $6/1024 " MB"}' 10490 MB ...
Post a Comment
Read more

MySQL: How to monitor MySQL Replication Lag in MilliSeconds With PMM and pt-heartbeat

Image
There could be various requirements at application end which need realtime slaves. Let me introduce a solution to a problem we used to face how can we monitor whether our slaves are real time or they are lagging in Milliseconds. Unfortunately there is no built in feature in MySQL to get Replication Lag in MilliSeconds. Perhaps there is a tool provisioned in pt-toolkit named as pt-heartbeat . It generates heartbeat events on master and monitoring system can monitor time difference on slave to calculate lag. How to deploy pt-heartbeat in your environment (Assuming OS as UBUNTU xx.xx): Step 1: Download pt-heartbeat using below command. wget http://percona.com/get/pt-heartbeat -P /usr/bin/ && chmod +x /usr/bin/pt-heartbeat Step 2: It requires a database where it can create a table. Let's create it on master. mysql -uUser -pPassword -e "create database if not exists percona;" Step 3: Create heartbeat table in percona db. pt-heartbeat -D percon...
3 comments
Read more

MySQL: How To Sync Specific Table(s) from Master to Slave

Image
Most of us used to get errors like (Row not found, Duplicate row etc) on slave in Master slave replication and sometimes it is very difficult to find unsynced data and fix it while we know table name(s). There are few recommended tools from percona to check replication integrity and fixed unsync data: 1. pt-table-checksum : performs an online replication consistency check by executing checksum queries on the master, which produces different results on replicas that are inconsistent with the master. 2. pt-table-sync : synchronizes data efficiently between MySQL tables. Recommended tools work well with limited data with less time, But what if table size is more and we don't have time to analyze each row of the table with recommended tools which is time consuming? We can achieve this with very simple step though: Let's assume a simple cluster(MySQL 5.7.XX) with master m1 and slave s1 with GTID based replication replicating a db(db_ankit) with 5 tabl...
1 comment
Read more