Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 4 Next »

It may only be down the weekly prod->stage synchronisation but the cdc logs (STAGE) showed the following error:

...
2020-06-16 11:14:39 ERROR Error on bin log position Position[BinlogPosition[mysql-bin.003382:68853804], lastHeartbeat=1592034510892]
2020-06-16 11:14:39 INFO  Binlog disconnected.
2020-06-16 11:14:44 WARN  Timed out waiting for heartbeat 1592306079319
2020-06-16 11:14:44 INFO  Stopping 4 tasks
...

Non-production resolution:

To find (and skip over) a problem statement:

  1. Decode the logfile sudo mysqlbinlog --base64-output=decode-rows /var/log/mysql/{mysql-bin_file_from_error} > /tmp/someTempFile.sql (binlogs are only readable by mysql user & group).

  2. Search for the position by the number in the error. From that point in the file you can find the next position by searching forward for end_log_pos.

  3. Update the binlog_position field in maxwell.positionsto move where the cdc will try to resume. I initially tried the first but this was also a problem. It may be that anything in the same transaction will also be an issue.

Misc thoughts:

It’s not appropriate for production without taking lots of mitigating actions (i.e. what to do about the transactions that haven’t been forwarded).

  • No labels