Replies: 19 comments
-
|
this: "Could not find first log file name in binary log index file" generally means maxwell is pointing at a missing binlog. you can try dropping the |
Beta Was this translation helpful? Give feedback.
-
|
closing, feel free to re-open if new information is found |
Beta Was this translation helpful? Give feedback.
-
|
@vmishra is your RDS instance Multi-AZ? I have same problem and investigating deeply. |
Beta Was this translation helpful? Give feedback.
-
|
@cemo was maxwell down for a period of time? "Could not find first log file name in binary log index" just means "the binlog isn't there", which can happen if mysql rotates the binlog away from underneath maxwell. |
Beta Was this translation helpful? Give feedback.
-
|
Yes, it was down for a period. I am almost sure it is related to this for
my case.
Cc @pasali
…On 23 Feb 2018 Fri at 01:51 Ben Osheroff ***@***.***> wrote:
@cemo <https://github.com/cemo> was maxwell down for a period of time?
"Could not find first log file name in binary log index" just means "the
binlog isn't there", which can happen if mysql rotates the binlog away from
underneath maxwell.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#847 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AACsfF0Al2OpBv8MKncKYL006gGzf1hJks5tXe95gaJpZM4RfxQ5>
.
|
Beta Was this translation helpful? Give feedback.
-
|
@osheroff, @cemo and i are in the same team. It does not matter if maxwell is down or not because i think RDS rotates the binlog away cause every time i fix this problem, next day it occurs again. Actually we have another maxwell and rds instance which are working well. So this make think that rds instance is the one which causes the problem. |
Beta Was this translation helpful? Give feedback.
-
|
@pasali hmmm. Is this a low-traffic server? Or is maxwell heavily filtered down to just a few inactive tables? I'm really not sure how this would happen with a running instance, please attach what logs you have... |
Beta Was this translation helpful? Give feedback.
-
|
there is 22 db schemes in the rds instance. do you think that maxwell does not catch up binglog changes ? |
Beta Was this translation helpful? Give feedback.
-
|
@vmishra important to change the binlog retention time for RDS. We have ours set to 24hrs. Monitors and automation are in place for the event maxwell dies, hopefully we can recover within 24hrs and pick up where it stopped. If it goes over 24hrs, you will get the posted errors and pretty sure you have to drop the maxwell schema and start over.
|
Beta Was this translation helpful? Give feedback.
-
|
Hi,all guys: |
Beta Was this translation helpful? Give feedback.
-
|
as I explaining in #884, even with no DML maxwell should receive a rotate-binlog event from the server, which should trigger a heartbeat, which should cause a position bump. Something in that chain may not be working properly, but it's hard to decipher from some of these problems if anyone actually has a consistent reproduction of the issue. @pasali if this problem is still persistent on the one RDS cluster, maybe you could drop maxwell into debug-logging mode on that server? |
Beta Was this translation helpful? Give feedback.
-
|
Hi All, The same issue occurs for me, Maxwell was up and running from last 4 weeks, but there was no activity in the Database for 2 weeks and then we start seeing the below error. com.github.shyiko.mysql.binlog.network.ServerException: Could not find first log file name in binary log index file |
Beta Was this translation helpful? Give feedback.
-
|
@nitingarg1 It's happening because the retention period for binlogs has passed and Maxwell isn't finding the last offset and the heartbeat. You need to drop the Maxwell schema and re-create the database and start again. |
Beta Was this translation helpful? Give feedback.
-
|
Do this can occur when the maxwell was running and listening to any changes
for this database?
Thanks
Nitin
…On Mon, Apr 1, 2019 at 2:04 PM Vikas Kumar ***@***.***> wrote:
@nitingarg1 <https://github.com/nitingarg1> It's happening because the
retention period for binlogs has passed and Maxwell isn't finding the last
offset and the heartbeat. You need to drop the Maxwell schema and re-create
the database and start again.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#847 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AsWomZAzxgT8KNlosQBja3OZBxqUHwm4ks5vcnRcgaJpZM4RfxQ5>
.
|
Beta Was this translation helpful? Give feedback.
-
|
@osheroff It would be great if you could help me out here. How do we handle this issue in Production? Let's say prod database was down for more than 24 hours and/or the maxwell was down for some time and we see this error. Is dropping the maxwell database and restarting going to catch up the lost binlog data? |
Beta Was this translation helpful? Give feedback.
-
|
@sinapali if the binlogs are gone, the data is gone too. there's just no recovering from that. |
Beta Was this translation helpful? Give feedback.
-
Hi is there any method to drop the database |
Beta Was this translation helpful? Give feedback.
-
|
Yes is there anyway to recover without dropping the whole maxwell schema? Only one of our database's have been unable to recover due to the binlog being tossed out. |
Beta Was this translation helpful? Give feedback.
-
|
@1kindjune you may be able to recover with |
Beta Was this translation helpful? Give feedback.
-
I am connecting to RDS Mysql database using Maxwell 1.11 to Kafka 1.11.0.1 3 node cluster
While I'm able to bootstrap the records, But during a normal run I see following warnings in the console logs, They are generated nonstop.
13:15:05,321 WARN ProducerConfig - The configuration 'block.on.buffer.full' was supplied but isn't a known config. 13:15:05,323 INFO AppInfoParser - Kafka version : 0.11.0.1 13:15:05,323 INFO AppInfoParser - Kafka commitId : c2a0d5f9b1f45bf5 13:15:05,340 INFO Maxwell - Maxwell v?? is booting (MaxwellKafkaProducer), starting at Position[BinlogPosition[mysql-bin-changelog.035789:443217], lastHeartbeat=0] 13:15:05,454 INFO MysqlSavedSchema - Restoring schema id 1 (last modified at Position[BinlogPosition[mysql-bin-changelog.035789:422], lastHeartbeat=0]) 13:15:05,608 INFO BinlogConnectorReplicator - Setting initial binlog pos to: mysql-bin-changelog.035789:443217 13:15:05,697 INFO BinaryLogClient - Connected to devdb.cadcssodcqya.ap-south-1.rds.amazonaws.com:3306 at mysql-bin-changelog.035789/443217 (sid:6379, cid:1279388) 13:15:05,698 INFO BinlogConnectorLifecycleListener - Binlog connected. 13:15:05,737 WARN BinlogConnectorLifecycleListener - Communication failure. com.github.shyiko.mysql.binlog.network.ServerException: Could not find first log file name in binary log index file at com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:882) [mysql-binlog-connector-java-0.13.0.jar:0.13.0] at com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:559) [mysql-binlog-connector-java-0.13.0.jar:0.13.0] at com.github.shyiko.mysql.binlog.BinaryLogClient$7.run(BinaryLogClient.java:793) [mysql-binlog-connector-java-0.13.0.jar:0.13.0] at java.lang.Thread.run(Thread.java:745) [?:1.8.0_92] 13:15:05,741 INFO BinlogConnectorLifecycleListener - Binlog disconnected. 13:15:05,798 WARN BinlogConnectorReplicator - replicator stopped at position: mysql-bin-changelog.035789:443217 -- restarting 13:15:05,880 INFO BinaryLogClient - Connected to devdb.cadcssodcqya.ap-south-1.rds.amazonaws.com:3306 at mysql-bin-changelog.035789/443217 (sid:6379, cid:1279394) 13:15:05,880 INFO BinlogConnectorLifecycleListener - Binlog connected. 13:15:05,920 WARN BinlogConnectorLifecycleListener - Communication failure. com.github.shyiko.mysql.binlog.network.ServerException: Could not find first log file name in binary log index file at com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:882) [mysql-binlog-connector-java-0.13.0.jar:0.13.0] at com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:559) [mysql-binlog-connector-java-0.13.0.jar:0.13.0] at com.github.shyiko.mysql.binlog.BinaryLogClient$7.run(BinaryLogClient.java:793) [mysql-binlog-connector-java-0.13.0.jar:0.13.0] at java.lang.Thread.run(Thread.java:745) [?:1.8.0_92] 13:15:05,920 INFO BinlogConnectorLifecycleListener - Binlog disconnected. 13:15:05,981 WARN BinlogConnectorReplicator - replicator stopped at position: mysql-bin-changelog.035789:443217 -- restarting 13:15:06,060 INFO BinaryLogClient - Connected to devdb.cadcssodcqya.ap-south-1.rds.amazonaws.com:3306 at mysql-bin-changelog.035789/443217 (sid:6379, cid:1279395) 13:15:06,060 INFO BinlogConnectorLifecycleListener - Binlog connected. 13:15:06,100 WARN BinlogConnectorLifecycleListener - Communication failure. com.github.shyiko.mysql.binlog.network.ServerException: Could not find first log file name in binary log index file at com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:882) [mysql-binlog-connector-java-0.13.0.jar:0.13.0] at com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:559) [mysql-binlog-connector-java-0.13.0.jar:0.13.0] at com.github.shyiko.mysql.binlog.BinaryLogClient$7.run(BinaryLogClient.java:793) [mysql-binlog-connector-java-0.13.0.jar:0.13.0] at java.lang.Thread.run(Thread.java:745) [?:1.8.0_92] 13:15:06,100 INFO BinlogConnectorLifecycleListener - Binlog disconnected. 13:15:06,161 WARN BinlogConnectorReplicator - replicator stopped at position: mysql-bin-changelog.035789:443217 -- restarting 13:15:06,240 INFO BinaryLogClient - Connected to devdb.cadcssodcqya.ap-south-1.rds.amazonaws.com:3306 at mysql-bin-changelog.035789/443217 (sid:6379, cid:1279396) 13:15:06,240 INFO BinlogConnectorLifecycleListener - Binlog connected. 13:15:06,280 WARN BinlogConnectorLifecycleListener - Communication failure. com.github.shyiko.mysql.binlog.network.ServerException: Could not find first log file name in binary log index file at com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:882) [mysql-binlog-connector-java-0.13.0.jar:0.13.0] at com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:559) [mysql-binlog-connector-java-0.13.0.jar:0.13.0] at com.github.shyiko.mysql.binlog.BinaryLogClient$7.run(BinaryLogClient.java:793) [mysql-binlog-connector-java-0.13.0.jar:0.13.0] at java.lang.Thread.run(Thread.java:745) [?:1.8.0_92] 13:15:06,280 INFO BinlogConnectorLifecycleListener - Binlog disconnected.Beta Was this translation helpful? Give feedback.
All reactions