23.5.3.2 NDB Cluster Log Events

An event report reported in the event logs has the following format:

datetime [string] severity -- message

For example:

09:19:30 2005-07-24 [NDB] INFO -- Node 4 Start phase 4 completed

This section discusses all reportable events, ordered by category and severity level within each category.

In the event descriptions, GCP and LCP mean “Global Checkpoint” and “Local Checkpoint”, respectively.

CONNECTION Events

These events are associated with connections between Cluster nodes.

Table 23.55 Events associated with connections between cluster nodes

EventPrioritySeverity LevelDescription
Connected8INFOData nodes connected
Disconnected8ALERTData nodes disconnected
CommunicationClosed8INFOSQL node or data node connection closed
CommunicationOpened8INFOSQL node or data node connection open
ConnectedApiVersion8INFOConnection using API version

CHECKPOINT Events

The logging messages shown here are associated with checkpoints.

Table 23.56 Events associated with checkpoints

EventPrioritySeverity LevelDescription
GlobalCheckpointStarted9INFOStart of GCP: REDO log is written to disk
GlobalCheckpointCompleted10INFOGCP finished
LocalCheckpointStarted7INFOStart of LCP: data written to disk
LocalCheckpointCompleted7INFOLCP completed normally
LCPStoppedInCalcKeepGci0ALERTLCP stopped
LCPFragmentCompleted11INFOLCP on a fragment has been completed
UndoLogBlocked7INFOUNDO logging blocked; buffer near overflow
RedoStatus7INFORedo status

STARTUP Events

The following events are generated in response to the startup of a node or of the cluster and of its success or failure. They also provide information relating to the progress of the startup process, including information concerning logging activities.

Table 23.57 Events relating to the startup of a node or cluster

EventPrioritySeverity LevelDescription
NDBStartStarted1INFOData node start phases initiated (all nodes starting)
NDBStartCompleted1INFOStart phases completed, all data nodes
STTORRYRecieved15INFOBlocks received after completion of restart
StartPhaseCompleted4INFOData node start phase X completed
CM_REGCONF3INFONode has been successfully included into the cluster; shows the node, managing node, and dynamic ID
CM_REGREF8INFONode has been refused for inclusion in the cluster; cannot be included in cluster due to misconfiguration, inability to establish communication, or other problem
FIND_NEIGHBOURS8INFOShows neighboring data nodes
NDBStopStarted1INFOData node shutdown initiated
NDBStopCompleted1INFOData node shutdown complete
NDBStopForced1ALERTForced shutdown of data node
NDBStopAborted1INFOUnable to shut down data node normally
StartREDOLog4INFONew redo log started; GCI keep X, newest restorable GCI Y
StartLog10INFONew log started; log part X, start MB Y, stop MB Z
UNDORecordsExecuted15INFOUndo records executed
StartReport4INFOReport started
LogFileInitStatus7INFOLog file initialization status
LogFileInitCompStatus7INFOLog file completion status
StartReadLCP10INFOStart read for local checkpoint
ReadLCPComplete10INFORead for local checkpoint completed
RunRedo8INFORunning the redo log
RebuildIndex10INFORebuilding indexes

NODERESTART Events

The following events are generated when restarting a node and relate to the success or failure of the node restart process.

Table 23.58 Events relating to restarting a node

EventPrioritySeverity LevelDescription
NR_CopyDict7INFOCompleted copying of dictionary information
NR_CopyDistr7INFOCompleted copying distribution information
NR_CopyFragsStarted7INFOStarting to copy fragments
NR_CopyFragDone10INFOCompleted copying a fragment
NR_CopyFragsCompleted7INFOCompleted copying all fragments
NodeFailCompleted8ALERTNode failure phase completed
NODE_FAILREP8ALERTReports that a node has failed
ArbitState6INFOReport whether an arbitrator is found or not; there are seven different possible outcomes when seeking an arbitrator, listed here:

  • Management server restarts arbitration thread [state=X]

  • Prepare arbitrator node X [ticket=Y]

  • Receive arbitrator node X [ticket=Y]

  • Started arbitrator node X [ticket=Y]

  • Lost arbitrator node X - process failure [state=Y]

  • Lost arbitrator node X - process exit [state=Y]

  • Lost arbitrator node X <error msg> [state=Y]
ArbitResult2ALERTReport arbitrator results; there are eight different possible results for arbitration attempts, listed here:

  • Arbitration check failed: less than 1/2 nodes left

  • Arbitration check succeeded: node group majority

  • Arbitration check failed: missing node group

  • Network partitioning: arbitration required

  • Arbitration succeeded: affirmative response from node X

  • Arbitration failed: negative response from node X

  • Network partitioning: no arbitrator available

  • Network partitioning: no arbitrator configured
GCP_TakeoverStarted7INFOGCP takeover started
GCP_TakeoverCompleted7INFOGCP takeover complete
LCP_TakeoverStarted7INFOLCP takeover started
LCP_TakeoverCompleted7INFOLCP takeover complete (state = X)
ConnectCheckStarted6INFOConnection check started
ConnectCheckCompleted6INFOConnection check completed
NodeFailRejected6ALERTNode failure phase failed

STATISTICS Events

The following events are of a statistical nature. They provide information such as numbers of transactions and other operations, amount of data sent or received by individual nodes, and memory usage.

Table 23.59 Events of a statistical nature

EventPrioritySeverity LevelDescription
TransReportCounters8INFOReport transaction statistics, including numbers of transactions, commits, reads, simple reads, writes, concurrent operations, attribute information, and aborts
OperationReportCounters8INFONumber of operations
TableCreated7INFOReport number of tables created
JobStatistic9INFOMean internal job scheduling statistics
ThreadConfigLoop9INFONumber of thread configuration loops
SendBytesStatistic9INFOMean number of bytes sent to node X
ReceiveBytesStatistic9INFOMean number of bytes received from node X
MemoryUsage5INFOData and index memory usage (80%, 90%, and 100%)
MTSignalStatistics9INFOMultithreaded signals

SCHEMA Events

These events relate to NDB Cluster schema operations.

Table 23.60 Events relating to NDB Cluster schema operations

EventPrioritySeverity LevelDescription
CreateSchemaObject8INFOSchema objected created
AlterSchemaObject8INFOSchema object updated
DropSchemaObject8INFOSchema object dropped

ERROR Events

These events relate to Cluster errors and warnings. The presence of one or more of these generally indicates that a major malfunction or failure has occurred.

Table 23.61 Events relating to cluster errors and warnings

EventPrioritySeverity LevelDescription
TransporterError2ERRORTransporter error
TransporterWarning8WARNINGTransporter warning
MissedHeartbeat8WARNINGNode X missed heartbeat number Y
DeadDueToHeartbeat8ALERTNode X declared “dead” due to missed heartbeat
WarningEvent2WARNINGGeneral warning event
SubscriptionStatus4WARNINGChange in subscription status

INFO Events

These events provide general information about the state of the cluster and activities associated with Cluster maintenance, such as logging and heartbeat transmission.

Table 23.62 Information events

EventPrioritySeverity LevelDescription
SentHeartbeat12INFOSent heartbeat
CreateLogBytes11INFOCreate log: Log part, log file, size in MB
InfoEvent2INFOGeneral informational event
EventBufferStatus7INFOEvent buffer status
EventBufferStatus27INFOImproved event buffer status information

Note注意

SentHeartbeat events are available only if NDB Cluster was compiled with VM_TRACE enabled.

SINGLEUSER Events

These events are associated with entering and exiting single user mode.

Table 23.63 Events relating to single user mode

EventPrioritySeverity LevelDescription
SingleUser7INFOEntering or exiting single user mode

BACKUP Events

These events provide information about backups being created or restored.

Table 23.64 Backup events

EventPrioritySeverity LevelDescription
BackupStarted7INFOBackup started
BackupStatus7INFOBackup status
BackupCompleted7INFOBackup completed
BackupFailedToStart7ALERTBackup failed to start
BackupAborted7ALERTBackup aborted by user
RestoreStarted7INFOStarted restoring from backup
RestoreMetaData7INFORestoring metadata
RestoreData7INFORestoring data
RestoreLog7INFORestoring log files
RestoreCompleted7INFOCompleted restoring from backup
SavedEvent7INFOEvent saved