java.security.PrivilegedExceptionAction<java.lang.Object>
, Serviceable
, ModuleControl
, ModuleSupportable
, Corruptable
, LogFactory
public final class LogToFile extends java.lang.Object implements LogFactory, ModuleControl, ModuleSupportable, Serviceable, java.security.PrivilegedExceptionAction<java.lang.Object>
The 'log' is a stream of log records. The 'log' is implemented as a series of numbered log files. These numbered log files are logically continuous so a transaction can have log records that span multiple log files. A single log record cannot span more then one log file. The log file number is monotonically increasing.
The log belongs to a log factory of a RawStore. In the current implementation, each RawStore only has one log factory, so each RawStore only has one log (which composed of multiple log files). At any given time, a log factory only writes new log records to one log file, this log file is called the 'current log file'.
A log file is named loglogNumber.dat
Everytime a checkpoint is taken, a new log file is created and all subsequent log records will go to the new log file. After a checkpoint is taken, old and useless log files will be deleted.
RawStore exposes a checkpoint method which clients can call, or a checkpoint is taken automatically by the RawStore when
This LogFactory is responsible for the formats of 2 kinds of file: the log file and the log control file. And it is responsible for the format of the log record wrapper.
Format of log control file
Format ID | FILE_STREAM_LOG_FILE |
Purpose | |
The log control file contains information about which log files are present and where the last checkpoint log record is located. | |
The log file contains log record which record all the changes to the database. The complete transaction log is composed of a series of log files. | |
The log record wrapper provides information for the log scan. | |
Upgrade | |
Disk Layout | int format id int obsolete log file version long the log instant (LogCounter) of the last completed checkpoint (logfile counter, position) int Derby major version int Derby minor version int subversion revision/build number byte Flags (beta flag (0 or 1), test durability flag (0 or 1)) byte spare (0) byte spare (0) byte spare (0) long spare (value set to 0) long checksum for control data written |
Modifier and Type | Field | Description |
---|---|---|
private int |
action |
|
private StorageFile |
activeFile |
|
private java.lang.String |
activePerms |
|
private long |
allowedToReadFileNumber |
|
private boolean |
backupInProgress |
|
(package private) long |
bootTimeLogFileNumber |
|
private static int |
CHECKPOINT_INTERVAL_MAX |
|
private static int |
CHECKPOINT_INTERVAL_MIN |
|
private DaemonService |
checkpointDaemon |
|
private boolean |
checkpointDaemonCalled |
|
(package private) long |
checkpointInstant |
|
private int |
checkpointInterval |
|
private java.util.zip.CRC32 |
checksum |
|
protected StandardException |
corrupt |
If not null then something is corrupt in the raw store and this represents the original error.
|
private CheckpointOperation |
currentCheckpoint |
|
private boolean |
databaseEncrypted |
|
(package private) java.lang.String |
dataDirectory |
|
protected DataFactory |
dataFactory |
|
static java.lang.String |
DBG_FLAG |
|
private static int |
DEFAULT_CHECKPOINT_INTERVAL |
|
private static java.lang.String |
DEFAULT_LOG_ARCHIVE_DIRECTORY |
|
private static int |
DEFAULT_LOG_BUFFER_SIZE |
|
private static int |
DEFAULT_LOG_SWITCH_INTERVAL |
|
static java.lang.String |
DUMP_LOG_FROM_LOG_FILE |
|
static java.lang.String |
DUMP_LOG_ONLY |
|
protected long |
endPosition |
|
private static int |
fid |
|
private StorageRandomAccessFile |
firstLog |
|
(package private) long |
firstLogFileNumber |
|
private boolean |
inCheckpoint |
|
private boolean |
inLogSwitch |
|
private boolean |
inRedo |
|
private boolean |
inReplicationMasterMode |
|
private boolean |
inReplicationSlaveMode |
|
private boolean |
inReplicationSlavePreMode |
True if the database has been booted in replication slave pre
mode, effectively turning off writes to the log file.
|
private static long |
INT_LENGTH |
|
private static byte |
IS_BETA_FLAG |
|
private static byte |
IS_DURABILITY_TESTMODE_NO_SYNC_FLAG |
When the derby.system.durability property is set to 'test', the store
system will not force sync calls in the following cases
- for the log file at each commit
- for the log file before data page is forced to disk
- for page allocation when file is grown
- for data writes during checkpoint
This means it is possible that the recovery system may not work properly,
committed transactions may be lost, and/or database may not
be in a consistent state.
|
private boolean |
isFrozen |
If frozen, don't allow anything on disk to change.
|
private boolean |
isWriteSynced |
Note: Why logging system support file sync and write sync ?
|
(package private) ProductVersionHolder |
jbmsVersion |
Product Version information.
|
private boolean |
jvmSyncErrorChecked |
Status for whether the check on the sync error on some JVMs has been
done or not.
|
private boolean |
keepAllLogs |
|
(package private) long |
lastFlush |
|
private static int |
LOG_BUFFER_SIZE_MAX |
|
private static int |
LOG_BUFFER_SIZE_MIN |
|
protected static int |
LOG_FILE_HEADER_PREVIOUS_LOG_INSTANT_OFFSET |
|
static int |
LOG_FILE_HEADER_SIZE |
|
static int |
LOG_RECORD_OVERHEAD |
|
private static int |
LOG_SWITCH_INTERVAL_MAX |
|
private static int |
LOG_SWITCH_INTERVAL_MIN |
|
protected static java.lang.String |
LOG_SYNC_STATISTICS |
|
private boolean |
logArchived |
|
private boolean |
logBeingFlushed |
|
private int |
logBufferSize |
|
(package private) java.lang.String |
logDevice |
|
(package private) long |
logFileNumber |
|
private long |
logFileToBackup |
|
private boolean |
logNotSynced |
|
protected LogAccessFile |
logOut |
|
private WritableStorageFactory |
logStorageFactory |
|
private int |
logSwitchInterval |
|
private boolean |
logSwitchRequired |
|
private long |
logWrittenFromLastCheckPoint |
|
private MasterFactory |
masterFactory |
|
private long |
maxLogFileNumber |
|
private int |
mon_flushCalls |
|
private boolean |
mon_LogSyncStatistics |
|
private int |
mon_numBytesToLog |
|
private int |
mon_numLogFlushWaits |
|
private int |
mon_syncCalls |
|
private int |
myClientNumber |
|
private static int |
OBSOLETE_LOG_VERSION_NUMBER |
|
private boolean |
onDiskBeta |
|
private int |
onDiskMajorVersion |
On disk database version information.
|
private int |
onDiskMinorVersion |
|
private RawStoreFactory |
rawStoreFactory |
|
protected boolean |
ReadOnlyDB |
|
private boolean |
recoveryNeeded |
|
private StandardException |
replicationSlaveException |
If this exception is set while in replication slave mode, the
exception will be thrown by the thread doing recovery will.
|
private java.lang.Object |
slaveRecoveryMonitor |
|
private boolean |
stopped |
|
static java.lang.String |
TEST_LOG_FULL |
Set to true if we want to simulate a log full condition
|
static java.lang.String |
TEST_LOG_INCOMPLETE_LOG_WRITE |
Set to true if we want the upcoming log record to be only partially
written.
|
static java.lang.String |
TEST_LOG_PARTIAL_LOG_WRITE_NUM_BYTES |
Set to the number of bytes we want the next log record to actually write
out, only used when TEST_LOG_INCOMPLETE_LOG_WRITE is on.
|
static java.lang.String |
TEST_LOG_SWITCH_LOG |
Set to true if we want the checkpoint to only switch the log but not
actually do the checkpoint
|
(package private) int |
test_logWritten |
DEBUG test only
|
static java.lang.String |
TEST_MAX_LOGFILE_NUMBER |
Set to true if we want to simulate max possible log file number is
being used.
|
(package private) int |
test_numRecordToFillLog |
|
static java.lang.String |
TEST_RECORD_TO_FILL_LOG |
Set to the number of log record we want to write before the log is
simulated to be full.
|
static java.lang.String |
TEST_SWITCH_LOG_FAIL1 |
Set to true if we want to simulate a log full condition while switching log
|
static java.lang.String |
TEST_SWITCH_LOG_FAIL2 |
|
private java.io.File |
toFile |
|
private static boolean |
wasDBInDurabilityTestModeNoSync |
keeps track of if the database was booted previously at any time with
derby.system.durability=test
|
LOG_DIRECTORY_NAME, MODULE, RT_READONLY, RUNTIME_ATTRIBUTES
DONE, REQUEUE
Constructor | Description |
---|---|
LogToFile() |
MT- not needed for constructor
|
Modifier and Type | Method | Description |
---|---|---|
void |
abortLogBackup() |
|
long |
appendLogRecord(byte[] data,
int offset,
int length,
byte[] optionalData,
int optionalDataOffset,
int optionalDataLength) |
Append length bytes of data to the log prepended by a long log instant
and followed by 4 bytes of length information.
|
private void |
backupLogFiles(java.io.File toDir,
long lastLogFileToBackup) |
|
void |
boot(boolean create,
java.util.Properties startParams) |
Boot up the log factory.
|
boolean |
canSupport(java.util.Properties startParams) |
See if this implementation can support any attributes that are listed in properties.
|
private void |
checkCorrupt() |
|
protected void |
checkForReplication(LogAccessFile log) |
Used by LogAccessFile to check if it should take the
replication master role, and thereby send log records to the
MasterFactory.
|
private boolean |
checkJvmSyncError(StorageFile logFile) |
In Java 1.4.2 and newer rws and rwd modes for RandomAccessFile
are supported.
|
boolean |
checkpoint(RawStoreFactory rsf,
DataFactory df,
TransactionFactory tf,
boolean wait) |
Checkpoint the rawStore.
|
void |
checkpointInRFR(LogInstant cinstant,
long redoLWM,
long undoLWM,
DataFactory df) |
redo a checkpoint during rollforward recovery
|
private boolean |
checkpointWithTran(RawTransaction cptran,
RawStoreFactory rsf,
DataFactory df,
TransactionFactory tf,
boolean wait) |
checkpoint with pre-start transaction
|
(package private) boolean |
checkVersion(int requiredMajorVersion,
int requiredMinorVersion) |
Check to see if a database has been upgraded to the required
level in order to use a store feature.
|
boolean |
checkVersion(int requiredMajorVersion,
int requiredMinorVersion,
java.lang.String feature) |
Check to see if a database has been upgraded to the required
level in order to use a store feature.
|
void |
createDataWarningFile() |
Create readme file in log directory warning users against touching
any files in the directory
|
private void |
createLogDirectory() |
Create the directory where transaction log should go.
|
protected long |
currentInstant() |
Get the current log instant - this is the log instant of the Next log
record to be written out
|
boolean |
databaseEncrypted() |
|
int |
decrypt(byte[] ciphertext,
int offset,
int length,
byte[] cleartext,
int outputOffset) |
|
void |
deleteLogFileAfterCheckpointLogFile() |
|
private void |
deleteObsoleteLogfiles() |
|
void |
deleteOnlineArchivedLogFiles() |
|
void |
disableLogArchiveMode() |
|
void |
enableLogArchiveMode() |
|
int |
encrypt(byte[] cleartext,
int offset,
int length,
byte[] ciphertext,
int outputOffset) |
|
void |
endLogBackup(java.io.File toDir) |
|
protected long |
endPosition() |
|
void |
failoverSlave() |
Used to make the slave stop appending log records, complete recovery
and boot the database.
|
private CheckpointOperation |
findCheckpoint(long checkpointInstant,
FileLogger logger) |
|
private static java.lang.Object |
findServiceModule(java.lang.Object serviceModule,
java.lang.String factoryInterface) |
Privileged startup.
|
private long |
firstLogInstant() |
Get the first valid log instant - this is the beginning of the first
log file
|
protected void |
flush(long fileNumber,
long wherePosition) |
Flush the log such that the log record written with the instant
wherePosition is guaranteed to be on disk.
|
void |
flush(LogInstant where) |
Flush all unwritten log record up to the log instance indicated to disk
and sync.
|
void |
flushAll() |
Flush all unwritten log record to disk and sync.
|
private void |
flushBuffer(long fileNumber,
long wherePosition) |
Flush all unwritten log record up to the log instance indicated to disk
without syncing.
|
void |
freezePersistentStore() |
Backup restore - stop sending log record to the log stream
|
java.lang.String |
getCanonicalLogPath() |
Return the canonical directory of the PARENT of the log directory.
|
private static ContextService |
getContextService() |
Privileged lookup of the ContextService.
|
private StorageFile |
getControlFileName() |
Return the control file name
|
int |
getEncryptedDataLength(int length) |
returns the length that will make the data to be multiple of encryption
block size based on the given length.
|
int |
getEncryptionBlockSize() |
return the encryption block size used during encrypted db creation
|
private long |
getFirstLogNeeded(CheckpointOperation checkpoint) |
Return the "oldest" log file still needed by recovery.
|
LogInstant |
getFirstUnflushedInstant() |
Get the instant of the first record which was not
flushed.
|
long |
getFirstUnflushedInstantAsLong() |
Get the log instant long value of the first log record that has not
been flushed.
|
StorageFile |
getLogDirectory() |
Return the location of the log directory.
|
private java.lang.String |
getLogDirPath(StorageFile logDir) |
|
void |
getLogFactoryProperties(PersistentSet set) |
Get JBMS properties relevant to the log factory
|
protected StorageRandomAccessFile |
getLogFileAtBeginning(long filenumber) |
Open a log file and position the file at the beginning.
|
protected StorageRandomAccessFile |
getLogFileAtPosition(long logInstant) |
Get a read-only handle to the log file positioned at the stated position
|
private StorageFile |
getLogFileName(long filenumber) |
Given a log file number, return its file name
|
private long |
getLogFileNumber() |
Return the current log file number.
|
StorageRandomAccessFile |
getLogFileToSimulateCorruption(long filenum) |
Get the log file to Simulate a log corruption
FOR UNIT TESTING USAGE ONLY
|
Logger |
getLogger() |
MT- not needed
|
private void |
getLogStorageFactory() |
|
private StorageFile |
getMirrorControlFileName() |
Return the mirror control file name
|
private static ModuleFactory |
getMonitor() |
Privileged Monitor lookup.
|
private static java.lang.Object |
getServiceModule(java.lang.Object serviceModule,
java.lang.String factoryInterface) |
Privileged module lookup.
|
int |
getTypeFormatId() |
Return my format identifier.
|
void |
initializeReplicationSlaveRole() |
Initializes logOut so that log received from the replication
master can be appended to the log file.
|
private boolean |
initLogFile(StorageRandomAccessFile newlog,
long number,
long prevLogRecordEndInstant) |
Initialize the log to the correct format with the given version and
log file number.
|
boolean |
inReplicationMasterMode() |
Used to determine if the replication master mode has been started,
and the logging for unlogged operations needs to be enabled.
|
boolean |
inRFR() |
|
boolean |
isCheckpointInLastLogFile() |
|
private static boolean |
isFullUpgrade(java.util.Properties startParams,
java.lang.String oldVersionInfo) |
Privileged startup.
|
boolean |
logArchived() |
Backup restore - is the log being archived to some directory?
|
protected void |
logErrMsg(java.lang.String msg) |
Print error message to user about the log
MT - not needed, informational only
|
protected void |
logErrMsg(java.lang.Throwable t) |
Print error message to user about the log
MT - not needed, informational only
|
private void |
logErrMsgForDurabilityTestModeNoSync() |
In case of boot errors, and if database is either booted
with derby.system.durability=test or was previously at any time booted in
this mode, mention in the error message that the error is probably
because the derby.system.durability was set.
|
private long |
logtest_appendPartialLogRecord(byte[] data,
int offset,
int length,
byte[] optionalData,
int optionalDataOffset,
int optionalDataLength) |
Writes out a partial log record - takes the appendLogRecord.
|
StandardException |
markCorrupt(StandardException originalError) |
Once the log factory is marked as corrupt then the raw store will
shut down.
|
protected LogScan |
openBackwardsScan(long startAt,
LogInstant stopAt) |
Scan backward from start position.
|
protected LogScan |
openBackwardsScan(LogInstant stopAt) |
Scan backward from end of log.
|
ScanHandle |
openFlushedScan(DatabaseInstant start,
int groupsIWant) |
Get a ScanHandle to scan flushed records from the log.
|
LogScan |
openForwardsFlushedScan(LogInstant startAt) |
Open a forward scan of the transaction log.
|
protected LogScan |
openForwardsScan(long startAt,
LogInstant stopAt) |
Scan Forward from start position.
|
LogScan |
openForwardsScan(LogInstant startAt,
LogInstant stopAt) |
Get a forwards scan
|
private StorageRandomAccessFile |
openLogFileInWriteMode(StorageFile logFile) |
open the given log file name for writes; if file can not be
be opened in write sync mode then disable the write sync mode and
open the file in "rw" mode.
|
int |
performWork(ContextManager context) |
Do whatever it is that you want the daemon to do for you.
|
private void |
preAllocateNewLogFile(StorageRandomAccessFile log) |
|
private void |
printErrorStack(java.lang.Throwable t) |
print stack trace from the Throwable including
its nested exceptions
|
protected boolean |
privCanWrite(StorageFile file) |
|
private boolean |
privCopyFile(java.io.File from,
StorageFile to) |
|
private boolean |
privCopyFile(StorageFile from,
java.io.File to) |
|
protected boolean |
privDelete(StorageFile file) |
|
protected boolean |
privExists(StorageFile file) |
|
private java.io.OutputStreamWriter |
privGetOutputStreamWriter(StorageFile file) |
|
private java.lang.String[] |
privList(java.io.File file) |
|
private java.lang.String[] |
privList(StorageFile file) |
|
protected boolean |
privMkdirs(StorageFile file) |
|
private StorageRandomAccessFile |
privRandomAccessFile(StorageFile file,
java.lang.String perms) |
|
private boolean |
privRemoveDirectory(StorageFile file) |
|
private long |
readControlFile(StorageFile logControlFileName,
java.util.Properties startParams) |
|
void |
recover(DataFactory df,
TransactionFactory tf) |
Recover the rawStore to a consistent state using the log.
|
private boolean |
restoreLogs(java.util.Properties properties) |
This function restores logs based on the following attributes
are specified on connection URL:
Attribute.CREATE_FROM (Create database from backup if it does not exist)
Attribute.RESTORE_FROM (Delete the whole database if it exists and then
restore it from backup)
Attribute.ROLL_FORWARD_RECOVERY_FROM:(Perform Rollforward Recovery;
except for the log directory everything else is replaced by the copy from
backup. log files in the backup are copied to the existing online log
directory.
|
java.lang.Object |
run() |
|
private boolean |
runBooleanAction(int action,
StorageFile file) |
|
boolean |
serviceASAP() |
If this work should be done as soon as possible, then return true.
|
boolean |
serviceImmediately() |
If this work should be done immediately on the user thread then return true.
|
void |
setDatabaseEncrypted(boolean isEncrypted,
boolean flushLog) |
Sets whether the database is encrypted, all the transaction log has
to be encrypted, and flush the log if requested.
|
private void |
setEndPosition(long newPosition) |
set the endPosition of the log and make sure the new position won't spill off the end of the log
|
void |
setRawStoreFactory(RawStoreFactory rsf) |
Make log factory aware of which raw store factory it belongs to
|
void |
startLogBackup(java.io.File toDir) |
|
void |
startNewLogFile() |
|
void |
startReplicationMasterRole(MasterFactory masterFactory) |
Make this LogFactory pass log records to the MasterFactory
every time a log record is appended to the log on disk, and
notify the MasterFactory when a log disk flush has taken place.
|
void |
stop() |
Stop the log factory
|
void |
stopReplicationMasterRole() |
Stop this LogFactory from passing log records to the
MasterFactory and from notifying the MasterFactory when a log
disk flush has taken place.
|
void |
stopReplicationSlaveRole() |
Stop the slave functionality for this LogFactory.
|
void |
switchLogFile() |
Switch to the next log file if possible.
|
private void |
syncFile(StorageRandomAccessFile raf) |
Utility routine to call sync() on the input file descriptor.
|
protected void |
testLogFull() |
Simulate a log full condition
if TEST_LOG_FULL is set to true, then the property
TEST_RECORD_TO_FILL_LOG indicates the number of times this function is
call before an IOException simulating a log full condition is raised.
|
java.lang.String |
toString() |
|
private void |
truncateLog(long firstLogNeeded) |
Get rid of old and unnecessary log files
|
private void |
truncateLog(CheckpointOperation checkpoint) |
Get rid of old and unnecessary log files
|
void |
unfreezePersistentStore() |
Backup restore - start sending log record to the log stream
|
private boolean |
verifyLogFormat(StorageFile logFileName,
long number) |
Verify that we the log file is of the right format and of the right
version and log file number.
|
private boolean |
verifyLogFormat(StorageRandomAccessFile log,
long number) |
Verify that we the log file is of the right format and of the right
version and log file number.
|
(package private) boolean |
writeControlFile(StorageFile logControlFileName,
long value) |
Carefully write out this value to the control file.
|
private static final long INT_LENGTH
private static int fid
public static final int LOG_FILE_HEADER_SIZE
protected static final int LOG_FILE_HEADER_PREVIOUS_LOG_INSTANT_OFFSET
public static final int LOG_RECORD_OVERHEAD
public static final java.lang.String DBG_FLAG
public static final java.lang.String DUMP_LOG_ONLY
public static final java.lang.String DUMP_LOG_FROM_LOG_FILE
protected static final java.lang.String LOG_SYNC_STATISTICS
private static final int OBSOLETE_LOG_VERSION_NUMBER
private static final int DEFAULT_LOG_SWITCH_INTERVAL
private static final int LOG_SWITCH_INTERVAL_MIN
private static final int LOG_SWITCH_INTERVAL_MAX
private static final int CHECKPOINT_INTERVAL_MIN
private static final int CHECKPOINT_INTERVAL_MAX
private static final int DEFAULT_CHECKPOINT_INTERVAL
private static final int DEFAULT_LOG_BUFFER_SIZE
private static final int LOG_BUFFER_SIZE_MIN
private static final int LOG_BUFFER_SIZE_MAX
private int logBufferSize
private static final byte IS_BETA_FLAG
private static final byte IS_DURABILITY_TESTMODE_NO_SYNC_FLAG
private static boolean wasDBInDurabilityTestModeNoSync
private static final java.lang.String DEFAULT_LOG_ARCHIVE_DIRECTORY
private int logSwitchInterval
private int checkpointInterval
java.lang.String dataDirectory
private WritableStorageFactory logStorageFactory
private boolean logBeingFlushed
protected LogAccessFile logOut
private StorageRandomAccessFile firstLog
protected long endPosition
long lastFlush
long logFileNumber
long bootTimeLogFileNumber
long firstLogFileNumber
private long maxLogFileNumber
private CheckpointOperation currentCheckpoint
long checkpointInstant
private DaemonService checkpointDaemon
private int myClientNumber
private volatile boolean checkpointDaemonCalled
private long logWrittenFromLastCheckPoint
private RawStoreFactory rawStoreFactory
protected DataFactory dataFactory
protected boolean ReadOnlyDB
private MasterFactory masterFactory
private boolean inReplicationMasterMode
private boolean inReplicationSlaveMode
private volatile StandardException replicationSlaveException
private boolean inReplicationSlavePreMode
SlaveFactory
private java.lang.Object slaveRecoveryMonitor
private long allowedToReadFileNumber
private boolean keepAllLogs
private boolean databaseEncrypted
private boolean recoveryNeeded
private boolean inCheckpoint
private boolean inRedo
private boolean inLogSwitch
private boolean stopped
java.lang.String logDevice
private boolean logNotSynced
private volatile boolean logArchived
private boolean logSwitchRequired
int test_logWritten
int test_numRecordToFillLog
private int mon_flushCalls
private int mon_syncCalls
private int mon_numLogFlushWaits
private boolean mon_LogSyncStatistics
private int mon_numBytesToLog
protected volatile StandardException corrupt
private boolean isFrozen
ProductVersionHolder jbmsVersion
private int onDiskMajorVersion
private int onDiskMinorVersion
private boolean onDiskBeta
private java.util.zip.CRC32 checksum
private boolean isWriteSynced
private boolean jvmSyncErrorChecked
private volatile long logFileToBackup
private volatile boolean backupInProgress
public static final java.lang.String TEST_LOG_SWITCH_LOG
public static final java.lang.String TEST_LOG_INCOMPLETE_LOG_WRITE
public static final java.lang.String TEST_LOG_PARTIAL_LOG_WRITE_NUM_BYTES
public static final java.lang.String TEST_LOG_FULL
public static final java.lang.String TEST_SWITCH_LOG_FAIL1
public static final java.lang.String TEST_SWITCH_LOG_FAIL2
public static final java.lang.String TEST_RECORD_TO_FILL_LOG
public static final java.lang.String TEST_MAX_LOGFILE_NUMBER
private int action
private StorageFile activeFile
private java.io.File toFile
private java.lang.String activePerms
public int getTypeFormatId()
public StandardException markCorrupt(StandardException originalError)
markCorrupt
in interface Corruptable
private void checkCorrupt() throws StandardException
StandardException
public Logger getLogger()
getLogger
in interface LogFactory
public void setRawStoreFactory(RawStoreFactory rsf)
setRawStoreFactory
in interface LogFactory
public void recover(DataFactory df, TransactionFactory tf) throws StandardException
In this implementation, the log is a stream of log records stored in
one or more flat files. Recovery is done in 2 passes: redo and undo.
Redo pass
In the redo pass, reconstruct the state of the rawstore by
repeating exactly what happened before as recorded in the log.
Undo pass
In the undo pass, all incomplete transactions are rolled back in
the order from the most recently started to the oldest.
MT - synchronization provided by caller - RawStore boot. This method is guaranteed to be the only method being called and can assume single thread access on all fields.
recover
in interface LogFactory
df
- - the data factorytf
- - the transaction factoryStandardException
- Standard Derby error policyLoggable.needsRedo(org.apache.derby.iapi.store.raw.Transaction)
,
FileLogger.redo(org.apache.derby.iapi.store.raw.xact.RawTransaction, org.apache.derby.iapi.store.raw.xact.TransactionFactory, org.apache.derby.impl.store.raw.log.StreamLogScan, long, long)
public boolean checkpoint(RawStoreFactory rsf, DataFactory df, TransactionFactory tf, boolean wait) throws StandardException
MT- Only one checkpoint is to be taking place at any given time.
The steps of a checkpoint are
freeze the log (for the transition to a new log file) flush current log file create and flush the new log file (with file number 1 higher than the previous log file). The new log file becomes the current log file. unfreeze the log
The earliest useful log record is determined by the repPoint and the undoLWM, whichever is earlier.
Every log file whose log file number is smaller than the earliest useful log record's log file number can be deleted.
Transactions can be at the following states w/r to a checkpoint - consider the log as a continous stream and not as series of log files for the sake of clarity. |(BT)-------(ET)| marks the begin and end of a transaction. . checkpoint started . |__undoLWM | . V |___redoLWM . |___TruncationLWM . | . V 1 |-----------------| 2 |--------------------------------| 3 |-------| 4 |--------------------------------------(end of log) 5 |-^-| . Checkpoint Log Record ---A--->|<-------B--------->|<-------------C-----------
There are only 3 periods of interest :
A) before undoLWM, B) between undo and redo LWM, C) after redoLWM.
Transaction 1 started in A and terminates in B.
During redo, we should only see log records and endXact from this
transaction in the first phase (between undoLWM and redoLWM). No
beginXact log record for this transaction will be seen.
Transaction 2 started in B (right on the undoLWM) and terminated in C.
Any transaction that terminates in C must have a beginXact at or
after undoLWM. In other words, no transaction can span A, B and C.
During redo, we will see beginXact, other log records and endXact
for this transaction.
Transaction 3 started in B and ended in B.
During redo, we will see beginXact, other log records and endXact
for this transaction.
Transaction 4 begins in B and never ends.
During redo, we will see beginXact, other log records.
In undo, this loser transaction will be rolled back.
Transaction 5 is the transaction taking the checkpoint.
The checkpoint action started way back in time but the checkpoint
log record is only written after the buffer cache has been flushed.
Note that if any time elapse between taking the undoLWM and the redoLWM, then it will create a 4th period of interest.
checkpoint
in interface LogFactory
rsf
- The RawStoreFactory to use to do the checkpoint.df
- The DataFactory to use to do the checkpoint.tf
- The TransactionFactory to use to do the checkpoint.wait
- If an existing checkpoint is in progress, then if
wait=true then this routine will wait for the
checkpoint to complete and the do another checkpoint
and wait for it to finish before returning.StandardException
- - encounter exception while doing
checkpoint.private boolean checkpointWithTran(RawTransaction cptran, RawStoreFactory rsf, DataFactory df, TransactionFactory tf, boolean wait) throws StandardException
rsf
- The RawStoreFactory to use to do the checkpoint.df
- The DataFactory to use to do the checkpoint.tf
- The TransactionFactory to use to do the checkpoint.wait
- If an existing checkpoint is in progress, then if
wait=true then this routine will wait for the
checkpoint to complete and the do another checkpoint
and wait for it to finish before returning.StandardException
- Derby Standard Error Policypublic void flush(LogInstant where) throws StandardException
MT - not needed, wrapper method
flush
in interface LogFactory
where
- flush log up to hereStandardException
- Standard Derby error policypublic void flushAll() throws StandardException
MT - not needed, wrapper method
StandardException
- Standard Derby error policyprivate boolean verifyLogFormat(StorageFile logFileName, long number) throws StandardException
MT - not needed, no global variables used
logFileName
- the name of the log filenumber
- the log file numberStandardException
- Standard Derby error policyprivate boolean verifyLogFormat(StorageRandomAccessFile log, long number) throws StandardException
MT - MT-unsafe, caller must synchronize
log
- the log filenumber
- the log file numberStandardException
- Standard Derby error policyprivate boolean initLogFile(StorageRandomAccessFile newlog, long number, long prevLogRecordEndInstant) throws java.io.IOException, StandardException
MT - synchornization provided by caller
newlog
- the new log file to be initializednumber
- the log file numberprevLogRecordEndInstant
- the end position of the previous log recordjava.io.IOException
- if new log file cannot be accessed or initializedStandardException
public void switchLogFile() throws StandardException
MT - log factory is single threaded thru a log file switch, the log is frozen for the duration of the switch
StandardException
private void flushBuffer(long fileNumber, long wherePosition) throws java.io.IOException, StandardException
MT - not needed, wrapper method
wherePosition
- flush log up to herejava.io.IOException
- Failed to flush to the logStandardException
private void truncateLog(CheckpointOperation checkpoint)
MT- only one truncate log is allowed to be taking place at any given time. Synchronized on this.
private void truncateLog(long firstLogNeeded)
firstLogNeeded
- The log file number of the oldest log file
needed for recovery.private long getFirstLogNeeded(CheckpointOperation checkpoint)
Returns the log file that contains the undoLWM, ie. the oldest log record of all uncommitted transactions in the given checkpoint. If no checkpoint is given then returns -1, indicating all log records may be necessary.
boolean writeControlFile(StorageFile logControlFileName, long value) throws java.io.IOException, StandardException
MT- synchronized by caller
java.io.IOException
StandardException
private long readControlFile(StorageFile logControlFileName, java.util.Properties startParams) throws java.io.IOException, StandardException
java.io.IOException
StandardException
private void createLogDirectory() throws StandardException
StandardException
- Standard Error Policypublic void createDataWarningFile() throws StandardException
createDataWarningFile
in interface LogFactory
StandardException
public StorageFile getLogDirectory() throws StandardException
LogFactory
getLogDirectory
in interface LogFactory
StandardException
- Standard Derby Error Policypublic java.lang.String getCanonicalLogPath()
LogFactory
getCanonicalLogPath
in interface LogFactory
private StorageFile getControlFileName() throws StandardException
MT- read only
StandardException
private StorageFile getMirrorControlFileName() throws StandardException
MT- read only
StandardException
private StorageFile getLogFileName(long filenumber) throws StandardException
MT- read only
StandardException
private CheckpointOperation findCheckpoint(long checkpointInstant, FileLogger logger) throws java.io.IOException, StandardException, java.lang.ClassNotFoundException
java.io.IOException
StandardException
java.lang.ClassNotFoundException
protected LogScan openBackwardsScan(long startAt, LogInstant stopAt) throws java.io.IOException, StandardException
MT- read only
java.io.IOException
- cannot access the logStandardException
- Standard Derby error policyprotected LogScan openBackwardsScan(LogInstant stopAt) throws java.io.IOException, StandardException
MT- read only
java.io.IOException
- cannot access the logStandardException
- Standard Derby error policypublic ScanHandle openFlushedScan(DatabaseInstant start, int groupsIWant) throws StandardException
LogFactory
MT- read only
openFlushedScan
in interface LogFactory
start
- - the LogInstant where we start our scan. null means
start at the beginning of the log. This function raises an error
if startAt is a LogInstant which is not in the log.groupsIWant
- - log record groups the scanner wants.StandardException
- Ooops.LogFactory.openFlushedScan(org.apache.derby.iapi.store.access.DatabaseInstant, int)
protected LogScan openForwardsScan(long startAt, LogInstant stopAt) throws java.io.IOException, StandardException
MT- read only
startAt
- - if startAt == INVALID_LOG_INSTANT,
start from the beginning of the log. Otherwise, start scan from startAt.stopAt
- - if not null, stop at this log instant (inclusive).
Otherwise, stop at the end of the logjava.io.IOException
- cannot access the logStandardException
- Standard Derby error policyprotected StorageRandomAccessFile getLogFileAtBeginning(long filenumber) throws java.io.IOException, StandardException
MT- read only
When the database is in slave replication mode only: Assumes that only recover() will call this method after initializeReplicationSlaveRole() has been called, and until slave replication has ended. If this changes, the current implementation will fail.
StandardException
- Standard Derby error policyjava.io.IOException
- cannot access the log at the new position.protected StorageRandomAccessFile getLogFileAtPosition(long logInstant) throws java.io.IOException, StandardException
MT- read only
java.io.IOException
- cannot access the log at the new position.StandardException
- Standard Derby error policypublic boolean canSupport(java.util.Properties startParams)
ModuleSupportable
The module can check for attributes in the properties to
see if it can fulfill the required behaviour. E.g. the raw
store may define an attribute called RawStore.Recoverable.
If a temporary raw store is required the property RawStore.recoverable=false
would be added to the properties before calling bootServiceModule. If a
raw store cannot support this attribute its canSupport method would
return null. Also see the Monitor class's prologue to see how the
identifier is used in looking up properties.
Actually a better way maybe to have properties of the form
RawStore.Attributes.mandatory=recoverable,smallfootprint and
RawStore.Attributes.requested=oltp,fast
canSupport
in interface ModuleSupportable
public void boot(boolean create, java.util.Properties startParams) throws StandardException
MT- caller provide synchronization
boot
in interface ModuleControl
StandardException
- log factory cannot start upMonitor
,
ModuleFactory
private void getLogStorageFactory() throws StandardException
StandardException
public void stop()
MT- caller provide synchronization (RESOLVE: this should be called AFTER dataFactory and transFactory are stopped)
stop
in interface ModuleControl
Monitor
,
ModuleFactory
private void deleteObsoleteLogfiles()
public boolean serviceASAP()
Serviceable
MT - MT safe
serviceASAP
in interface Serviceable
public boolean serviceImmediately()
Serviceable
serviceImmediately
in interface Serviceable
public void getLogFactoryProperties(PersistentSet set) throws StandardException
LogFactory
getLogFactoryProperties
in interface LogFactory
StandardException
- Standard Derby Error Policypublic int performWork(ContextManager context)
Serviceable
performWork
in interface Serviceable
context
- the contextManager set up by the DaemonService. There
may or may not be the necessary context on it, depending on which other
Serviceable object it has done work for.public long appendLogRecord(byte[] data, int offset, int length, byte[] optionalData, int optionalDataOffset, int optionalDataLength) throws StandardException
This method is synchronized to ensure log records are added sequentially to the end of the log.
MT- single threaded through this log factory. Log records are appended one at a time.
StandardException
- Log Full.protected long currentInstant()
MT - This method is synchronized to ensure that it always points to the end of a log record, not the middle of one.
protected long endPosition()
private long getLogFileNumber()
MT - this method is synchronized so that it is not in the middle of being changed by swithLogFile
private long firstLogInstant()
MT- synchronized on this
protected void flush(long fileNumber, long wherePosition) throws StandardException
MT - only one flush is allowed to be taking place at any given time (RESOLVE: right now it single thread thru the log factory while the log is frozen)
StandardException
- cannot sync log fileprivate void syncFile(StorageRandomAccessFile raf) throws StandardException
StandardException
public LogScan openForwardsFlushedScan(LogInstant startAt) throws StandardException
MT- read only
openForwardsFlushedScan
in interface LogFactory
startAt
- - the LogInstant where we start our scan. null means
start at the beginning of the log. This function raises an error
if startAt is a LogInstant which is not in the log.StandardException
- Standard Derby exception policypublic LogScan openForwardsScan(LogInstant startAt, LogInstant stopAt) throws StandardException
openForwardsScan
in interface LogFactory
startAt
- - the LogInstant where we start our scan. null means
start at the beginning of the log. This function raises an error
if startAt is a LogInstant which is not in the log.stopAt
- - the LogInstant where we stop our scan. null means
stop at the end of the log. This function raises an error
if stopAt is a LogInstant which is not in the log.StandardException
- Standard Derby error policypublic final boolean databaseEncrypted()
public void setDatabaseEncrypted(boolean isEncrypted, boolean flushLog) throws StandardException
Log needs to be flushed first if the cryptographic state of the database changes (for instance re-encryption with a new key).
setDatabaseEncrypted
in interface LogFactory
isEncrypted
- true
if the database is encrypted,
false
if notflushLog
- true
if log needs to be flushed,
false
otherwiseStandardException
public void startNewLogFile() throws StandardException
startNewLogFile
in interface LogFactory
StandardException
public boolean isCheckpointInLastLogFile() throws StandardException
isCheckpointInLastLogFile
in interface LogFactory
StandardException
public void deleteLogFileAfterCheckpointLogFile() throws StandardException
deleteLogFileAfterCheckpointLogFile
in interface LogFactory
StandardException
public int encrypt(byte[] cleartext, int offset, int length, byte[] ciphertext, int outputOffset) throws StandardException
StandardException
- Standard Derby Error PolicyRawStoreFactory.encrypt(byte[], int, int, byte[], int, boolean)
public int decrypt(byte[] ciphertext, int offset, int length, byte[] cleartext, int outputOffset) throws StandardException
StandardException
- Standard Derby Error PolicyRawStoreFactory.decrypt(byte[], int, int, byte[], int)
public int getEncryptionBlockSize()
public int getEncryptedDataLength(int length)
public LogInstant getFirstUnflushedInstant()
This only works after running recovery the first time.
MT - RESOLVE:
getFirstUnflushedInstant
in interface LogFactory
public long getFirstUnflushedInstantAsLong()
LogFactory
getFirstUnflushedInstantAsLong
in interface LogFactory
public void freezePersistentStore() throws StandardException
freezePersistentStore
in interface LogFactory
StandardException
- Standard Derby error policypublic void unfreezePersistentStore() throws StandardException
unfreezePersistentStore
in interface LogFactory
StandardException
- Standard Derby error policypublic boolean logArchived()
logArchived
in interface LogFactory
boolean checkVersion(int requiredMajorVersion, int requiredMinorVersion)
requiredMajorVersion
- required database Engine major versionrequiredMinorVersion
- required database Engine minor versionpublic boolean checkVersion(int requiredMajorVersion, int requiredMinorVersion, java.lang.String feature) throws StandardException
checkVersion
in interface LogFactory
requiredMajorVersion
- required database Engine major versionrequiredMinorVersion
- required database Engine minor versionfeature
- Non-null to throw an exception, null to return the
state of the version match. true
if the database has been upgraded to
the required level, false
otherwise.StandardException
- if the database is not at the require version
when feature
feature is
not null
.protected void logErrMsg(java.lang.String msg)
protected void logErrMsg(java.lang.Throwable t)
private void logErrMsgForDurabilityTestModeNoSync()
MT - not needed, informational only
private void printErrorStack(java.lang.Throwable t)
t
- trace starts from this errorprivate long logtest_appendPartialLogRecord(byte[] data, int offset, int length, byte[] optionalData, int optionalDataOffset, int optionalDataLength) throws StandardException
StandardException
protected void testLogFull() throws java.io.IOException
java.io.IOException
public StorageRandomAccessFile getLogFileToSimulateCorruption(long filenum) throws java.io.IOException, StandardException
java.io.IOException
StandardException
public boolean inReplicationMasterMode()
inReplicationMasterMode
in interface LogFactory
public void enableLogArchiveMode() throws StandardException
enableLogArchiveMode
in interface LogFactory
StandardException
public void disableLogArchiveMode() throws StandardException
disableLogArchiveMode
in interface LogFactory
StandardException
public void deleteOnlineArchivedLogFiles()
deleteOnlineArchivedLogFiles
in interface LogFactory
public void startLogBackup(java.io.File toDir) throws StandardException
startLogBackup
in interface LogFactory
StandardException
private void backupLogFiles(java.io.File toDir, long lastLogFileToBackup) throws StandardException
StandardException
public void endLogBackup(java.io.File toDir) throws StandardException
endLogBackup
in interface LogFactory
StandardException
public void abortLogBackup()
abortLogBackup
in interface LogFactory
public boolean inRFR()
inRFR
in interface LogFactory
public void checkpointInRFR(LogInstant cinstant, long redoLWM, long undoLWM, DataFactory df) throws StandardException
checkpointInRFR
in interface LogFactory
cinstant
- The LogInstant of the checkpointredoLWM
- Redo Low Water Mark in the check point recordundoLWM
- Undo Low Water Mark in the checkpointdf
- - the data factoryStandardException
public void startReplicationMasterRole(MasterFactory masterFactory) throws StandardException
startReplicationMasterRole
in interface LogFactory
masterFactory
- The MasterFactory service responsible for
controlling the master side replication behaviour.StandardException
- Standard Derby exception policy,
thrown on replication startup error. Will only be thrown if
replication is attempted started on a readonly database, i.e,
never thrown here.public void stopReplicationMasterRole()
stopReplicationMasterRole
in interface LogFactory
public void stopReplicationSlaveRole() throws StandardException
StandardException
- Standard Derby exception policySlaveDatabase
protected void checkForReplication(LogAccessFile log)
log
- The LogAccessFile that will take the replication
master role iff this database is master.public void initializeReplicationSlaveRole() throws StandardException
StandardException
- Standard Derby error policypublic void failoverSlave()
private boolean restoreLogs(java.util.Properties properties) throws StandardException
StandardException
private void preAllocateNewLogFile(StorageRandomAccessFile log) throws java.io.IOException, StandardException
java.io.IOException
StandardException
private StorageRandomAccessFile openLogFileInWriteMode(StorageFile logFile) throws java.io.IOException
java.io.IOException
private java.lang.String getLogDirPath(StorageFile logDir)
private boolean checkJvmSyncError(StorageFile logFile) throws java.io.IOException
logFile
- information about the log file to be openedStandardException
- Standard Derby exceptionjava.io.IOException
protected boolean privExists(StorageFile file)
protected boolean privDelete(StorageFile file)
private StorageRandomAccessFile privRandomAccessFile(StorageFile file, java.lang.String perms) throws java.io.IOException
java.io.IOException
private java.io.OutputStreamWriter privGetOutputStreamWriter(StorageFile file) throws java.io.IOException
java.io.IOException
protected boolean privCanWrite(StorageFile file)
protected boolean privMkdirs(StorageFile file) throws java.io.IOException
java.io.IOException
private java.lang.String[] privList(java.io.File file)
private java.lang.String[] privList(StorageFile file)
private boolean privCopyFile(StorageFile from, java.io.File to) throws StandardException
StandardException
private boolean privCopyFile(java.io.File from, StorageFile to)
private boolean privRemoveDirectory(StorageFile file)
private boolean runBooleanAction(int action, StorageFile file)
private void setEndPosition(long newPosition)
public final java.lang.Object run() throws java.io.IOException, StandardException
run
in interface java.security.PrivilegedExceptionAction<java.lang.Object>
java.io.IOException
StandardException
public java.lang.String toString()
toString
in class java.lang.Object
private static ContextService getContextService()
private static ModuleFactory getMonitor()
private static java.lang.Object findServiceModule(java.lang.Object serviceModule, java.lang.String factoryInterface) throws StandardException
StandardException
private static java.lang.Object getServiceModule(java.lang.Object serviceModule, java.lang.String factoryInterface)
private static boolean isFullUpgrade(java.util.Properties startParams, java.lang.String oldVersionInfo) throws StandardException
StandardException
Apache Derby V10.14 Internals - Copyright © 2004,2018 The Apache Software Foundation. All Rights Reserved.