org.apache.tephra.persist

Class HDFSTransactionStateStorage

java.lang.Object

com.google.common.util.concurrent.AbstractIdleService

org.apache.tephra.persist.AbstractTransactionStateStorage

org.apache.tephra.persist.HDFSTransactionStateStorage

All Implemented Interfaces:

com.google.common.util.concurrent.Service, TransactionStateStorage

public class HDFSTransactionStateStorage
extends AbstractTransactionStateStorage

Handles persistence of transaction snapshot and logs to a directory in HDFS. The directory used for file storage is configured using the data.tx.snapshot.dir configuration property. Both snapshot and transaction log files are suffixed with a timestamp to allow easy ordering. Snapshot files are written with the filename "snapshot.<timestamp>". Transaction log files are written with the filename "txlog.<timestamp>".

Nested Class Summary

Nested classes/interfaces inherited from interface com.google.common.util.concurrent.Service

com.google.common.util.concurrent.Service.Listener, com.google.common.util.concurrent.Service.State

Field Summary

Fields inherited from class org.apache.tephra.persist.AbstractTransactionStateStorage

codecProvider

Constructor Summary

Constructors

Constructor and Description
HDFSTransactionStateStorage(org.apache.hadoop.conf.Configuration hConf, SnapshotCodecProvider codecProvider, MetricsCollector metricsCollector)

Method Summary

Methods

Modifier and Type	Method and Description
TransactionLog	createLog(long timestamp) Creates a new TransactionLog.
void	deleteLogsOlderThan(long timestamp) Removes any transaction logs with a timestamp older than the given value.
long	deleteOldSnapshots(int numberToKeep) Removes any snapshots prior to the numberToKeep most recent.
TransactionSnapshot	getLatestSnapshot() Returns the most recent snapshot that has been successfully written.
TransactionVisibilityState	getLatestTransactionVisibilityState() Returns the most recent transaction visibility state that has been successfully written.
String	getLocation() Returns a string representation of the location used for persistence.
List<TransactionLog>	getLogsSince(long timestamp) Returns all TransactionLogs with a timestamp greater than or equal to the given timestamp.
List<String>	listLogs() Returns the (non-qualified) names of available logs.
List<String>	listSnapshots() Returns the (non-qualified) names of available snapshots.
static void	main(String[] args) Reads a transaction state snapshot or transaction log from HDFS and prints the entries to stdout.
void	setupStorage() Create the directories required for the transaction state stage.
protected void	shutDown()
protected void	startUp()
void	writeSnapshot(TransactionSnapshot snapshot) Persists a snapshot of transaction state.

Methods inherited from class org.apache.tephra.persist.AbstractTransactionStateStorage

writeSnapshot

Methods inherited from class com.google.common.util.concurrent.AbstractIdleService

addListener, executor, isRunning, start, startAndWait, state, stop, stopAndWait, toString

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface com.google.common.util.concurrent.Service

addListener, isRunning, start, startAndWait, state, stop, stopAndWait

Constructor Detail

HDFSTransactionStateStorage

@Inject
public HDFSTransactionStateStorage(org.apache.hadoop.conf.Configuration hConf,
                                  SnapshotCodecProvider codecProvider,
                                  MetricsCollector metricsCollector)

Method Detail

startUp

protected void startUp()
                throws Exception

Specified by:

startUp in class com.google.common.util.concurrent.AbstractIdleService

Throws:

Exception

shutDown

protected void shutDown()
                 throws Exception

Specified by:

shutDown in class com.google.common.util.concurrent.AbstractIdleService

Throws:

Exception

writeSnapshot

public void writeSnapshot(TransactionSnapshot snapshot)
                   throws IOException

Description copied from interface: TransactionStateStorage

Persists a snapshot of transaction state.

Throws:

IOException

getLatestSnapshot

public TransactionSnapshot getLatestSnapshot()
                                      throws IOException

Description copied from interface: TransactionStateStorage

Returns the most recent snapshot that has been successfully written. Note that this may return null if no completed snapshot files are found.

Throws:

IOException

getLatestTransactionVisibilityState

public TransactionVisibilityState getLatestTransactionVisibilityState()
                                                               throws IOException

Description copied from interface: TransactionStateStorage

Returns the most recent transaction visibility state that has been successfully written. Note that this may return null if no completed snapshot files are found.

Returns:

TransactionVisibilityState

Throws:

IOException

deleteOldSnapshots

public long deleteOldSnapshots(int numberToKeep)
                        throws IOException

Description copied from interface: TransactionStateStorage

Removes any snapshots prior to the numberToKeep most recent.

Parameters:

numberToKeep - The number of most recent snapshots to keep.

Returns:

The timestamp of the oldest snapshot kept.

Throws:

IOException - If an error occurs while deleting old snapshots.

listSnapshots

public List<String> listSnapshots()
                           throws IOException

Description copied from interface: TransactionStateStorage

Returns the (non-qualified) names of available snapshots.

Throws:

IOException

getLogsSince

public List<TransactionLog> getLogsSince(long timestamp)
                                  throws IOException

Description copied from interface: TransactionStateStorage

Returns all TransactionLogs with a timestamp greater than or equal to the given timestamp. Note that the returned list is guaranteed to be sorted in ascending timestamp order.

Throws:

IOException

createLog

public TransactionLog createLog(long timestamp)
                         throws IOException

Description copied from interface: TransactionStateStorage

Creates a new TransactionLog.

Throws:

IOException

deleteLogsOlderThan

public void deleteLogsOlderThan(long timestamp)
                         throws IOException

Description copied from interface: TransactionStateStorage

Removes any transaction logs with a timestamp older than the given value. Logs must be removed based on timestamp to ensure we can fully recover state based on a given snapshot.

Parameters:

timestamp - The timestamp to delete up to. Logs with a timestamp less than this value will be removed.

Throws:

IOException - If an error occurs while removing logs.

setupStorage

public void setupStorage()
                  throws IOException

Description copied from interface: TransactionStateStorage

Create the directories required for the transaction state stage.

Throws:

IOException - If an error occurred during the creation of required directories for transaction state storage.

listLogs

public List<String> listLogs()
                      throws IOException

Description copied from interface: TransactionStateStorage

Returns the (non-qualified) names of available logs.

Throws:

IOException

getLocation

public String getLocation()

Description copied from interface: TransactionStateStorage

Returns a string representation of the location used for persistence.

main

public static void main(String[] args)

Reads a transaction state snapshot or transaction log from HDFS and prints the entries to stdout. Supports the following options: -s read snapshot state (defaults to the latest) -l read a transaction log [filename] reads the given file

Parameters:

args -