org.apache.tephra.janitor

Interface TransactionPruningPlugin

All Known Implementing Classes:

HBaseTransactionPruningPlugin

public interface TransactionPruningPlugin

Data janitor interface to manage the invalid transaction list.

An invalid transaction can only be removed from the invalid list after the data written by the invalid transactions has been removed from all the data stores. The term data store is used here to represent a set of tables in a database that have the same data clean up policy, like all Apache Phoenix tables in an HBase instance.

Typically every data store will have a background job which cleans up the data written by invalid transactions. Prune upper bound for a data store is defined as the largest invalid transaction whose data has been cleaned up from that data store.

 prune-upper-bound = min(max(invalid list), min(in-progress list) - 1)
 

where invalid list and in-progress list are from the transaction snapshot used to clean up the invalid data in the data store.

There will be one such plugin per data store. The plugins will be executed as part of the Transaction Service. Each plugin will be invoked periodically to fetch the prune upper bound for its data store. Invalid transaction list can pruned up to the minimum of prune upper bounds returned by all the plugins.

Method Summary

Methods

Modifier and Type	Method and Description
void	destroy() Called once during the shutdown of the Transaction Service.
long	fetchPruneUpperBound(long time, long inactiveTransactionBound) Called periodically to fetch prune upper bound for a data store.
void	initialize(org.apache.hadoop.conf.Configuration conf) Called once when the Transaction Service starts up.
void	pruneComplete(long time, long maxPrunedInvalid) Called after successfully pruning the invalid list using the prune upper bound returned by fetchPruneUpperBound(long, long).

Method Detail

initialize

void initialize(org.apache.hadoop.conf.Configuration conf)
                throws IOException

Called once when the Transaction Service starts up.

Parameters:

conf - configuration for the plugin

Throws:

IOException

fetchPruneUpperBound

long fetchPruneUpperBound(long time,
                        long inactiveTransactionBound)
                          throws IOException

Called periodically to fetch prune upper bound for a data store. The plugin examines the state of data cleanup in the data store, and determines an upper bound for invalid transactions such that any invalid transaction smaller than or equal to this upper bound is guaranteed to have all its writes removed from the data store. It then returns this upper bound as the prune upper bound for this data store.

Parameters:

time - start time of this prune iteration in milliseconds

inactiveTransactionBound - the largest invalid transaction that can be possibly removed from the invalid list for the given time. This is an upper bound determined by the Transaction Service, based on its knowledge of in-progress and invalid transactions that may still have active processes and therefore future writes. The plugin will typically return a reduced upper bound based on the state of the invalid transaction data clean up in the data store.

Returns:

prune upper bound for the data store

Throws:

IOException

pruneComplete

void pruneComplete(long time,
                 long maxPrunedInvalid)
                   throws IOException

Called after successfully pruning the invalid list using the prune upper bound returned by fetchPruneUpperBound(long, long). The largest invalid transaction that was removed from the invalid list is passed as a parameter in this call. The plugin can use this information to clean up its state.

Parameters:

time - start time of this prune iteration in milliseconds (same value as passed to fetchPruneUpperBound(long, long) in the same run)

maxPrunedInvalid - the largest invalid transaction that was removed from the invalid list

Throws:

IOException

destroy

void destroy()

Called once during the shutdown of the Transaction Service.