Example usage for org.apache.cassandra.db.compaction SizeTieredCompactionStrategy createSSTableAndLengthPairs

Introduction

In this page you can find the example usage for org.apache.cassandra.db.compaction SizeTieredCompactionStrategy createSSTableAndLengthPairs.

Prototype

public static List<Pair<SSTableReader, Long>> createSSTableAndLengthPairs(Iterable<SSTableReader> sstables) 

Source Link

Usage

From source file:com.jeffjirsa.cassandra.db.compaction.TimeWindowCompactionStrategy.java

License:Apache License

/**
 * @param buckets list of buckets, sorted from newest to oldest, from which to return the newest bucket within thresholds.
 * @param minThreshold minimum number of sstables in a bucket to qualify.
 * @param maxThreshold maximum number of sstables to compact at once (the returned bucket will be trimmed down to this).
 * @return a bucket (list) of sstables to compact.
 *///from  w w  w  . j  a  v a  2  s. co  m
@VisibleForTesting
static List<SSTableReader> newestBucket(HashMultimap<Long, SSTableReader> buckets, int minThreshold,
        int maxThreshold, TimeUnit sstableWindowUnit, int sstableWindowSize,
        SizeTieredCompactionStrategyOptions stcsOptions, long now) {
    // If the current bucket has at least minThreshold SSTables, choose that one.
    // For any other bucket, at least 2 SSTables is enough.
    // In any case, limit to maxThreshold SSTables.

    TreeSet<Long> allKeys = new TreeSet<>(buckets.keySet());

    Iterator<Long> it = allKeys.descendingIterator();
    while (it.hasNext()) {
        Long key = it.next();
        Set<SSTableReader> bucket = buckets.get(key);
        logger.debug("Key {}, now {}", key, now);
        if (bucket.size() >= minThreshold && key >= now) {
            // If we're in the newest bucket, we'll use STCS to prioritize sstables
            List<Pair<SSTableReader, Long>> pairs = SizeTieredCompactionStrategy
                    .createSSTableAndLengthPairs(bucket);
            List<List<SSTableReader>> stcsBuckets = SizeTieredCompactionStrategy.getBuckets(pairs,
                    stcsOptions.bucketHigh, stcsOptions.bucketLow, stcsOptions.minSSTableSize);
            logger.debug("Using STCS compaction for first window of bucket: data files {} , options {}", pairs,
                    stcsOptions);
            List<SSTableReader> stcsInterestingBucket = SizeTieredCompactionStrategy
                    .mostInterestingBucket(stcsBuckets, minThreshold, maxThreshold);

            // If the tables in the current bucket aren't eligible in the STCS strategy, we'll skip it and look for other buckets
            if (!stcsInterestingBucket.isEmpty())
                return stcsInterestingBucket;
        } else if (bucket.size() >= 2 && key < now) {
            logger.debug("bucket size {} >= 2 and not in current bucket, compacting what's here: {}",
                    bucket.size(), bucket);
            return trimToThreshold(bucket, maxThreshold);
        } else {
            logger.debug("No compaction necessary for bucket size {} , key {}, now {}", bucket.size(), key,
                    now);
        }
    }
    return Collections.<SSTableReader>emptyList();
}

---