public class

DataProcessor

extends Object
java.lang.Object
   ↳ org.rrd4j.data.DataProcessor

Class Overview

Class which should be used for all calculations based on the data fetched from RRD files. This class supports ordinary DEF datasources (defined in RRD files), CDEF datasources (RPN expressions evaluation), SDEF (static datasources - extension of Rrd4j) and PDEF (plottables, see Plottable for more information.

Typical class usage:

 final long t1 = ...
 final long t2 = ...
 DataProcessor dp = new DataProcessor(t1, t2);
 // DEF datasource
 dp.addDatasource("x", "demo.rrd", "some_source", "AVERAGE");
 // DEF datasource
 dp.addDatasource("y", "demo.rrd", "some_other_source", "AVERAGE");
 // CDEF datasource, z = (x + y) / 2
 dp.addDatasource("z", "x,y,+,2,/");
 // ACTION!
 dp.processData();
 // Dump calculated values
 System.out.println(dp.dump());
 

Summary

Constants
int DEFAULT_PIXEL_COUNT Constant representing the default number of pixels on a Rrd4j graph (will be used if no other value is specified with setStep() method.
boolean DEFAULT_POOL_USAGE_POLICY Constant that defines the default RrdDbPool usage policy.
Public Constructors
DataProcessor(long t1, long t2)
Creates new DataProcessor object for the given time span.
DataProcessor(Date d1, Date d2)
Creates new DataProcessor object for the given time span.
DataProcessor(Calendar gc1, Calendar gc2)
Creates new DataProcessor object for the given time span.
Public Methods
void addDatasource(String name, Plottable plottable)

Adds a custom, plottable datasource (PDEF).

void addDatasource(String name, String file, String dsName, ConsolFun consolFunc)

Adds simple datasource (DEF).

void addDatasource(String name, String defName, ConsolFun consolFun)

Adds static source (SDEF).

void addDatasource(String name, String file, String dsName, ConsolFun consolFunc, String backend)

Adds simple source (DEF).

void addDatasource(String name, FetchData fetchData)
Adds DEF datasource with datasource values already available in the FetchData object.
void addDatasource(String name, String rpnExpression)

Adds complex source (CDEF).

String dump()
Dumps timestamps and values of all datasources in a tabelar form.
double get95Percentile(String sourceName)
This method is just an alias for getPercentile(String) method.
double getAggregate(String sourceName, ConsolFun consolFun)
Returns single aggregated value for a single datasource.
Aggregates getAggregates(String sourceName)
Returns all (MIN, MAX, LAST, FIRST, AVERAGE and TOTAL) aggregated values for a single datasource.
long getEndingTimestamp()
Returns ending timestamp.
long getFetchRequestResolution()
Returns desired RRD archive step (resolution) in seconds to be used while fetching data from RRD files.
long getLastRrdArchiveUpdateTime()
Returns time when last RRD archive was updated (all RRD files are considered).
double getPercentile(String sourceName, double percentile)
The same as getPercentile(String) but with a possibility to define custom percentile boundary (different from 95).
double getPercentile(String sourceName)
Used by ISPs which charge for bandwidth utilization on a "95th percentile" basis.
int getPixelCount()
Returns the number of pixels (target graph width).
String[] getSourceNames()
Returns array of datasource names defined in this DataProcessor.
long getStep()
Returns the time step used for data processing.
long[] getTimestamps()
Returns consolidated timestamps created with the processData() method.
long[] getTimestampsPerPixel()
Calculates timestamps which correspond to individual pixels on the graph based on the graph width set with a setPixelCount(int) method call.
long[] getTimestampsPerPixel(int pixelCount)
Calculates timestamps which correspond to individual pixels on the graph.
double[] getValues(String sourceName)
Returns calculated values for a single datasource.
double[][] getValues()
Returns an array of all datasource values for all datasources.
double[] getValuesPerPixel(String sourceName)
Method used to calculate datasource values which should be presented on the graph based on the graph width set with a setPixelCount(int) method call.
double[] getValuesPerPixel(String sourceName, int pixelCount)
Method used to calculate datasource values which should be presented on the graph based on the desired graph width.
boolean isPoolUsed()
Returns boolean value representing RrdDbPool usage policy.
static void main(String[] args)
Cute little demo.
void processData()
Method that should be called once all datasources are defined.
void setFetchRequestResolution(long fetchRequestResolution)
Sets desired RRD archive step in seconds to be used internally while fetching data from RRD files.
void setPixelCount(int pixelCount)
Sets the number of pixels (target graph width).
void setPoolUsed(boolean poolUsed)
Sets the RrdDbPool usage policy.
void setStep(long step)
Roughly corresponds to the --step option in RRDTool's graph/xport commands.
[Expand]
Inherited Methods
From class java.lang.Object

Constants

public static final int DEFAULT_PIXEL_COUNT

Constant representing the default number of pixels on a Rrd4j graph (will be used if no other value is specified with setStep() method.

Constant Value: 600 (0x00000258)

public static final boolean DEFAULT_POOL_USAGE_POLICY

Constant that defines the default RrdDbPool usage policy. Defaults to false (i.e. the pool will not be used to fetch data from RRD files)

Constant Value: false

Public Constructors

public DataProcessor (long t1, long t2)

Creates new DataProcessor object for the given time span. Ending timestamp may be set to zero. In that case, the class will try to find the optimal ending timestamp based on the last update time of RRD files processed with the processData() method.

Parameters
t1 Starting timestamp in seconds without milliseconds
t2 Ending timestamp in seconds without milliseconds

public DataProcessor (Date d1, Date d2)

Creates new DataProcessor object for the given time span. Ending date may be set to null. In that case, the class will try to find optimal ending date based on the last update time of RRD files processed with the processData() method.

Parameters
d1 Starting date
d2 Ending date

public DataProcessor (Calendar gc1, Calendar gc2)

Creates new DataProcessor object for the given time span. Ending date may be set to null. In that case, the class will try to find optimal ending date based on the last update time of RRD files processed with the processData() method.

Parameters
gc1 Starting Calendar date
gc2 Ending Calendar date

Public Methods

public void addDatasource (String name, Plottable plottable)

Adds a custom, plottable datasource (PDEF). The datapoints should be made available by a class extending Plottable class.

Parameters
name source name.
plottable class that extends Plottable class and is suited for graphing.

public void addDatasource (String name, String file, String dsName, ConsolFun consolFunc)

Adds simple datasource (DEF). Simple source name can be used:

  • To specify sources for line, area and stack plots.
  • To define complex sources

Parameters
name source name.
file Path to RRD file.
dsName Datasource name defined in the RRD file.
consolFunc Consolidation function that will be used to extract data from the RRD

public void addDatasource (String name, String defName, ConsolFun consolFun)

Adds static source (SDEF). Static sources are the result of a consolidation function applied to *any* other source that has been defined previously.

Parameters
name source name.
defName Name of the datasource to calculate the value from.
consolFun Consolidation function to use for value calculation

public void addDatasource (String name, String file, String dsName, ConsolFun consolFunc, String backend)

Adds simple source (DEF). Source name can be used:

  • To specify sources for line, area and stack plots.
  • To define complex sources

Parameters
name Source name.
file Path to RRD file.
dsName Data source name defined in the RRD file.
consolFunc Consolidation function that will be used to extract data from the RRD file ("AVERAGE", "MIN", "MAX" or "LAST" - these string constants are conveniently defined in the ConsolFun class).
backend Name of the RrdBackendFactory that should be used for this RrdDb.

public void addDatasource (String name, FetchData fetchData)

Adds DEF datasource with datasource values already available in the FetchData object. This method is used internally by Rrd4j and probably has no purpose outside of it.

Parameters
name Source name.
fetchData Fetched data containing values for the given source name.

public void addDatasource (String name, String rpnExpression)

Adds complex source (CDEF). Complex sources are evaluated using the supplied RPN expression.

Complex source name can be used:

  • To specify sources for line, area and stack plots.
  • To define other complex sources.

Rrd4j supports the following RPN functions, operators and constants: +, -, *, /, %, SIN, COS, LOG, EXP, FLOOR, CEIL, ROUND, POW, ABS, SQRT, RANDOM, LT, LE, GT, GE, EQ, IF, MIN, MAX, LIMIT, DUP, EXC, POP, UN, UNKN, NOW, TIME, PI, E, AND, OR, XOR, PREV, PREV(sourceName), INF, NEGINF, STEP, YEAR, MONTH, DATE, HOUR, MINUTE, SECOND, WEEK, SIGN and RND.

Rrd4j does not force you to specify at least one simple source name as RRDTool.

For more details on RPN see RRDTool's rrdgraph man page.

Parameters
name source name.
rpnExpression RPN expression containig comma (or space) delimited simple and complex source names, RPN constants, functions and operators.

public String dump ()

Dumps timestamps and values of all datasources in a tabelar form. Very useful for debugging.

Returns
  • Dumped object content.

public double get95Percentile (String sourceName)

This method is just an alias for getPercentile(String) method. Used by ISPs which charge for bandwidth utilization on a "95th percentile" basis.

The 95th percentile is the highest source value left when the top 5% of a numerically sorted set of source data is discarded. It is used as a measure of the peak value used when one discounts a fair amount for transitory spikes. This makes it markedly different from the average.

Read more about this topic at Rednet or Bytemark.

Parameters
sourceName Datasource name
Returns
  • 95th percentile of fetched source values

public double getAggregate (String sourceName, ConsolFun consolFun)

Returns single aggregated value for a single datasource.

Parameters
sourceName Datasource name
consolFun Consolidation function to be applied to fetched datasource values. Valid consolidation functions are MIN, MAX, LAST, FIRST, AVERAGE and TOTAL (these string constants are conveniently defined in the ConsolFun class)
Returns
  • MIN, MAX, LAST, FIRST, AVERAGE or TOTAL value calculated from the data for the given datasource name
Throws
IllegalArgumentException Thrown if invalid datasource name is specified, or if datasource values are not yet calculated (method processData() was not called)

public Aggregates getAggregates (String sourceName)

Returns all (MIN, MAX, LAST, FIRST, AVERAGE and TOTAL) aggregated values for a single datasource.

Parameters
sourceName Datasource name
Returns
  • Object containing all aggregated values
Throws
IllegalArgumentException Thrown if invalid datasource name is specified, or if datasource values are not yet calculated (method processData() was not called)

public long getEndingTimestamp ()

Returns ending timestamp. Basically, this value is equal to the ending timestamp specified in the constructor. However, if the ending timestamps was zero, it will be replaced with the real timestamp when the processData() method returns. The real value will be calculated from the last update times of processed RRD files.

Returns
  • Ending timestamp in seconds

public long getFetchRequestResolution ()

Returns desired RRD archive step (resolution) in seconds to be used while fetching data from RRD files. In other words, this value will used as the last parameter of RrdDb.createFetchRequest() method when this method is called internally by this DataProcessor.

Returns
  • Desired archive step (fetch resolution) in seconds.

public long getLastRrdArchiveUpdateTime ()

Returns time when last RRD archive was updated (all RRD files are considered).

Returns
  • Last archive update time for all RRD files in this DataProcessor

public double getPercentile (String sourceName, double percentile)

The same as getPercentile(String) but with a possibility to define custom percentile boundary (different from 95).

Parameters
sourceName Datasource name.
percentile Boundary percentile. Value of 95 (%) is suitable in most cases, but you are free to provide your own percentile boundary between zero and 100.
Returns
  • Requested percentile of fetched source values

public double getPercentile (String sourceName)

Used by ISPs which charge for bandwidth utilization on a "95th percentile" basis.

The 95th percentile is the highest source value left when the top 5% of a numerically sorted set of source data is discarded. It is used as a measure of the peak value used when one discounts a fair amount for transitory spikes. This makes it markedly different from the average.

Read more about this topic at Rednet or Bytemark.

Parameters
sourceName Datasource name
Returns
  • 95th percentile of fetched source values

public int getPixelCount ()

Returns the number of pixels (target graph width). See setPixelCount(int) for more information.

Returns
  • Target graph width

public String[] getSourceNames ()

Returns array of datasource names defined in this DataProcessor.

Returns
  • array of datasource names

public long getStep ()

Returns the time step used for data processing. Initially, this method returns zero. Once processData() is finished, the method will return the real value used for all internal computations. Roughly corresponds to the --step option in RRDTool's graph/xport commands.

Returns
  • Step used for data processing.

public long[] getTimestamps ()

Returns consolidated timestamps created with the processData() method.

Returns
  • array of timestamps in seconds

public long[] getTimestampsPerPixel ()

Calculates timestamps which correspond to individual pixels on the graph based on the graph width set with a setPixelCount(int) method call.

Returns
  • Array of timestamps

public long[] getTimestampsPerPixel (int pixelCount)

Calculates timestamps which correspond to individual pixels on the graph.

Parameters
pixelCount Graph width
Returns
  • Array of timestamps

public double[] getValues (String sourceName)

Returns calculated values for a single datasource. Corresponding timestamps can be obtained from the getTimestamps() method.

Parameters
sourceName Datasource name
Returns
  • an array of datasource values
Throws
IllegalArgumentException Thrown if invalid datasource name is specified, or if datasource values are not yet calculated (method processData() was not called)

public double[][] getValues ()

Returns an array of all datasource values for all datasources. Each row in this two-dimensional array represents an array of calculated values for a single datasource. The order of rows is the same as the order in which datasources were added to this DataProcessor object.

Returns
  • All datasource values for all datasources. The first index is the index of the datasource, the second index is the index of the datasource value. The number of datasource values is equal to the number of timestamps returned with getTimestamps() method.
Throws
IllegalArgumentException Thrown if invalid datasource name is specified, or if datasource values are not yet calculated (method processData() was not called)

public double[] getValuesPerPixel (String sourceName)

Method used to calculate datasource values which should be presented on the graph based on the graph width set with a setPixelCount(int) method call. Each value returned represents a single pixel on the graph. Corresponding timestamp can be found in the array returned from getTimestampsPerPixel() method.

Parameters
sourceName Datasource name
Returns
  • Per-pixel datasource values
Throws
IllegalArgumentException Thrown if datasource values are not yet calculated (method processData() was not called)

public double[] getValuesPerPixel (String sourceName, int pixelCount)

Method used to calculate datasource values which should be presented on the graph based on the desired graph width. Each value returned represents a single pixel on the graph. Corresponding timestamp can be found in the array returned from getTimestampsPerPixel() method.

Parameters
sourceName Datasource name
pixelCount Graph width
Returns
  • Per-pixel datasource values
Throws
IllegalArgumentException Thrown if datasource values are not yet calculated (method processData() was not called)

public boolean isPoolUsed ()

Returns boolean value representing RrdDbPool usage policy.

Returns
  • true, if the pool will be used internally to fetch data from RRD files, false otherwise.

public static void main (String[] args)

Cute little demo. Uses demo.rrd file previously created by basic Rrd4j demo.

Parameters
args Not used
Throws
IOException

public void processData ()

Method that should be called once all datasources are defined. Data will be fetched from RRD files, RPN expressions will be calculated, etc.

Throws
IOException Thrown in case of I/O error (while fetching data from RRD files)

public void setFetchRequestResolution (long fetchRequestResolution)

Sets desired RRD archive step in seconds to be used internally while fetching data from RRD files. In other words, this value will used as the last parameter of RrdDb.createFetchRequest() method when this method is called internally by this DataProcessor. If this method is never called, fetch request resolution defaults to 1 (smallest possible archive step will be chosen automatically).

Parameters
fetchRequestResolution Desired archive step (fetch resoltuion) in seconds.

public void setPixelCount (int pixelCount)

Sets the number of pixels (target graph width). This number is used only to calculate pixel coordinates for Rrd4j graphs (methods getValuesPerPixel(String) and getTimestampsPerPixel()), but has influence neither on datasource values calculated with the processData() method nor on aggregated values returned from getAggregates(String) and similar methods. In other words, aggregated values will not change once you decide to change the dimension of your graph.

The default number of pixels is defined by constant DEFAULT_PIXEL_COUNT and can be changed with a setPixelCount(int) method.

Parameters
pixelCount The number of pixels. If you process RRD data in order to display it on the graph, this should be the width of your graph.

public void setPoolUsed (boolean poolUsed)

Sets the RrdDbPool usage policy.

Parameters
poolUsed true, if the pool should be used to fetch data from RRD files, false otherwise.

public void setStep (long step)

Roughly corresponds to the --step option in RRDTool's graph/xport commands. Here is an explanation borrowed from RRDTool:

"By default rrdgraph calculates the width of one pixel in the time domain and tries to get data at that resolution from the RRD. With this switch you can override this behavior. If you want rrdgraph to get data at 1 hour resolution from the RRD, then you can set the step to 3600 seconds. Note, that a step smaller than 1 pixel will be silently ignored."

I think this option is not that useful, but it's here just for compatibility.

Parameters
step Time step at which data should be fetched from RRD files. If this method is not used, the step will be equal to the smallest RRD step of all processed RRD files. If no RRD file is processed, the step will be roughly equal to the with of one graph pixel (in seconds).