DataCleaner

6 Nov 2021

Library "DataCleaner"
Functions for acquiring outlier levels and acquiring a cleaned version of a series.

outlierLevel(src, len, level) Gets the (standard deviation) outlier level for a given series.
  Parameters:
    src: The series to average and add a multiple of the standard deviation to.
    len: The The number of bars to measure.
    level: The positive or negative multiple of the standard deviation to apply to the average. A positive number will be the upper boundary and a negative number will be the lower boundary.
  Returns: The average of the series plus the multiple of the standard deviation.

cleanUsing(src, result, len, maxDeviation) Returns an array representing the result series with (outliers provided by the source) removed.
  Parameters:
    src: The source series to read from.
    result: The result series.
    len: The maximum size of the resultant array.
    maxDeviation: The positive or negative multiple of the standard deviation to apply to the average. A positive number will be the upper boundary and a negative number will be the lower boundary.
  Returns: An array containing the cleaned series.

clean(src, len, maxDeviation) Returns an array representing the source series with outliers removed.
  Parameters:
    src: The source series to read from.
    len: The maximum size of the resultant array.
    maxDeviation: The positive or negative multiple of the standard deviation to apply to the average. A positive number will be the upper boundary and a negative number will be the lower boundary.
  Returns: An array containing the cleaned series.

outlierLevelAdjusted(src, level, len, maxDeviation) Gets the (standard deviation) outlier level for a given series after a single pass of removing any outliers.
  Parameters:
    src: The series to average and add a multiple of the standard deviation to.
    level: The positive or negative multiple of the standard deviation to apply to the average. A positive number will be the upper boundary and a negative number will be the lower boundary.
    len: The The number of bars to measure.
    maxDeviation: The optional standard deviation level to use when cleaning the series. The default is the value of the provided level.
  Returns: The average of the series plus the multiple of the standard deviation.

8 Nov 2021

Nota Keluaran

v2 Added simple utility for cleaning arrays.

Added:
cleanArray(src, maxDeviation) Returns an array representing the source array with outliers removed.
  Parameters:
    src: The source series to read from.
    maxDeviation: The positive or negative multiple of the standard deviation to apply to the average. A positive number will be the upper boundary and a negative number will be the lower boundary.
  Returns: An array containing the cleaned series.

4 Dis 2021

Nota Keluaran

v3 Bugfix: cleanArray should avoid empty arrays.

5 Dis 2021

Nota Keluaran

v4 Exposes simpler "naOutliers" method.

Added:
naOutliers(src, len, maxDeviation) Returns only values that are within the maximum deviation.
  Parameters:
    src: The series to average and add a multiple of the standard deviation to.
    len: The The number of bars to measure.
    maxDeviation: The positive or negative multiple of the standard deviation to apply to the average. A positive number will be the upper boundary and a negative number will be the lower boundary.
  Returns: The average of the series plus the multiple of the standard deviation.

5 Dis 2021

Nota Keluaran

v5 Better doc comments.

15 Dis 2021

Nota Keluaran

v6 Added naOutliersArray for keeping the original array indexes but setting the values to NA when they are outliers.

Added:
naArrayOutliers(src, maxDeviation) Returns an array representing the source array with outliers removed.
  Parameters:
    src: The array to set outliers to N/A.
    maxDeviation: The maximum deviation before considered an outlier.
  Returns: True if there were any outliers; otherwise false.

8 Jan 2022

Nota Keluaran

v7 Added normalize function and improved naOutliers.

Added:
normalize(src, len, maxDeviation, baseline) Returns the source value adjusted by its standard deviation.
  Parameters:
    src: The series to measure.
    len: The number of bars to measure the standard deviation.
    maxDeviation: The maximum deviation before considered an outlier.
    baseline: The value considered to be at center. Typically zero.

17 Jan 2023

Nota Keluaran

v8 Fix normalize funciton.

18 Mac 2023

Nota Keluaran

v9 Major Update:

• Allow for a standard deviation function that returns both the mean and the standard deviation and avoid double calculation.
  
• Expose the option to use WMA instead of SMA when averaging.
  
• Expose the option to use SMA as a smoothing function.
  

Added:
stdev(src)
  Calculates and returns both the mean and the standard deviation.
  Parameters:
    src: The array to use for the calculation.

Deviation
  Contains the mean (average) and the value of the standard deviation.
  Fields:
    mean: The mean (average).
    stdev

Updated:
outlierLevel(src, len, level, useWma, smoothing)
  Gets the (standard deviation) outlier level for a given series.
  Parameters:
    src: The series to average and add a multiple of the standard deviation to.
    len: The The number of bars to measure.
    level: The positive or negative multiple of the standard deviation to apply to the average. A positive number will be the upper boundary and a negative number will be the lower boundary.
    useWma
    smoothing
  Returns: The average of the series plus the multiple of the standard deviation.

naOutliers(src, len, maxDeviation, useWma, smoothing)
  Returns only values that are within the maximum deviation.
  Parameters:
    src: The series to filter results from.
    len: The The number of bars to measure.
    maxDeviation: The maximum deviation before considered an outlier.
    useWma: When true, the calulcation is done using an improved weighted moving average to avoid giving significance to older values.
    smoothing

normalize(src, len, maxDeviation, baseline, useWma, smoothing)
  Returns the source value adjusted by its standard deviation.
  Parameters:
    src: The series to measure.
    len: The number of bars to measure the standard deviation.
    maxDeviation: The maximum deviation before considered an outlier.
    baseline: The value considered to be at center. Typically zero.
    useWma
    smoothing

outlierLevelAdjusted(src, len, level, maxDeviation, useWma, smoothing)
  Gets the (standard deviation) outlier level for a given series after a single pass of removing any outliers.
  Parameters:
    src: The series to average and add a multiple of the standard deviation to.
    len: The The number of bars to measure.
    level: The positive or negative multiple of the standard deviation to apply to the average. A positive number will be the upper boundary and a negative number will be the lower boundary.
    maxDeviation: The optional standard deviation level to use when cleaning the series. The default is the value of the provided level.
    useWma
    smoothing
  Returns: The average of the series plus the multiple of the standard deviation.

18 Mac 2023

Nota Keluaran

v10 Allow for custom input if used as an indicator.

18 Mac 2023

Nota Keluaran

v11 Documentation update

Updated:
outlierLevel(src, len, level, useWma, smoothing)
  Gets the (standard deviation) outlier level for a given series.
  Parameters:
    src: The series to average and add a multiple of the standard deviation to.
    len: The The number of bars to measure.
    level: The positive or negative multiple of the standard deviation to apply to the average. A positive number will be the upper boundary and a negative number will be the lower boundary.
    useWma: When true, the calulcation is done using an improved weighted moving average to avoid giving significance to older values.
    smoothing: The number of extra bars to help in smoothing out the result so that large spikes dont' occur from recent data.
  Returns: The average of the series plus the multiple of the standard deviation.

naOutliers(src, len, maxDeviation, useWma, smoothing)
  Returns only values that are within the maximum deviation.
  Parameters:
    src: The series to filter results from.
    len: The The number of bars to measure.
    maxDeviation: The maximum deviation before considered an outlier.
    useWma: When true, the calulcation is done using an improved weighted moving average to avoid giving significance to older values.
    smoothing: The number of extra bars to help in smoothing out the result so that large spikes dont' occur from recent data.

normalize(src, len, maxDeviation, baseline, useWma, smoothing)
  Returns the source value adjusted by its standard deviation.
  Parameters:
    src: The series to measure.
    len: The number of bars to measure the standard deviation.
    maxDeviation: The maximum deviation before considered an outlier.
    baseline: The value considered to be at center. Typically zero.
    useWma: When true, the calulcation is done using an improved weighted moving average to avoid giving significance to older values.
    smoothing: The number of extra bars to help in smoothing out the result so that large spikes dont' occur from recent data.

outlierLevelAdjusted(src, len, level, maxDeviation, useWma, smoothing)
  Gets the (standard deviation) outlier level for a given series after a single pass of removing any outliers.
  Parameters:
    src: The series to average and add a multiple of the standard deviation to.
    len: The The number of bars to measure.
    level: The positive or negative multiple of the standard deviation to apply to the average. A positive number will be the upper boundary and a negative number will be the lower boundary.
    maxDeviation: The optional standard deviation level to use when cleaning the series. The default is the value of the provided level.
    useWma: When true, the calulcation is done using an improved weighted moving average to avoid giving significance to older values.
    smoothing: The number of extra bars to help in smoothing out the result so that large spikes dont' occur from recent data.
  Returns: The average of the series plus the multiple of the standard deviation.