Moment-Based Adaptive DetectionMBAD (Moment-Based Adaptive Detection) : a method applicable to a wide range of purposes, like outlier or novelty detection, that requires building a sensible interval/set of thresholds. Unlike other methods that are static and rely on optimizations that inevitably lead to underfitting/overfitting, it dynamically adapts to your data distribution without any optimizations, MLE, or stuff, and provides a set of data-driven adaptive thresholds, based on closed-form solution with O(n) algo complexity.
1.5 years ago, when I was still living in Versailles at my friend's house not knowing what was gonna happen in my life tomorrow, I made a damn right decision not to give up on one idea and to actually R&D it and see what’s up. It allowed me to create this one.
The Method Explained
I’ve been wandering about z-values, why exactly 6 sigmas, why 95%? Who decided that? Why would you supersede your opinion on data? Based on what? Your ego?
Then I consciously noticed a couple of things:
1) In control theory & anomaly detection, the popular threshold is 3 sigmas (yet nobody can firmly say why xD). If your data is Laplace, 3 sigmas is not enough; you’re gonna catch too many values, so it needs a higher sigma.
2) Yet strangely, the normal distribution has kurtosis of 3, and 6 for Laplace.
3) Kurtosis is a standardized moment, a moment scaled by stdev, so it means "X amount of something measured in stdevs."
4) You generate synthetic data, you check on real data (market data in my case, I am a quant after all), and you see on both that:
lower extension = mean - standard deviation * kurtosis ≈ data minimum
upper extension = mean + standard deviation * kurtosis ≈ data maximum
Why not simply use max/min?
- Lower info gain: We're not using all info available in all data points to estimate max/min; we just pick the current higher and lower values. Lol, it’s the same as dropping exponential smoothing with alpha = 0 on stationary data & calling it a day.
You can’t update the estimates of min and max when new data arrives containing info about the matter. All you can do is just extend min and max horizontally, so you're not using new info arriving inside new data.
- Mixing order and non-order statistics is a bad idea; we're losing integrity and coherence. That's why I don't like the Hurst exponent btw (and yes, I came up with better metrics of my own).
- Max & min are not even true order statistics, unlike a percentile (finding which requires sorting, which requires multiple passes over your data). To find min or max, you just need to do one traversal over your data. Then with or without any weighting, 100th percentile will equal max. So unlike a weighted percentile, you can’t do weighted max. Then while you can always check max and min of a geometric shape, now try to calculate the 56th percentile of a pentagram hehe.
TL;DR max & min are rather topological characteristics of data, just as the difference between starting and ending points. Not much to do with statistics.
Now the second part of the ballet is to work with data asymmetry:
1) Skewness is also scaled by stdev -> so it must represent a shift from the data midrange measured in stdevs -> given asymmetric data, we can include this info in our models. Unlike kurtosis, skewness has a sign, so we add it to both thresholds:
lower extension = mean - standard deviation * kurtosis + standard deviation * skewness
upper extension = mean + standard deviation * kurtosis + standard deviation * skewness
2) Now our method will work with skewed data as well, omg, ain’t it cool?
3) Hold up, but what about 5th and 6th moments (hyperskewness & hyperkurtosis)? They should represent something meaningful as well.
4) Perhaps if extensions represent current estimated extremums, what goes beyond? Limits, beyond which we expect data not to be able to pass given the current underlying process generating the data?
When you extend this logic to higher-order moments, i.e., hyperskewness & hyperkurtosis (5th and 6th moments), they measure asymmetry and shape of distribution tails, not its core as previous moments -> makes no sense to mix 4th and 3rd moments (skewness and kurtosis) with 5th & 6th, so we get:
lower limit = mean - standard deviation * hyperkurtosis + standard deviation * hyperskewness
upper limit = mean + standard deviation * hyperkurtosis + standard deviation * hyperskewness
While extensions model your data’s natural extremums based on current info residing in the data without relying on order statistics, limits model your data's maximum possible and minimum possible values based on current info residing in your data. If a new data point trespasses limits, it means that a significant change in the data-generating process has happened, for sure, not probably—a confirmed structural break.
And finally we use time and volume weighting to include order & process intensity information in our model.
I can't stress it enough: despite the popularity of these non-weighted methods applied in mainstream open-access time series modeling, it doesn’t make ANY sense to use non-weighted calculations on time series data . Time = sequence, it matters. If you reverse your time series horizontally, your means, percentiles, whatever, will stay the same. Basically, your calculations will give the same results on different data. When you do it, you disregard the order of data that does have order naturally. Does it make any sense to you? It also concerns regressions applied on time series as well, because even despite the slope being opposite on your reversed data, the centroid (through which your regression line always comes through) will be the same. It also might concern Fourier (yes, you can do weighted Fourier) and even MA and AR models—might, because I ain’t researched it extensively yet.
I still can’t believe it’s nowhere online in open access. No chance I’m the first one who got it. It’s literally in front of everyone’s eyes for centuries—why no one tells about it?
How to use
That’s easy: can be applied to any, even non-stationary and/or heteroscedastic time series to automatically detect novelties, outliers, anomalies, structural breaks, etc. In terms of quant trading, you can try using extensions for mean reversion trades and limits for emergency exits, for example. The market-making application is kinda obvious as well.
The only parameter the model has is length, and it should NOT be optimized but picked consciously based on the process/system you’re applying it to and based on the task. However, this part is not about sharing info & an open-access instrument with the world. This is about using dem instruments to do actual business, and we can’t talk about it.
∞
Method
RGB Color Codes Chart█ OVERVIEW
This indicator is an educational indicator to make pine coders easier to input color code.
Color code displayed either in hex or rgb code or both.
█ INSPIRATIONS
RGB Color Codes Chart
Table Color For Pairing Black And White
█ FEATURES
Hover table cell to see all properties of color such as Hex code and RGB code via tooltip.
Cell can be show either Full, HEX, RGB, R, G, B or na.
█ LIMITATION
This code does not consider usage of color.new()
█ CONSIDERATION
Code consideration to be used such as color.r(), color.g(), color.b() and color.rgb()
█ EXAMPLE OF USAGE / EXPLAINATION
Regression Channel Alternative MTF V2█ OVERVIEW
This indicator is a predecessor to Regression Channel Alternative MTF , which is coded based on latest update of type, object and method.
█ IMPORTANT NOTES
This indicator is NOT true Multi Timeframe (MTF) but considered as Alternative MTF which calculate 100 bars for Primary MTF, can be refer from provided line helper.
The timeframe scenarios are defined based on Position, Swing and Intraday Trader.
Suppported Timeframe : W, D, 60, 15, 5 and 1.
Channel drawn based on regression calculation.
Angle channel is NOT supported.
█ INSPIRATIONS
These timeframe scenarios are defined based on Harmonic Trading : Volume Three written by Scott M Carney.
By applying channel on each timeframe, MW or ABCD patterns can be easily identified manually.
This can also be applied on other chart patterns.
█ CREDITS
Scott M Carney, Harmonic Trading : Volume Three (Reaction vs. Reversal)
█ TIMEFRAME EXPLAINED
Higher / Distal : The (next) longer or larger comparative timeframe after primary pattern has been identified.
Primary / Clear : Timeframe that possess the clearest pattern structure.
Lower / Proximate : The (next) shorter timeframe after primary pattern has been identified.
Lowest : Check primary timeframe as main reference.
█ FEATURES
Color is determined by trend or timeframe.
Some color is depends on chart contrast color.
Color is determined by trend or timeframe.
█ EXAMPLE OF USAGE / EXPLAINATION
Zigzag Array ExperimentalThis is experimental script for zigzag which uses type, method and array. Not recommend for actual usage, for pine script study maybe useful.
In this experiment, I use type as coded below. It seems have limitation as specially when push as array. As Trading View recommendation, pushing float and int into array especially for type not guarantee to work. I agree with that. Preferred to push array as line or label especially for types.
// @type Used for point especially for array
// @field x int value for bar_index
// @field y float value for price
// @field sty label style
// @field col color for text label
// @field str high or low string
type point
int x = na
float y = na
string sty = na
color col = na
string str = na
I simulate the arrays as below.
var dirLine = array.new()
var dirLabel = array.new()
var dirPoint = array.new()
....
dirPoint.unshift(zigzag.createPoint(0))
dirLabel.unshift(zigzag.createLabel(fontSize, 0, true))
dirLine.unshift(zigzag.createLine(width, switchLine, 0, true))
Here are some results.
RSI Divergence Method█ OVERVIEW
This is a divergence indicator based on Relative Strength Index (RSI).
My attempt to make this indicator updated based on latest pine script features such as type, object and method.
█ FEATURES
1. Color of plot and label is based on contrast color of chart background. Able to customize color from style menu.
2. Big divergence (Regular Divergence) is based on lime / red color.
3. Small divergence (Hidden Divergence) is based on contrast color of chart background.
█ EXAMPLES / USAGES
Forecasting - Seasonal Naive MethodFor completeness here is a naive method with seasonality. The idea behind naive method with seasonality is to take last value from same season and treat it as a forecast. Its counterpart, naive method without seasonality, involves taking last mean value, i.e forecast = sma(x, p) .
Forecasting - Simple Mean MethodThis is a continuation of my series on forecasting techniques. The idea behind the Simple Mean method is to somehow extend historical mean to the future. In this case a forecast equals to last value plus average change.
Forecasting - Holt’s Linear Trend ForecastingHolt's Forecasting method
Holt (1957) extended simple exponential smoothing to allow the forecasting of data with a trend. This method involves a forecast equation and two smoothing equations (one for the level and one for the trend):
Forecast equation: ŷ = l + h * b
Level equation: l = alpha * y + (1 - alpha) * (l + b)
Trend equation: b = beta * (l - l) + (1 - beta) * b
where:
l (or l) denotes an estimate of the level of the series at time t,
b (or b) denotes an estimate of the trend (slope) of the series at time t,
alpha is the smoothing parameter for the level, 0 ≤ alpha ≤ 1, and
beta is the smoothing parameter for the trend, 0 ≤ beta ≤ 1.
As with simple exponential smoothing, the level equation here shows that l is a weighted average of observation y and the one-step-ahead training forecast for time t, here given by l+b. The trend equation shows that b is a weighted average of the estimated trend at time t based on l-l and b, the previous estimate of the trend.
The forecast function is not flat but trending. The h-step-ahead forecast is equal to the last estimated level plus h times the last estimated trend value. Hence the forecasts are a linear function of h.
B3 HL2 Method Candle PainterThis script is similar to the "Hi-Lo" or "Clear" methods of painting bars. Instead of using the tips/edges of the candles like those two, the "(H+L)/2" method uses the change in (high+low)/2 to paint the bars. This gives you some similar results if you were to be binary with the candle coloring. However, my coloring scheme is not entirely binary. There are 5 possible colors:
HL2>LastHigh = Bright Green
HL2LastHL2 = Dull Green
HL2<LastHL2 = Dull Red
Bar Change (close - open) is going against the indicator = Gray
Free to share and enjoy!
~B3
666 & BKThis method uses common numbers which rule the word, 666, so ma 6,12,18,66,180 and the boss method whose ma 7, 14, 21, 77, 231.
