RobPer: An R Package to Calculate Periodograms for Light Curves Based on Robust Regression
Abstract
An important task in astroparticle physics is the detection of periodicities in irregularly sampled time series, called light curves. The classic Fourier periodogram cannot deal with irregular sampling and with the measurement accuracies that are typically given for each observation of a light curve. Hence, methods to fit periodic functions using weighted regression were developed in the past to calculate periodograms.
We present the R package RobPer which allows to combine different periodic functions and regression techniques to calculate periodograms. Possible regression techniques are least squares, least absolute deviations, least trimmed squares, M-, S- and -regression. Measurement accuracies can be taken into account including weights. Our periodogram function covers most of the approaches that have been tried earlier and provides new model-regression-combinations that have not been used before.
To detect valid periods, RobPer applies an outlier search on the periodogram instead of using fixed critical values that are theoretically only justified in case of least squares regression, independent periodogram bars and a null hypothesis allowing only normal white noise. Finally, the package also includes a generator to generate artificial light curves.
Keywords: periodogram, light curves, period detection, irregular sampling, robust regression, outlier detection, Cramér-von-Mises distance minimization, time series analysis, beta distribution, measurement accuracies, astroparticle physics, weighted regression, regression model
1 Introduction
We introduce the R (R Core Team, 2015) package RobPer (Thieler et al., 2015), which can be used to calculate periodograms and detect periodicities in irregularly sampled time series. Our special objective are light curves, which occur in astroparticle physics and are irregularly sampled times series consisting of unequally spaced observation times , observed values and measurement accuracies . The measurement accuracies give information about how precise the were measured. They can be interpreted as estimates for the standard deviations of the observed values. The observed values possibly contain a periodic fluctuation with fluctuation period and the irregularly spaced observation times are realizations of random variables with a periodically shaped density.
Such periodicity in the pattern of the observation times is a typical phenomenon, as the sampling of astroparticle physics’ time series is influenced among others by astronomical constellations. For example, plotting a histogram of the observation times for the gamma particle source Mrk 421 modulo the period shows an unequal distribution over a cycle of this length (see Figure 1). This is due to the fact that observations cannot be sampled during full moon and the moon period is similar to .
So we assume the following model for the observations indexed by :
| (1) | ||||||
| (2) | ||||||
| (3) | ||||||
| (4) | ||||||
where denotes the th ordered observation time in and is a periodic sampling density with period . The observation times and the observed values are realizations of and , respectively. We assume the observation times to be measured without error. is the systematic periodic component in the observations, corresponding to an unknown periodic function and the period we are searching for. is additive noise.
To detect a periodic fluctuation with period in the observed values , it is not possible to use the standard periodogram of Fourier analysis. This method can only be applied to time series with equidistant observation times, while light curves are typically irregularly sampled. A setting-adapted procedure, the Deeming periodogram (Deeming, 1975), is not recommendable either in this case, because it is known to react to a periodicity in the sampling (see Hall and Li 2006).
In order to determine periodicity in light curves, other methods than the classical Fourier periodogram or the Deeming periodogram should be used. Popular periodogram methods in astroparticle physics are for example the Lomb-Scargle periodogram (Scargle, 1982) or the phase dispersion minimization periodogram (Stellingwerf, 1978). These and many other approaches can be generalized to fitting periodic functions to the light curve using least squares regression and calculating periodogram bars based on and , where is the remaining variance in the residuals of the fit and is the overall variance in the observed values . An even broader class of periodogram methods additionally allows application of robust regression instead of least squares regression and weighted regression to take the measurement accuracies into account.
The function RobPer in our homonymous R package calculates a periodogram of a light curve based on fitting periodic functions to using least squares or a robust regression technique, optionally taking measurement accuracies into account using weighted regression. The coefficient of determination corresponding to the objective function of the regression technique is used as periodogram bar. This proceeding incorporates analogues to most of the existing periodograms and introduces several new techniques. Preliminary implementations of most of these periodogram methods have been compared by Thieler et al. (2013). Here, we explain the usage of the R package RobPer, which makes improved and extended methods for period detection publicly available.
This article is organized as follows: In Section 2, the usage and the structure of the function RobPer are explained. Especially, the different periodic functions and regression techniques are discussed and related to the existing periodogram methods. Diagrams which show how this R function is implemented in detail are displayed in Appendix 0.A. Section 3 is devoted to the question how to find valid periods using a periodogram. Thieler et al. (2013) propose robust fitting of a beta distribution combined with outlier detection. The function betaCvMfit in the package RobPer performs this. In Section 4, the function tsgen is presented which allows to generate artificial light curves. Some examples for how to use the package are given in Section 5. Section 6 concludes with a summary.
The RobPer software package is available from the Comprehensive R Archive Network (CRAN) at https://CRAN.R-project.org/package=RobPer. Other R packages implementing periodograms of irregularly sampled time series are the packages lomb (Ruf 1999, function lsp), cts (Wang 2013, function spec.ls) and nlts (Bjornstad 2013, function spec.lomb). They calculate the Lomb-Scargle periodogram, which is based on the least squares fit of a sine function. Furthermore, the package GeneCycle (Ahdesmäki et al. 2012, function robust.spectrum) fits sine functions using robust M-regression to calculate a periodogram based on the square of the estimated amplitude. None of these functions permits taking measurement accuracies using weighted regression into account and most of them (apart from the function spec.lomb) have restrictions concerning the trial periods fitted.
2 Calculate periodograms with RobPer
The R function RobPer calculates a periodogram of a given light curve . This is done by fitting a periodic function to the data . The function has parameters entering linearly. It has a period of 1 and is transformed by for each given trial period . A simple example is . The periodogram bars for the different trial periods are defined as the coefficients of determination of the respective fits. Using weighted regression with weights makes it possible to take the measurement accuracies into account. As the shape of the true fluctuation in Equation 3 is usually unknown, we will typically have .
| Argument | Comment |
|---|---|
| ts or | Light curve or , ; |
| If weighting = FALSE the measurement accuracies column may be omitted. | |
| weighting {T, F} | If TRUE, weighted regression is performed to take into account the . |
| periods | Trial periods . |
| regression | Regression technique (see Section 2.2), possible choices: |
| "L2", "L1", "LTS", "S", "huber", "bisquare", "tau". | |
| model | Periodic fluctuation to be fitted (see Section 2.1), possible choices: |
| "step", "2step", "sine", "fourier(2)", "fourier(3)", "splines". | |
| steps | Number of steps per cycle for periodic step functions. |
| Default: 10 | |
| var1 {T, F} | TRUE sets variance estimate to one for weighted M-regression. |
| Default: weighting | |
| tol | Precision for convergence criteria. |
| Used in case of M-regression and in case of LTS regression if LTSopt = TRUE. | |
| Default: | |
| genoudcontrol | Settings for genoud (see paragraph about LTS regression in Section 2.2): |
| max.generations, wait.generations, pop.size | |
| Used if regression = "bisquare" or LTSopt = TRUE & regression = "LTS". | |
| Default: {50, 5, 50} | |
| LTSopt {T, F} | Determines whether the regression result of ltsReg should be optimized. |
| Default: TRUE if regression = "LTS" | |
| taucontrol {T, F} | Settings for -regression: |
| N, kk, tt, rr, approximate. | |
| Used if regression = "tau", rr only necessary for approximate = TRUE. | |
| Default: {100, 2, 5, 2, FALSE} | |
| Scontrol | Settings for S-regression: |
| N, kk, tt, b, cc, seed. | |
| Used in case of regression = "S". | |
| seed can be fixed in order to get reproducible results or can be left empty. | |
| Default: {N, 2, 5, 0.5, 1.547, NULL} | |
| with N = 50 if weighting = FALSE and N = 200 if weighting = TRUE. | |
| Return value | |
| periodogram | Vector of periodogram bars belonging to the trial periods. |
| Possibly warnings |
Table 1 gives an overview over all arguments of RobPer. The possible shapes of the function that may be fitted by RobPer are presented in Section 2.1. Fitting them using least squares regression is in many cases equivalent to already existing periodogram methods (see Table 2 or Thieler et al. 2013 for a more detailed discussion).
In addition to least squares regression, RobPer offers a selection of robust regression techniques to fit , see Section 2.2. All regression techniques implemented in RobPer are based on minimizing an objective value
| (5) |
with respect to the unknown parameter value , where is the design matrix containing the known components of at the measurement times with being a trial period and the vector of observations . In the simple example mentioned above, the th row of has the elements and . The function is chosen according to the regression method, e.g., for least squares regression. Using the same regression technique, the location of the observations can be estimated minimizing
| (6) |
with being an -variate vector of ones in case of unweighted regression. The periodogram bar can then be calculated as . This definition for the coefficient of determination does not only apply for least squares regression, but also for least absolute deviation- () and M-regression in general (see Maronna et al. 2006, p. 171) as well as for S-, least trimmed squares- (LTS) and -regression (see Croux and Dehon 2003).
If it is intended to take given measurement accuracies into account, weighted regression can be performed. In this case, the terms , and in the two fitted models
| (7) | ||||
| (8) | ||||
| with | (9) |
are replaced by , and , respectively. In the following, we will focus on the case of unweighted regression and only point out the handling of weighted regression, when both procedures differ.
Table 2 displays periodogram methods following the principle of fitting periodic functions. Up to now, weighted regression or robust regression in affiliation with periodic step functions has only been performed by Thieler et al. (2013), though the unweighted least squares versions belong to the most popular periodogram methods in this area of research. S- or -regression, which are also available in RobPer, have not been investigated up to now in this context.
| Model | Regression technique | Publication |
|---|---|---|
| (Name of the method) | ||
| step | L2 | Leahy et al. (1983) |
| (epoch folding) | ||
| L2 | Schwarzenberg-Czerny (1989) | |
| (analysis of variance) | ||
| L2, L1, huber, bisquare | Thieler et al. (2013) | |
| 2step | L2 | Stellingwerf (1978) |
| (phase dispersion minimization) | ||
| L2, L1, huber, bisquare | Thieler et al. (2013) | |
| sine | L2 | Scargle (1982) |
| (Lomb-Scargle) | ||
| L2 | Zechmeister and Kürster (2009) | |
| (generalized Lomb-Scargle) | ||
| L2 | Cumming et al. (1999) | |
| (floating mean) | ||
| L2 | Ferraz-Mello (1981) | |
| (date compensated Fourier transform*) | ||
| L2 | Reegen (2007) | |
| (SigSpec*) | ||
| L1 | Li (2009)*, Li (2010)* | |
| LTS | Ahdesmäki et al. (2007)* | |
| bisquare | Ahdesmäki et al. (2007)* | |
| huber | Zhang and Chan (2005)* | |
| L2, L1, huber, bisquare | Thieler et al. (2013) | |
| fourier(2), | L2 | Hall et al. (2000) |
| fourier(3) | L2 | Palmer (2009) |
| (Fast-) | ||
| L2, L1, huber, bisquare | Thieler et al. (2013) | |
| splines | L2 | Akerlof et al. (1994) |
| L2 | Hall et al. (2000) | |
| L2 | Oh et al. (2004) | |
| (generalized cross validation) | ||
| huber | Oh et al. (2004) | |
| (robust cross validation) | ||
| L2, L1, huber, bisquare | Thieler et al. (2013) |
2.1 Periodic function fitted: Argument model
For each trial period , (given by the argument periods, see Table 1), a periodic function (defined by model) is fitted to the light curve (using regression technique regression). Implemented periodic functions include step functions, sine functions, Fourier series and spline functions.
2.1.1 Step functions
Many periodogram methods from astroparticle physics such as the epoch folding periodogram (Leahy et al., 1983) or the analysis of variance periodogram (Schwarzenberg-Czerny, 1989) can be interpreted as fitting a step function to a light curve (see Schwarzenberg-Czerny 1998 or Thieler et al. 2013). They use periodogram bars related to , and the numbers of steps per cycle.
Another typical periodogram method in astroparticle physics is the phase dispersion minimization periodogram (PDM, Stellingwerf 1978). Depending on the particular setting the periodogram bar in many cases equals the mean of the coefficients of determination of two fits with different step functions with staggered jumps (see Thieler et al. 2013 or Thieler 2013 for more details).
RobPer provides two options to fit periodic step functions. The number of steps per cycle is controlled by the argument steps. Using model = "step", a single periodic step function with steps of equal width is fitted for each trial period. Performing regression = "L2", model = "step" is equivalent to calculating an epoch folding- or analysis of variance periodogram. Using model = "2step", two different step functions with opposed jump times and steps of equal width are fitted separately and the periodogram bar is the mean of both coefficients of determination. This is the only option where two periodic functions are fitted for one trial period. It is included to provide the PDM periodogram with overlapping bins.
2.1.2 Sine functions
Sine functions are periodic and quite popular for investigating periodicity. The classic periodogram of Fourier analysis for equally sampled time series represents the explained variance SE of a least squares fit of a sine model to the zero-centered time series. The Lomb-Scargle periodogram (Scargle, 1982) works equivalently for unequally sampled time series.
As the mean of an irregularly sampled time series is not identical to the least squares fit of an intercept in a sine model, more recent methods use the uncentered data and fit a model with intercept, e.g., the floating mean periodogram by Cumming et al. (1999) and the generalized Lomb-Scargle periodogram by Zechmeister and Kürster (2009). Performing regression = "L2", model = "sine" is equivalent to calculating those periodograms and in case of equidistant observation times also equivalent to the Fourier periodogram.
Some other methods as the Date Compensated Fourier Transform by Ferraz-Mello (1981), the SigSpec periodogram by Reegen (2007) or robust approaches by Ahdesmäki et al. (2007) and Zhang and Chan (2005) apply the same regression step as the floating mean- and the generalized Lomb-Scargle periodogram, but use the squared amplitude of the fitted sinusoid as the periodogram bar. In case of regular sampling, this is another representation of the classical periodogram of Fourier analysis. As the amplitude is a concept closely related to trigonometric functions, RobPer uses the coefficient of determination only, to obtain a general method independent of the periodic function chosen.
2.1.3 Further periodic functions
Recently, fitting more complex periodic functions has been proposed for periodograms. Fourier series (see Hall et al. 2000 and Palmer 2009) and periodic splines (see Akerlof et al. 1994, Hall et al. 2000 and Oh et al. 2004) may provide better adaptivity compared to sine functions, but still present a continuous function, unlike the step function. RobPer offers the possibility to fit Fourier series of second (model = "fourier(2)") or third (model = "fourier(3)") degree or a periodic spline function with four knots per cycle (model = "splines"). For the latter option, B-splines are generated using the function spline.des from the package splines (Bates and Venables, 2016).
2.2 Regression techniques: Argument regression
Instead of fitting the models mentioned above by the popular least squares regression (see Table 2), RobPer also allows application of six robust regression techniques, see Table 3. Robust regression techniques like least absolute deviations, least trimmed squares (Rousseeuw and Yohai, 1984) and M-regression (Huber and Ronchetti, 1981) have already been used to fit sines (evaluating the squared amplitude) by Zhang and Chan (2005), Ahdesmäki et al. (2007), Li (2009) and Li (2010). M-regression with the Huber function was applied to fit periodic splines by Oh et al. (2004). Thieler et al. (2013) use least absolute deviations and M-regression and all models described in this article to calculate periodograms based on the coefficient of determination.
To the best of our knowledge, S- (Rousseeuw and Yohai, 1984) and -regression (Yohai and Zamar, 1988) have not been used before in periodogram calculation. For the latter, RobPer uses the algorithms Fast-S from Salibian-Barrera and Yohai (2006) and Fast- from Salibian-Barrera et al. (2008) and slightly modified versions of the code distributed with the respective publication (see the respective paragraphs entitled in Section 2.2). The following paragraphs outline the algorithms used by RobPer for calculating the different regression estimators. For the basic definitions of these regression techniques we refer to the literature mentioned above and the book by Maronna et al. (2006).
| Regression technique | regression | R function (package) |
|---|---|---|
| Least squares | "L2" | lm (stats, R Core Team 2014) |
| Least absolute deviations | "L1" | rq (quantreg, Koenker 2015) |
| Least trimmed squares | "LTS" | ltsReg (robustbase, Rousseeuw et al. 2015) |
| M-regression | ||
| …with Huber function | "huber" | Own implementation. |
| …with Bisquare function | "bisquare" | lmrob..M..fit (robustbase, Rousseeuw et al. 2015) |
| S-regression | "S" | Slightly modified code from Salibian-Barrera and Yohai (2006). |
| -regression | "tau" | Slightly modified code from Salibian-Barrera et al. (2008). |
2.2.1 LTS regression
The R function ltsReg from package robustbase (Rousseeuw et al., 2015) is used to perform LTS regression in RobPer. In preliminary studies we observed that the function can have problems finding a good solution for some of the candidate periods. This results in coefficients of determination which are too small or sometimes even negative. By setting LTSopt = TRUE, it is possible to let RobPer further optimize the solution of ltsReg by using the R function genoud from package rgenoud (Mebane, Jr. and Sekhon, 2011). This function uses an evolutionary approach to improve the given solution, locally optimizing the temporarily best solutions in a gradient descent algorithm. Further arguments pop.size (size of one generation), max.generations (maximum of generations before stopping the algorithm) and wait.generations (maximum number of generations to wait for an improvement of the optimization criterion) control the behavior of the algorithm and can be set in RobPer by the argument genoudcontrol (see Table 1). The argument tol controls the precision for convergence criteria.
A further problem we observed is that ltsReg sometimes aborts the fit. However, it is typically able to perform the fit if it is run again. In case of a crash, RobPer calls ltsReg up to three times. After the third failed attempt, the respective periodogram bar is set to NA, or a least absolute deviation regression is performed. The latter is done, if the ltsReg regression result should be further processed, using the genoud algorithm or using the LTS result as initial estimate for an M-regression fit (see next paragraph).
2.2.2 M-regression
In case of M-regression, a periodogram bar, i.e., the coefficient of determination is calculated from the values
| (10) |
and
| (11) |
where is an estimate of the error scale in the regression model. As explained above, Equations 10 and 11 represent the minimization criteria of the fits of the chosen periodic fluctuation ( in Equation 5) and of a location estimate (SY in Equation 6), respectively. The function is a distance measure. The vector consists of ones in case of unweighted regression. As mentioned before, in case of weighted regression, , and the rows of the design matrix are standardized by the measurement accuracy (see Figure 12 in Appendix 0.A).
The value is obtained in an initial estimation of the periodic fluctuation, calculating a scale estimate of the fitted residuals. In principle, one could use a different estimate of calculated from fitting only an intercept in Equation 11, but Maronna et al. (2006, p. 171) recommend using the scale estimate from the (larger) regression model. In our context this means that depends on the trial period and cannot be calculated globally. On the other hand this ensures that the regression model , is a generalization of the intercept model , and thus and .
So for this regression technique, an implementation is needed where the scale estimate can be fixed in advance. For M-regression using the biweight function, the function lmrob..M..fit from package robustbase by Rousseeuw et al. (2015) is used. This R function includes Huber M-regression only as a limiting case of Hampel M-regression with all but one of its tuning constants set to very large values. In other R functions known to us for M-regression (rlm from package MASS byVenables and Ripley 2002, iwlsm from package RSiena by Ripley et al. 2013 and robustregBS and robustRegH from package robustreg by Johnson 2015), the scale estimate cannot be fixed in advance. Hence M-regression using the Huber function is newly implemented for RobPer. Like the functions specified before, this implementation is based on an iteratively reweighted least squares (IRWLS) approach (see Maronna et al. 2006, pp. 104–105), and meets our special requirements. For M-regression using the biweight function, the implementation makes also use of the function genoud from package rgenoud (see previous paragraph) to overcome possible problems with local optima.
As noted above, weighted regression scales observed values and design matrices by the measurement accuracies. The variance of the error is expected to be about one then. Hence it can be reasonable to set to one. This can be done in RobPer setting the argument var1 to TRUE, as is recommendable in our experience in case of weighted M-regression.
To calculate a periodogram bar using M-regression with IRWLS, three initial estimates are needed: A scale estimate (if not set to one) and initial location estimates and for and . The initial estimates should be obtained using robust techniques. As proposed by Maronna et al. (2006, p. 105) we use the median (weighted if the shall be taken into account) to initially estimate . For , LTS regression (see previous paragraph) is used. It has a high breakdown point and is appropriate in situations with many observations not agreeing with the best fit. This situation will often occur in periodogram calculation, as many trial periods and thus many wrong models are fitted to the light curve. The scale estimate is calculated as the (weighted) median of the residuals of the LTS fit.
2.2.3 S-regression
In case of regression = "S", RobPer uses the Fast-S algorithm by Salibian-Barrera and Yohai (2006) to perform S-regression for fitting the periodic function efficiently. The algorithm starts with a set of N parameter candidates, locally optimizes them using kk iterations, then optimizes the tt best of these candidates until convergence and finally chooses the best parameter candidate.
The R function FastS used in RobPer is a slightly modified version of the function fast.s published by Salibian-Barrera and Yohai (2006). It was changed in order to work more efficiently in the context given here, especially when fitting step functions, and to specify one parameter candidate in advance. This candidate is set to
| (12) |
where denotes the dimension of the linear model of the periodic function and denotes the location estimate. arises from plugging in the fit obtained from the location model into the parametrization of the full model. This ensures that fitting the full periodic function will not give a worse fit than fitting only a location parameter. Otherwise it could happen that SY SE and the coefficient of determination (which has to be in ) would be negative.
Further changes in FastS are:
- 1.
The arguments k and best.r are renamed to kk and tt to unify notation as in FastTau. The arguments int, N, kk, tt, b, cc and seed are merged to a list Scontrol, which is also an argument of RobPer (except for int, which is fixed in RobPer).
- 2.
If an intercept column is added to the design matrix (using Scontrol$int = TRUE), this is done before the dimension of the design matrix is determined (instead of doing this first and redoing it in case of Scontrol$int = TRUE).
- 3.
To find a subsample in general position, regressors are sampled from the set of rows of the design matrix ignoring the frequency of occurrence in . For each regressor , one value is then sampled from the entries of belonging to this regressor. In case of a step function to be fitted, one observation per step is drawn to get a subsample.
- 4.
If no subsample can be found in 100 trials, FastS returns NA. RobPer then releases a warning, but can calculate further periodogram bars for other trial periods.
- 5.
The internal functions loss.S, re.s, f.w, scale1, our.solve and rho are now defined outside FastS. Otherwise R would have to redefine them for each periodogram bar.
- 6.
The subfunction norm is replaced by the function norm(..., "2") from the package base (R Core Team, 2016).
- 7.
The labels of the return values are changed for better interpretation.
2.2.4 -regression
In case of regression = "tau", -regression is used to fit the periodic function. RobPer uses the Fast- algorithm of Salibian-Barrera et al. (2008) which works according to the same optimizing principle as FastS for S-regression (see previous paragraph), i.e., optimizing N candidates in kk iterations and further optimizing the tt best of these until convergence. Since computation of the objective value is expensive, it is possible to approximate it with rr iteration steps when choosing approximate = TRUE. For more details see Salibian-Barrera et al. (2008).
The R function FastTau used in RobPer is a slightly modified version of the R code published in Salibian-Barrera et al. (2008) with similar changes as in FastS compared to fast.s (see previous paragraph). The changes are:
- 1.
A candidate for , see Equation 12, is allowed.
- 2.
Arguments N, kk, tt, rr and approximate are combined to a list taucontrol, which is also an argument for RobPer.
- 3.
Subsamples in general position are found as in FastS (change 3 in the previous paragraph).
- 4.
If no subsample can be found, FastTau returns NA instead of a break using the stop function. This allows RobPer to release a warning, while calculating further periodogram bars for other trial periods.
- 5.
A block of code used several times to check new regression parameter candidates for providing the best optimization value so far has been modularized into the subfunction checkbest.
- 6.
Due to rounding errors, it may happen in the IRWLS algorithm that negative values close to zero occur, although they have to be non-negative by theory. This is avoided by setting such values to zero.
- 7.
The subfunction randomset is replaced by the R function sample from the base package as both functions fulfill the same task and sample is faster.
- 8.
The labels of the return values are changed for better interpretation.
3 Fit beta distributions with betaCvMfit
In this section we present the function betaCvMfit, which robustly fits a beta distribution to a sample using Cramér-von-Mises (CvM) distance minimization. The function is adapted from R code by Brenton R. Clarke for fitting a gamma distribution (see Clarke et al. 2012) using CvM distance minimization. Section 3.1 motivates the application of this function, while its usage is explained in more detail in Section 3.2.
3.1 Motivation
After a periodogram is calculated, one might be interested in the automatic detection of significant periods. A period shall be called significant, if the respective periodogram bar is atypical from the distribution of the applied criterion under the null hypothesis of no periodic fluctuation. To determine significance, this distribution needs to be known or estimated. Let be the -quantile of this distribution. Assuming independent identically distributed periodogram bars we get
| (13) |
A single periodogram bar calculated as described in Section 2 using unweighted least squares regression is -distributed, where denotes the beta distribution and is the dimension of the model. This result can be found in Schwarzenberg-Czerny (1998) or easily be deduced from Seber and Lee (2003, p. 110) and Gupta and Nadarajah (2004, p. 51). Already small violations of the assumptions made about the method or the light curve disturb this proceeding. In this work, we consider weighted and robust regression in addition to ordinary least squares. Besides, we have to take into account small deviations from our model assumptions like bad estimates . An example is shown in Figure 2. Panel 2a shows the weighted least squares periodogram (using a sine model) of a light curve only consisting of white noise. The observed values were generated as
| (14) | ||||
| with and being realizations from | ||||
| (15) | ||||
| (16) | ||||
| The value of is given for all , and is chosen to fulfill | ||||
| (17) | ||||
where denotes the empirical variance. This means, there is roughly an extra 20 percent noise which is not explained by the measurement accuracies. Evidently, no periodogram bar is outstanding, but using the quantile of a distribution (dashed line), several periods are found automatically.
To circumvent these problems, Thieler et al. (2013) propose to relax the assumption of a predefined -distribution and only assume that the periodogram values can be approximated by any beta distribution. As peculiar periods are expected to show up as outliers, robustly fitting a -distribution to is proposed. The authors use CvM distance minimization for this, which has been recommended by Clarke et al. (2012) for fitting gamma distributions in the presence of outliers. The CvM is defined as
| (18) |
where is the ordered sample, is the empirical distribution function and is the distribution function of .
Panel 2b shows the predefined (solid) and the CvM-fitted (dashed) beta density for a periodogram calculated from the only-noise-data described above. While the quantile of the predefined distribution is about 0.03, the related quantile of the fitted distribution is 0.16 and no period is detected automatically.
The above approach falls within the framework of outlier detection described by Davies and Gather (1993) and is successfully used by Thieler et al. (2013) in the context discussed here. However, it assumes independent periodogram bars. This may cause problems when the periodogram peaks are broad (because the assumption of independency of the periodogram bars is violated): Then it can be hard for the automatism to find any outlying periodogram value, as there are many high values. One might try to ease this problem choosing a selection of trial periods with large distances or considering only the periods referring to local maxima in the periodogram as (roughly) independent trial periods (modifying and expanding an approach of Zechmeister and Kürster 2009) and fit the beta distribution to them using a CvM fit.
Simulations indicate that the beta distribution describes the distribution under the null hypothesis rather well for the different periodograms. Nevertheless, in the following we will call detected periods “valid” and not “significant” to stress that our approach to detect periods lacks a theoretical justification.
3.2 The R function betaCvMfit
The function betaCvMfit fits a -distribution with mean to a sample vector data using CvM distance minimization and has been applied in Thieler et al. (2013) for fitting beta distributions to periodograms to detect valid periods.
As it may happen that the periodogram bars become negative due to fitting problems, the function sets all negative entries of data to zero. If the logical argument CvM is set to TRUE, a CvM fit is calculated. As initial values for the optimization, the moment estimates of the beta distribution
| (19) |
are used. If the argument rob is set to TRUE, the median and the median absolute deviation from the median (MAD) are used instead of the arithmetic mean for and the standard deviation for , respectively. In case of a very small estimate (which happens particularly if is the MAD), the function stops as it is not possible to calculate the estimates and shown above. The parameters of a beta distribution are strictly positive. Since it can happen that or are negative, the initial estimates are clipped to be at least 0.00001. If CvM is set to FALSE, the CvM distance is not optimized, and the initial estimates and are returned.
Figure 3 shows the different fits varying the arguments CvM and rob for 50 -distributed observations containing 10 percent outliers between 0.8 and 1.
4 Generate light curves with tsgen
To investigate our periodogram methods in simulations, we implemented the R function tsgen to generate artificial light curves. A preliminary version of this function is used in Thieler et al. (2013). The light curves are generated as realizations of the model
| (20) | ||||
| (21) | ||||
| (22) | ||||
| (23) | ||||
| (24) |
where denotes the th ordered observation time in and is a periodic sampling density with period . The noise component is a power law noise (see Timmer and König 1995) with power exponent and is white noise in case of . Inserting another noise component and two types of outliers, this extended model allows to generate data violating the model introduced in Section 1.
The function calls several autonomous subfunctions one by one which perform individual simulation steps. These are:
- 1.
Generate a sampling (using sampler, see Section 4.1).
- 2.
Generate a periodic signal (using signalgen, see Section 4.2).
- 3.
Add noise with related measurement accuracies and a noise component unrelated to the (using lc_noise, see Section 4.3).
- 4.
Disturb the light curve replacing measurement accuracies by outliers, or replacing observations by aperiodic features (using disturber, see Section 4.4).
Table 4 lists all arguments for the subfunctions. The gray-shaded arguments are also arguments for tsgen, which passes them to the respective subfunction.
| Argument | Subfunction | Comment |
|---|---|---|
| ps | sampler, disturber | Sampling period . Default 1. |
| ncycles | sampler | Number of sampling cycles. |
| npoints | sampler | Sample size . |
| ttype | sampler | Distribution of the unsorted observation times. Options are: "equi" (equidistant sampling), "unif" (uniform sampling), "sine" (sine-shaped density, see Section 4.1) and "trian" (triangular density, see Section 4.1). |
| tt | signalgen, lc_noise, disturber | Observation times , e.g., return value from sampler. |
| pf | signalgen | Fluctuation period . Default 1. |
| ytype | signalgen | Type of periodic fluctuation . Options: "const" (constant), "sine" (sine), "trian" (triangular function) and "peak" (peak function). |
| sig | lc_noise | Values of the periodic fluctuation, e.g., return value from signalgen. |
| SNR | lc_noise | Relation . |
| redpart | lc_noise | Fraction of noise not related to measurement accuracies. |
| alpha | lc_noise | Power law coefficient of the noise component . Set to zero for . |
| y | disturber | Observed values , e.g., return value from lc_noise. |
| s | disturber | Measurement accuracies , e.g., return value from lc_noise. |
| s.outlier.fraction | disturber | Fraction of measurement accuracies to be replaced by outliers. |
| interval {TRUE, FALSE} | disturber | If TRUE, the belonging to a random time interval are disturbed. |
4.1 Generate sampling using sampler
The R function sampler is used to sample observation times in the interval with a possibly periodic sampling of period . The sampling pattern depends on the argument ttype (see Table 4). If a periodic pattern is chosen, the observed time interval covers cycles of it.
In case of ttype = "equi", the observation times are equidistantly sampled with . For ttype = "unif", the observation times are drawn independently from a uniform distribution on . Both these sampling schemes are aperiodic, the sampling period only influences the duration of the sampling.
For ttype = "sine" and ttype = "trian", the observation times are sampled from a periodic density with sampling period . First, observation cycles are drawn from a discrete uniform distribution on to determine the cycle the th observation is part of. Second, observation phases are sampled with density
| (for ttype = "sine") | (25) | ||||
| (26) |
To sample from , the function BBsolve from package BB (Varadhan and Gilbert, 2009), is used. The unsorted observation times are then generated using
| (27) |
The sine-shaped density is motivated by sampling patterns observed in real data, see Panel 1b. The triangular shaped density offers an alternative periodic sampling design. Separately sampling observation cycle and phase was proposed by Hall and Yin (2003).
As the result, sampler returns the ordered observation times .
4.2 Generate periodic signal using signalgen
To generate the periodic component in the observed values, the R function signalgen is used. The values with fluctuation period at observation times are generated using
| (28) |
The observation times, the fluctuation period and the shape of are arguments of signalgen (see Table 4). In case of ytype = "const", is defined as
| (29) | ||||
| so there is no (periodic) fluctuation. This setting can be used to investigate the false alarm probability of a period detection method. In case of ytype = "sine", is defined as | ||||
| (30) | ||||
| This is a typical assumption in the literature. For ytype = "trian", | ||||
| (31) | ||||
| with is used. This triangular shaped function was originally implemented in order to be able to choose between different periodic shapes. The light curve observed for CoRoT ID 0105288363 (Chadid et al., 2011) shows that functions with a similar shape are quite realistic. When choosing ytype = "peak", is generated using | ||||
| (32) | ||||
This function mostly shows values close to zero and large values for only one time unit per cycle. This “peak” occurring in each cycle has an asymmetric shape.
As the result, signalgen returns the periodic component of the observed values.
4.3 Add noise and measurement accuracies using lc_noise
The R function lc_noise is used to generate measurement accuracies and add noise to a periodic fluctuation (see Table 4). The measurement accuracies are sampled from a gamma distribution. This choice is motivated by real data from Tluczykont et al. (2010). As shown in Equation 4, the noise component is a realization of with .
A second noise component does not depend on the . It is generated as red noise, i.e., following a power law with power law index . For we get white noise. Flicker noise (pink noise) is generated using and brown noise using . The power law noise is generated using subfunctions TK95_uneq and TK95. The latter generates an equidistant time series of power law noise according to Timmer and König (1995). For irregular observation times, a noise series resulting from TK95 is used and an unequally sampled noise series is generated following Uttley et al. (2002).
The noise components are scaled so that the variance of the has approximately the proportion redpart in the overall noise variance and that SNR is the ratio , where is the empirical variance of vector . Note that the white noise components’ variances are exactly , so that the are not estimates but true values. In this sense, the measurement accuracies of a generated light curve are more informative for our artificial data than for real light curves, where the measurement accuracies are estimates. Allowing for a second noise component makes it possible to lower the information of the measurement accuracies with respect to the overall noise in the observed values.
The function lc_noise returns the observed values .
4.4 Disturb light curve using disturber
The last subfunction applied in tsgen is disturber, which can be used to disturb a given light curve (see Table 4). It replaces a given fraction of measurement accuracies by the smaller value , in a subset of . As small measurement accuracies stand for precise observations, the influence of observations with disturbed measurement accuracies rises in case of a weighted fit. For unweighted regression, this type of disturbance does not affect the result of the fit.
Optionally, disturber also replaces observed values by atypical values. For this, a time interval within the interval is randomly chosen and all observed values belonging to this time interval are replaced by a peak function:
| (33) |
where denotes the density of the standard normal distribution. If the are intended to be disturbed and the light curve is shorter than , the function will stop with an error message.
The function returns the modified vectors and . If the option to change values is not used (see Table 4) and the fraction of outlying measurement accuracies is set to zero, and are returned unchanged.
5 Application
In this section, we give examples how to use the RobPer package for light curve analysis. We start with an artificial example, also given in the manual, and then analyze some real data.
5.1 Artificial example
To generate an artificial light curve, tsgen can be used:
R> library("RobPer")R> set.seed(22)R> lightcurve <- tsgen(ttype = "sine", ytype = "peak", pf = 7,+ redpart = 0.1, s.outlier.fraction = 0.1, interval = TRUE,+ npoints = 200, ncycles = 25, ps = 20, SNR = 3, alpha = 0)This light curve has a sine-shaped sampling (ttype) with sampling period 20 (ps) and covers a time interval of about 25 sampling cycles (ncycles), so 500 time units. It consists of 200 observations (npoints) and the observed values contain a peak-shaped periodic fluctuation (ytype) with fluctuation period 7 (pf). The measurement accuracies are related to about 90 percent of the noise component (1-redpart), the rest of the noise is white as well (alpha). The empirical variance of the periodic fluctuating component in the observed values is three times larger than the empirical variance in the noise component (SNR). The light curve contains 10 per cent outliers in the measurement accuracies (s.outlier.fraction) and atypical observed values (interval).
Alternatively, the functions sampler, signalgen, lc_noise and disturber can be used to generate the same light curve, see Section 4.
Sampling observation times:
R> set.seed(22)R> tt <- sampler(ttype = "sine", npoints = 200, ncycles = 25, ps = 20)Generate periodic fluctuation:
R> yf <- signalgen(tt, ytype = "peak", pf = 7)Add noise and scale signal to the right SNR:
R> temp <- lc_noise(tt, sig = yf, SNR = 3, redpart = 0.1, alpha = 0)R> y <- temp$yR> s <- temp$sReplace measurement accuracies by tiny outliers and include a peak:
R> temp <- disturber(tt, y, s, ps = 20, s.outlier.fraction = 0.1,+ interval = TRUE)The result is the same:
R> all(cbind(tt, temp$y, temp$s) == lightcurve)
Figure 4 shows plots of the generated light curve.
In the next step, we calculate a periodogram of the light curve. The periodogram is calculated fitting a step model using unweighted M-regression with the Huber function. The light curve spans a time interval of approximately ncycles ps time units, so it is sensible to investigate periods up to 50 (one tenth, see Halpern et al. 2003).
R> PP <- RobPer(lightcurve, model = "splines", regression = "huber",+ weighting = FALSE, var1 = FALSE, periods = 1:50)
Outstanding periodogram bars are sought fitting a beta distribution to the periodogram values using Cramér-von-Mises distance minimization (CvM) and determining the -quantile with as the number of periodogram bars.
R> betavalues <- betaCvMfit(PP)R> crit.val <- qbeta((0.95)^(1 / 50), shape1 = betavalues[1],+ shape2 = betavalues[2])Panel 5a depicts the histogram of the periodogram bars, the beta distribution fitted (solid line) and its -quantile (solid vertical line). Further fits of a beta distribution (method of moments, dashed, and robust method of moments, dotted) and their respective -quantiles are shown as well.
R> hist(PP, breaks = 20, freq = FALSE, xlim = c(0, 0.08), col = "grey",+ main = "", xlab="Periodogram")R> betafun <- function(x) dbeta(x, shape1 = betavalues[1],+ shape2 = betavalues[2])R> curve(betafun, add = TRUE, lwd = 2)R> abline(v = crit.val, lwd = 2)Application of method of moments:
R> par.mom <- betaCvMfit(PP, rob = FALSE, CvM = FALSE)R> myf.mom <- function(x) dbeta(x, shape1 = par.mom[1], shape2 = par.mom[2])R> curve(myf.mom, add = TRUE, lwd = 2, lty = 2)R> crit.mom <- qbeta((0.95)^(1 / 50), shape1 = par.mom[1],+ shape2 = par.mom[2])R> abline(v = crit.mom, lwd = 2, lty = 2)Application of robust method of moments:
R> par.rob <- betaCvMfit(PP, rob = TRUE, CvM = FALSE)R> myf.rob <- function(x) dbeta(x, shape1 = par.rob[1], shape2 = par.rob[2])R> curve(myf.rob, add = TRUE, lwd = 2, lty = 3)R> crit.rob <- qbeta((0.95)^(1 / 50), shape1 = par.rob[1],+ shape2 = par.rob[2])R> abline(v = crit.rob, lwd = 2, lty = 3)R> legend("topright", lty = 1:3, legend = c("CvM", "Moments",+ "Robust moments"), bg = "white", lwd = 2)R> box()
Using the quantile of the CvM fit (solid line), a period of 7 time units seems to be valid, see Panel 5b. Twice this period, which is 14, might be valid, too. So the real periodic fluctuation of is well recognized within the disturbed signal, as intended. Of course, a periodic function with period is also periodic with period , .
R> plot(1:50, PP, xlab = "Trial period", ylab = "Periodogram", main = "",+ type = "l")R> abline(h = crit.val, lwd = 2)R> text(7, PP[7]-0.002,7, pos=4)R> text(14, PP[14]+0.002,14, pos=4)While the robust M-regression recognizes the real periodic fluctuation, fitting the same model by least squares regression does not, as shown in Figure 6. Only the periodogram is calculated in another way.
R> PP <- RobPer(lightcurve, model = "splines", regression = "L2",+ weighting = FALSE, var1 = FALSE, periods = 1:50)The analysis proceeds as before.
5.2 Disturbed data from GROJ0422+32
The first real data set we analyze is a light curve for gamma ray emission of the source GROJ0422+32, obtained by the BATSE Earth Occultation Monitoring project of the NASA. These experiments are described in Harmon et al. (2002) and Harmon et al. (2004). The data have been kindly provided by the NASA, are available from http://gammaray.nsstc.nasa.gov/batse/occultation, and are shown in Panel 7a.
A large peak is visible starting at about 48900 Markarian Julian days (which corresponds to December 10 1991 in the Gregorian calendar), a so called gamma ray burst. It occasionally occurs in gamma ray observations and can be considered as outlier. The light curve covers a time interval of about 3312 days, so following Halpern et al. (2003) we consider periods up to 330 days (about one tenth of the overall duration of the light curve). Figure 7b shows the periodogram obtained fitting a sine function using least squares regression, which is the classical approach in astroparticle physics. It is calculated using
R> data(star_groj0422.32)R> PP <- RobPer(star_groj0422.32, periods = 1:330, model = "sine",+ regression = "L2", weighting = FALSE)Periodograms for -regression and M-regression using the Huber function are obtained replacing "L2" by "tau" or "huber" in the code above. The respective periodograms are shown in Panels 7c and 7d. All three periodograms do not show any outstanding peak. Apart from this, the periodograms using robust regression have a completely different shape than the least squares periodogram, which seems to have problems with the gamma ray burst. It might be questionable if the least squares periodogram can find a periodic structure in the observations in the presence of the gamma ray burst. We add a sine with period 30 and amplitude 0.005 to the observed values and repeat the analysis. The results can be seen in Figure 8. In Panel 8a it is visible that we did not introduce a strong periodic behavior. Nevertheless, the robust periodograms, Panels 8c and 8d, easily detect it, while there is only a small local peak in the least squares periodogram in Panel 8b. The horizontal lines in Panels 8c and 8d show the respective -quantiles of the CvM-fitted beta distribution and are calculated from a periodogram PP using
R> shapes <- betaCvMfit(PP)R> Crit <- qbeta(0.95^(1 / 330), shape1 = shapes[1], shape2 = shapes[2])So, as opposed to least squares regression, robust techniques are able to detect an (added) periodic fluctuation although the data are disturbed seriously by the gamma ray burst.
5.3 Data from Markarian 421 and 501
A further real data example are gamma ray light curves from Markarian 421 (Mrk 421) and Markarian 501 (Mrk 501), kindly provided by the Gamma Astronomy group of the Deutsches Elektronen-Synchrotron. The data have been collected from various original sources, combined, and published by Tluczykont et al. (2010), and are available from http://astro.desy.de/gamma_astronomy/magic/projects/light_curve_archive/index_eng.html. See the RobPer manual for details about the original sources and references.
The light curve obtained for Mrk 421 is shown in Panel 1a on page 1. Periodograms obtained fitting a sine are shown in Figure 9. Using the least squares periodogram in Panel 9a, no valid period is detected, but considering the shape of the periodogram, one might wonder if there is a periodicity of 31 hidden in the same way as when adding a small periodic fluctuation to the GROJ0422+32 data, see Panel 8b. However, the periodograms for -regression in Panel 9b and Huber M-regression in Panel 9c show a different behavior from Figure 8, so this does not seem to be the case. Especially, the least squares and the Huber M periodogram show a quite similar behavior regarding the local maxima. This could mean that there are not many observations weighted down in Huber M-regression.
Another light curve, obtained for Mrk 501, and periodograms using least squares regression, -regression and Huber M-regression are shown in Figure 10. Here we apply step regression, which is equivalent to epoch folding or phase dispersion minimization when using least squares regression (see Section 2). The periodogram is calculated applying
R> data(Mrk501)R> RobPer(Mrk501, periods = 1:400, model = "step", regression = "L2",+ weighting = FALSE)in case of least squares regression and with regression = "tau" or regression = "huber" in case of - or Huber M-regression, respectively. For least squares regression in Panel 10b and Huber M-regression in Panel 10d we see a broad peak between the trial periods 200 and 300, much too broad to be considered as valid period (see Halpern et al. 2003). For -regression in Panel 10c, this behavior is not observed.
In the examples from the previous section, robust techniques recognize some periodicity in a light curve, while the least squares periodogram only provides a slightly atypical behavior for the trial period in question. Here it is the other way round: the least squares periodogram does not indicate a valid period, but exhibits some interesting feature similar to the previous data set, where a periodicity was hidden in noisy data. This initial suspicion cannot be confirmed by using robust regression instead of least squares regression. In summary, using our methods, we do not find a periodicity in the light curves for Mrk 421 and Mrk 501, neither using least squares nor robust regression.
6 Conclusions
The R package RobPer presented in this work allows searching for periodicity in irregularly sampled time series, possibly taking into account additional information on the precision of the measurement, if available. These are the typical characteristics of light curves, that is time series occurring in astroparticle physics. The periodogram is calculated fitting periodic functions to the light curve. The user can choose between six different periodic functions and seven different regression techniques, meaning that 42 possible combinations are offered, not taking into account further options like choosing the number of steps for the step model or using weighted regression. The function betaCvMfit allows to search for prominent periodogram bars as outliers in a beta distribution robustly fitted to the periodogram. The function tsgen allows generation of artificial light curves for investigative use.
Acknowledgments
The authors are grateful to the reviewers for their constructive comments which led to several improvements of the paper and the programs. The financial support of the Deutsche Forschungsgemeinschaft (DFG) (within the Collaborative Research Center SFB 876 “Providing Information by Resource-Constrained Data Analysis”, project C3, and Graduate School GK 1032 “Statistische Modellbildung”) is gratefully acknowledged. We thank the ITMC at TU Dortmund University for providing computer resources on LiDO. We thank the NASA and the Deutsches Elektronen-Synchrotron for providing data.
References
- GeneCycle: identification of periodically expressed genes. Note: R package version 1.1.2 External Links: Link Cited by: §1.
- Robust regression for periodicity detection in non-uniformly sampled time-course gene expression data. BMC Bioinformatics 8 (1), pp. 233–248. External Links: Document Cited by: §2.1.2, §2.2, Table 2.
- Application of cubic splines to the spectral analysis of unequally spaced data. The Astrophysical Journal 436, pp. 787–794. External Links: Document Cited by: §2.1.3, Table 2.
- splines: regression spline functions and classes. Note: Base R package version 3.2.4 External Links: Link Cited by: §2.1.3.
- nlts: (non)linear time series analysis. Note: R package version 0.2-0 External Links: Link Cited by: §1.
- CoRoT light curves of blazhko rr lyrae stars. Astronomy & Astrophysics 527, pp. A146. External Links: Document Cited by: §4.2.
- A fast robust method for fitting gamma distributions. Statistical Papers 53 (4), pp. 1001–1014. External Links: Document Cited by: §3.1, §3.
- Estimators of the multiple correlation coefficient: local robustness and confidence intervals. Statistical Papers 44 (3), pp. 315–334. External Links: Document Cited by: §2.
- The lick planet search: detectability and mass thresholds. The Astrophysical Journal 526 (2), pp. 890–915. External Links: Document Cited by: §2.1.2, Table 2.
- The identification of multiple outliers. Journal of the American Statistical Association 88 (423), pp. 782–792. External Links: Document Cited by: §3.1.
- Fourier analysis with unequally-spaced data. Astrophysics and Space Science 36 (1), pp. 137–158. External Links: Document Cited by: §1.
- Estimation of periods from unequally spaced observations. The Astronomical Journal 86 (4), pp. 619–624. External Links: Document Cited by: §2.1.2, Table 2.
- Handbook of beta distribution and its applications. Dekker, New York, Basel. Cited by: §3.1.
- Using the periodogram to estimate period in nonparametric regression. Biometrika 93 (2), pp. 411–424. External Links: Document Cited by: §1.
- Nonparametric estimation of a periodic function. Biometrika 87 (3), pp. 545–557. External Links: Document Cited by: §2.1.3, Table 2.
- Nonparametric methods for deconvolving multiperiodic functions. Journal of the Royal Statistical Society B 65 (4), pp. 869–886. External Links: Document Cited by: §4.1.
- An extreme ultraviolet explorer atlas of seyfert galaxy light curves: search for periodicity. The Astrophysical Journal 585, pp. 665–676. External Links: Document Cited by: §5.1, §5.2, §5.3.
- The burst and transient source experiment earth occultation technique. The Astrophysical Journal Supplement Series 138 (1), pp. 149–183. External Links: Document Cited by: §5.2.
- The burst and transient source experiment (batse) earth occultation catalog of low-energy gamma-ray sources. The Astrophysical Journal Supplement Series 154 (2), pp. 585–622. External Links: Document Cited by: §5.2.
- Robust statistics. Vol. 1, John Wiley & Sons. Cited by: §2.2.
- robustreg: robust regression functions. Note: R package version 0.1-9 External Links: Link Cited by: §2.2.2.
- quantreg: quantile regression. Note: R package version 5.21 External Links: Link Cited by: Table 3.
- On searches for pulsed emission with application to four globular cluster x-ray sources: ngc 1851, 6441, 6624, and 6712. The Astrophysical Journal 266 (1), pp. 160–170. External Links: Document Cited by: §2.1.1, Table 2.
- A robust spectral analyzer for one-dimensional and multi-dimensional data analysis. Note: US Patent Application2009/0112954 A1 Cited by: §2.2, Table 2.
- A nonlinear method for robust spectral analysis. IEEE Transactions on Signal Processing 58 (5), pp. 2466–2474. External Links: Document Cited by: §2.2, Table 2.
- Robust statistics: theory and methods. John Wiley & Sons, Chichester. External Links: Document Cited by: §2.2.2, §2.2.2, §2.2.2, §2.2, §2.
- Genetic optimization using derivatives: the rgenoud package for R. Journal of Statistical Software 42 (11), pp. 1–26. External Links: Document Cited by: §2.2.1.
- Sinnbilder für struktogramme nach nassi-shneiderman. Beuth, Berlin, Vienna, Zurich. Cited by: Appendix 0.A.
- Period analysis of variable stars by robust smoothing. Journal of the Royal Statistical Society C 53 (1), pp. 15–30. External Links: Document Cited by: §2.1.3, §2.2, Table 2.
- A fast chi-squared technique for period search of irregularly sampled data. The Astrophysical Journal 695, pp. 496–502. External Links: Document Cited by: §2.1.3, Table 2.
- stats: R statistical functions. Note: R package version 3.0.3, part of R 3.0.3 Cited by: Table 3.
- R: a language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. External Links: Link Cited by: §1.
- base: base R functions. Note: Base R package version 3.2.4 External Links: Link Cited by: item 6.
- SigSpec – i. frequency- and phase-resolved significance in fourier space. Astronomy & Astrophysics 467 (3), pp. 1353–1371. External Links: Document Cited by: §2.1.2, Table 2.
- RSiena: siena – simulation investigation for empirical network analysis. Note: R package version 1.1-232 External Links: Link Cited by: §2.2.2.
- robustbase: basic robust statistics. Note: R package version 0.92-5 External Links: Link Cited by: §2.2.1, §2.2.2, Table 3.
- Robust regression by means of s-estimators. In Robust and Nonlinear Time Series Analysis, J. Franke, W. Härdle, and D. Martin (Eds.), Lecture Notes in Statistics No. 26, pp. 256–272. Cited by: §2.2, §2.2.
- The lomb-scargle periodogram in biological rhythm research: analysis of incomplete and unequally spaced time-series. Biological Rhythm Research 30, pp. 178–201. External Links: Document Cited by: §1.
- The fast- estimator for regression. Journal of Computational and Graphical Statistics 17 (3), pp. 659–682. External Links: Document Cited by: §2.2.4, §2.2.4, §2.2, Table 3.
- A fast algorithm for s-regression estimates. Journal of Computational and Graphical Statistics 15 (2), pp. 414–427. External Links: Document Cited by: §2.2.3, §2.2.3, §2.2, Table 3.
- Studies in astronomical time series analysis. II. statistical aspects of spectral analysis of unevenly spaced data. The Astrophysical Journal 263, pp. 835–853. External Links: Document Cited by: §1, §2.1.2, Table 2.
- On the advantage of using analysis of variance for period search. Monthly Notices of the Royal Astronomical Society 241, pp. 153–165. External Links: Document Cited by: §2.1.1, Table 2.
- The distribution of empirical periodograms: lomb-scargle and pdm spectra. Monthly Notices of the Royal Astronomical Society 301 (3), pp. 831–840. External Links: Document Cited by: §2.1.1, §3.1.
- Linear regression analysis. 2nd edition, John Wiley & Sons, Hoboken, New Jersey. External Links: Document Cited by: §3.1.
- Period determination using phase dispersion minimization. The Astrophysical Journal 224, pp. 953–960. External Links: Document Cited by: §1, §2.1.1, Table 2.
- Periodicity detection in irregularly sampled light curves by robust regression and outlier detection. Statistical Analysis and Data Mining 6 (1), pp. 73–89. External Links: Document Cited by: §1, §1, §2.1.1, §2.1.1, §2.2, Table 2, §2, §2, §3.1, §3.1, §3.2, §4.
- Robuste verfahren zur periodendetektion in ungleichmäßig beobachteten lichtkurven. Doctoral thesis, TU Dortmund University. Cited by: §2.1.1.
- RobPer: robust periodogram and periodicity detection methods. Note: R package version 1.2.1, with contributions from Clarke BR, Ligges U, Salibian-Barrera M, Willems G, Yohai V External Links: Link Cited by: §1.
- On generating power law noise. Astronomy and Astrophysics 300, pp. 707–710. Cited by: §4.3, §4.
- Long-term lightcurves from combined unified very high energy gamma-ray data. Astronomy and Astrophysics 524, pp. A48. External Links: Document Cited by: Figure 1, §4.3, §5.3.
- Measuring the Broad-Band Power Spectra of Active Galactic Nuclei with RXTE. Monthly Notices of the Royal Astronomical Society 332 (1), pp. 231–250. External Links: Document Cited by: §4.3.
- BB: an R package for solving a large system of nonlinear equations and for optimizing a high-dimensional nonlinear objective function. Journal of Statistical Software 32 (4), pp. 1–26. External Links: Document Cited by: §4.1.
- Modern applied statistics with S. 4th edition, Springer-Verlag, New York. External Links: Document Cited by: §2.2.2.
- cts: an R package for continuous time autoregressive models via kalman filter. Journal of Statistical Software 53 (5), pp. 1–19. External Links: Document Cited by: §1.
- High breakdown-point estimates of regression by means of the minimization of an efficient scale. Journal of the American Statistical Association 83 (402), pp. 406–413. External Links: Document Cited by: §2.2.
- The generalised lomb-scargle periodogram. a new formalism for the floating-mean and keplerian periodograms. Astronomy and Astrophysics 496 (2), pp. 577–584. External Links: Document Cited by: §2.1.2, Table 2, §3.1.
- Robust adaptive lomb periodogram for time-frequency analysis of signals with sinusoidal and transient components. In Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP’05), Vol. 4, pp. 493–496. Cited by: §2.1.2, §2.2, Table 2.
Appendix 0.A Implementation diagrams for RobPer
In this appendix, the structure of the RobPer function is displayed as Nassi-Shneiderman diagram (structogram after Norm DIN 66261). Figure 11 contains a reading guidance for the blocks used in the structogram. The structogram for RobPer is displayed in Figure 12, for the algorithm singleFUN in Figure 0.A and for the function IRWLS in Figure 14. The arguments and return values of the latter are shown in Table 5. The following definitions are used:
| (34) | ||||||
| (35) | ||||||
| (36) | ||||||
| (37) | ||||||
| (38) | ||||||
| (39) | ||||||
The normalization constant can be set to due to the scale invariance of the least squares estimation used in the iteratively reweighted least squares (IRWLS) step.
| B1 | |
|---|---|
| B2 | |
| B3 | First run B1, afterwards run B2, at last run B3. Horizontal lines between subsequent blocks are sometimes omitted for better readability. |
| case | |||
|---|---|---|---|
| 1 | 2 | 3 | |
| B1 | B2 | B3 | If case 1, run B1; if case 2, run B2; if case 3, run B3. |
| sub | |
|---|---|
| Run sub (some algorithm, code or function outsourced). |
| condition | ||
|---|---|---|
| block | Reiteration of a block with a check in advance, whether a condition is fulfilled (e.g., a for-loop) | |
| block | ||
|---|---|---|
| condition | Reiteration of a block with a check afterwards, whether a condition is fulfilled (e.g., by if(!...)break) | |
| runifgen | R> eval(parse(text = runifgen)) | ||
| R> t <- tau * 30 | |||
| Sort . | R> t <- sort(t) | ||
| Choose wei randomly from {TRUE, FALSE}. | R> wei <- sample(c(TRUE, FALSE), 1) | ||
| wei | R> if (wei) { | ||
| + eval(parse(text = runifgen)) | |||
| TRUE | FALSE | + s <- tau } | |
| runifgen | R> if(!wei) { | ||
| + s <- rep(0.5, 100) } | |||
| R> y <- numeric(100) | |||
| For | R> for (i in 1:100) { | ||
| Choose from | + y[i] <- rnorm(1, mean = sin(2 * pi / 5 * + t[i]), sd = s) } | ||
Block runifgen:
R> runifgen <- paste("
+ tau <- numeric(100)
+ i <- 1
+ repeat {
Choose from
+ tau[i] <- runif(1)
+ i <- i+1
+ if(!i < 101) break }")
Check arguments, remove incomplete cases
regression
"L2"
"L1"
"LTS"
"bisquare"
"huber"
"S" "tau"
dummy
weighting
FALSE
TRUE
regression
"huber" "bisquare"
"L2" "L1"
"S" "tau"
"LTS"
is the regression estimate in .
is the scale value of the fit to
model {"step", "2step"}
FALSE
TRUE
model
"sine"
"fourier(2)"
"fourier(3)"
"splines"
is the LTS estimate with
trimming )in .
model = "2step"
FALSE
TRUE
design model
design "step"
For ,
singleFUN
model = "2step"
FALSE
TRUE
For ,
singleFUN
Return value: Per(),
Xgen(model, p) with p as the period
Enough independent rows in
FALSE
TRUE
regression {"huber", "bisquare"}
TRUE
FALSE
number of columns of
ltsReg(...,nsamp=50):
is the LTS estimate with trimming in
.
is the regression estimate in
.
var1
regression="LTS"
design {"step", "stepB"}
| Argument | Symbol | Explanation |
|---|---|---|
| yy | yy | Observed values |
| matrix_ | Design matrix | |
| W: | W | Weight function |
| residuals_ | Vector of residuals | |
| scale_ | (Estimate of) Standard deviation | |
| tol | tol | Precision for convergence |
| Return value | ||
| tempIRWLS$coeff | Fitted vector of parameters |
| Return value: | |
|---|---|
Affiliation
Anita M. Thieler Faculty of Statistics TU Dortmund University 44221 Dortmund, Germany E-mail: anita.thieler@tu-dortmund.de URL: http://www.statistik.tu-dortmund.de/thieler-en.html