There is a number of broad data series comparability aspects to address. Many of them depend on data sources conventions and only require simple transformations, others can become more articulated and be based on researchers' decisions. The fact that we work with several sources makes the picture a little more complicated and it is hopefully worth the attempt to openly discuss these matters. Below is an effort in this direction, despite the list is not exhaustive and edge cases may easily come up.
- By convention, we shall convert all relevant return variables to decimal. This should take place preferably right into the parsers/scrapers, as in turn data sources use different conventions and/or different implicit data conversions upon data import may take place.
- Continuously compounded returns are usually preferred for statistical tractability. However, it may be the case returns are provided in simple (discrete) form, as for example with Prof. French's online library. Sources' documentation or underlying data series are needed to reconcile these variables.
- Likewise, we have to deal with different values authors/providers use in place of truly missing data, i.e. unobserved or unregistered data points. In Prof. French's library, for example, such values are indicated by -99.0, -99.99, or -999, depending on the data set at hand. Clearly, if such numeric values go undetected, they will bias the analysis.
- Furthermore, another broad topic to address is time-series dates cross-matching. This aspect is perhaps difficult to consider in a general setting since the very beginning, as we are "exploring" many possible sources each in turn with different sets of variables involved. Tackling such problems on a case-by-case basis may be better for now. Anyways, if one has to use available data series, often little other than resorting to some sort of values interpolation can be done, as measurements reflect corresponding different methodologies. For example, relative to other series, some indexes are rebalanced with a more or less different timing either by convention or as their criteria rely in turn on market and/or economic and/or accounting variables. Additionally, these series can become available at changing points in time or be later corrected. All aspects difficult to precisely know in advance.
The above points are then relevant to the actual data fitting. One practical consideration is that, generally speaking, every sort of "missing value" is treated in R as NA (modulo data types). @braverock pointed out that we should deal with NAs explicitly and, well, he's right and this is also useful to get a grasp on the aforementioned issues. However, provided no fitting routine can work with unavailable observations, the default behavior of most of the ones I know of is to na.omit. But one can always override his own na.action we can pass through. In principle, useRs are supposed to know their own data input and I believe action options freedom is best left in their hands whenever possible.
There is a number of broad data series comparability aspects to address. Many of them depend on data sources conventions and only require simple transformations, others can become more articulated and be based on researchers' decisions. The fact that we work with several sources makes the picture a little more complicated and it is hopefully worth the attempt to openly discuss these matters. Below is an effort in this direction, despite the list is not exhaustive and edge cases may easily come up.
The above points are then relevant to the actual data fitting. One practical consideration is that, generally speaking, every sort of "missing value" is treated in
RasNA(modulo data types). @braverock pointed out that we should deal withNAs explicitly and, well, he's right and this is also useful to get a grasp on the aforementioned issues. However, provided no fitting routine can work with unavailable observations, the default behavior of most of the ones I know of is tona.omit. But one can always override his ownna.actionwe can pass through. In principle, useRs are supposed to know their own data input and I believe action options freedom is best left in their hands whenever possible.