Anglo-French Workshop
In a state?

Although the use of cross-section survey data is now commonplace in applied economics, the evaluation of models based on such data has only very recently attracted the attention it deserves. This new emphasis on model evaluation has occurred almost simultaneously in both France and the UK; it was therefore natural for the Centre to bring together these researchers at a workshop on 13/14 March, organized by ATE Co-director Richard Blundell.

Models based on cross-section data are very different from those involving aggregate time-series data: they rely more heavily on assumptions concerning unobservable attributes or characteristics of individuals in the sample. Policy prescriptions based on estimated cross-section models will only be reliable if the assumptions concerning these unobservable attributes provide a good approximation to reality. The papers presented at the March workshop focussed on simple methods by which the researcher could assess whether these underlying assumptions were reasonable.

In cross-section data, individuals are typically observed to be in different 'states', e.g. employed or unemployed, a purchaster or a non-purchaser of a particular good. An individual is either 'in' or 'out' of a particular state and this form of data is therefore 'discrete' or 'zero-one' in nature. Now suppose we observe an individual to be in a particular state. We can then record their 'level of activity' within this state. This level of activity is a more conventional variable, which can assume a range of values, although it is generally taken to be greater than zero. For example, we can record the number of hours worked for an individual who is in employment, or the level of expenditure of those who are purchasers of a particular good. Cross-section models are distinctive because they combine 'discrete' variables (whether in or out of a particular state) with 'continuous censored' variables (the positive level of activity of those in the state). This requires researchers to specify carefully the factors influencing the probability that an individual is in a particular state and the factors determining the level of activity of individuals in that state. Some assumptions must be made in order to model these probabilities and activity levels, yet these assumptions are often left untested, even though they are are crucial to the policy conclusions which are drawn from the model.

The first paper of the workshop, by Richard Blundell (University College, London, and CEPR) and Costas Meghir (UCL), underlined the importance of these considerations. In 'Bivariate Limited Dependent Variable Models of Individual Labour Supply and Commodity Demand Behaviour', Blundell and Meghir developed a model in which the individual's level of activity once in a particular state was determined separately from their probability of occupying that state. They illustrated this approach using two examples: the first from the labour market, where the state of employment or unemployment was determined separately from the hours worked when in employment; and the second from consumer behaviour, where the decision to buy or not to buy was modelled separately from the decision of how much to buy. Blundell and Meghir discussed simple methods for testing the underlying assumptions of these models. They also presented procedures for testing that the hypothesis that the same behavioural model could explain both the probability of being in a state and the level of activity once in that state.

The discussion focussed on the desirability of model selection procedures which relied on a sequence of 'reliability' tests. The tests in the sequence were calculated from the same data set and were therefore related in a complex fashion. It was therefore difficult to determine the overall properties of the evaluation procedure: papers presented by Alain Trognon and Richard Smith later in the workshop returned to this important issue. Some workshop participants questioned whether, even for the large data sets common in cross-section analysis (often 2000 or more observations), Blundell and Meghir were justified in their use of asymptotic tests.

Francois Bourguignon (Centre d'Economie Quantitative et Comparative, Paris) presented the workshop's second paper, entitled 'Internal Labour Markets: An Econometric Analysis'. Bourguignon used a cross-section of work histories to describe the transition between states in the labour market. He noted that analyses of work histories are often based on the records of an individual firm and are therefore unreliable: individuals leave the firm, and the investigator loses track of their work history. Those who leave the firm are usually not representative of all employees, and the data available from those who remain are not a random sample of the firm's workforce: it is therefore difficult to retrieve an unbiased picture of transition probabilities. The data used by Bourguignon and his co-author Pierre-Andre Chiappori were drawn, however, from the records of a large single firm. Work histories were relatively complete, because few individuals left the firm and transitions occurred almost wholly within the firm, which constituted an almost perfect internal labour market. Bourguignon and Chiappori modelled the probability of an individual being observed at different 'states' or stages of seniority within the firm. The transition probabilities, that is, the probabilities of promotion were influenced by both individual and career characteristics. Workshop participants noted that fluctuations in the demand for labour, assumed constant in Bourguignon's model, could have led to biased estimates of the effects of age and length of service on promotion.

The random components in many cross-section models are assumed to behave as if drawn from a normal distribution. Papers presented by Manuel Arrellano (Institute of Economics and Statistics, Oxford) and Tim Fry (Manchester) proposed interesting alternatives to the normality assumption. In his paper, 'Estimating Contaminated Limited Dependent Variable Models', Arrellano analyzed models in which the random components were formed from a mixture of normal errors with different variances. This distribution offered a computationally simple alternative to the usual assumption of normality, Arrellano argued. As a by- product, it also led to a simple test for the appropriateness of the conventional normality assumption.

Almost every conventional evaluation procedure for econometric models is based on an analysis of the model's residuals, the differences between the predictions of the model and the observed data. The papers presented by Alain Trognon (Institut National de la Statistique et des Etudes Economiques, Paris) and Andrew Chesher (Bristol and CEPR) attempted to develop the theory of residuals analysis for cross-section models. The combination of discrete and continuous variables makes such models very different from their time-series counterparts: residuals cannot be analysed in the conventional time-series fashion, and new approaches are needed. Trognon and Chesher developed the notion of 'generalized residuals' whose statistical properties are relatively simple to establish and which can therefore serve as the basis for tests of model 'reliability'. These generalized residuals are defined as the expectations of the underlying or 'latent' residuals, conditional upon the observed data and the structure of the model.

Trognon's paper, entitled 'Using Simulated and Generalized Residuals' and written jointly with Alain Monfort, Christian Gourieroux and Eric Renault (INSEE), developed a general theory of such residuals which could be applied to many of the commonly used cross-section models. Generalized residuals could be used, for example, to test the assumptions that the random components were distributed normally, with a constant variance and independently of each other. The investigator would not need to undertake further estimation of the model in order to test these assumptions. Trognon discussed an alternative procedure, in which, given the parameters of an estimated model, simulated data can be generated and simulated residuals calculated. Trognon showed how these simulated residuals can then be used to plot residuals and to calculate model reliability tests in much the same way as residuals are used conventionally. In the discussion of the paper it was pointed out that this simulation procedure did not make model estimation easier, but only simplified the formulae for specification tests. Since these tests depended on simulated residuals, this simplification was at the expense of introducing additional 'noise' by virtue of the simulation itself.

The problems of using 'generalized residuals' were further outlined by Andrew Chesher (Bristol and CEPR) in his paper 'Graphical Analysis of Censored Data', written jointly with Margaret Irish (Bristol). Generalized residuals do not possess the same statistical distribution as conventional residuals: their analysis is therefore somewhat more complicated. Chesher illustrated the problems involved by considering the omission of variables from a cross-section model explaining the duration of unemployment. Variables that are correctly omitted from the model can appear as if they belong in the model, if correlated with variables which are included in the regression. To overcome this problem and to make plots of 'generalized residuals' easier to interpret, Chesher introduced a method of smoothing the residuals which provided a clearer picture of potential misspecification.

Blundell and Meghir had argued in their opening paper of the workshop that many cross-section models involved random components which were best analysed as mixtures of discrete and continuous dependent variables. Richard Smith (Manchester) in his paper, 'Testing the Normality Assumption in Multivariate Simultaneous Limited Dependent Variable Models', described a methodology for testing a key assumption underlying such models. Smith's test statistic involved higher order moments of the generalized residuals and bore a close resemblance to the popular tests based on the Information Matrix. The subsequent discussion focussed on whether this result held generally for all cross- section models. It was argued that Smith's exact correspondence between the 'generalized residual' and Information Matrix tests disappears in models where there is selection bias, i.e. where data are not observed for some part of the underlying population. Participants concluded that the elements of tests based on 'generalized residuals' were always present in the Information Matrix test. The choice of test statistic therefore depended on whether it was non-normality or some broader alternative, like parameter variation, which was thought to be the potential source of misspecification.

When dealing with large data sets and large numbers of decision variables, the ability to group decisions in some hierarchical ordering is crucial for tractability in estimation. Francois Laisney (Toulouse), in his paper 'A Method for the Description of Separability Properties of Large Empirical Demand Systems', described a simple algorithm for grouping consumer decisions in such a hierarchy. This procedure was applied to a model of consumer behaviour in France with some success. However, it was concluded that the resulting grouping depends rather heavily on the broad assumptions about preferences or technology that are made at the outset.

Conventional econometric theory is based on the estimation of parameters, that is, numbers which characterise particular probability distributions: the normal distribution, for example is characterised by its mean and variance. The disadvantage of such parametric techniques is their reliance on the assumed form of the underlying distribution; other techniques have been devised which avoid parameter estimation and its reliance on distributional assumptions. In the final paper of the workshop, Tony Lancaster (Hull) described the properties of a 'semi- parametric' model for cross-section data on unemployment durations. Although Lancaster's model did not need strong distributional assumptions, the test statistics he had developed behaved quite poorly even for the large samples typical of cross- section data. This confirmed what many participants feared: that non-parametric modelling could result in a considerable loss of efficiency in estimation and testing. But for the particular test that Lancaster was considering, a transformation was suggested that seemed at first sight to improve the properties of the test considerably.

As a result of this workshop, there was general agreement that a useful set of procedures for assessing the adequacy of cross- section models was now available. But the properties of these test statistics, when applied to the data sets typically used in applied microeconomics, still remained somewhat uncertain and required further research.