Interval Regression with Sample Selection

Arne Henningsen    Sebastian Petersen    Géraldine Henningsen

This ‘vignette’ is largely based on Petersen, Henningsen and Henningsen (2017).

1 Model Specification

The general specification of an interval regression model with sample selection is:

yiS∗ =𝜷S′⁢𝒙iS+εiS (1)
yiS ={0if yiS∗≤01otherwise (2)
yiO∗ =𝜷O′⁢𝒙iO+εiO (3)
yiO ={unknownif ⁢yiS=01if α1<yiO∗≤α2 and yiS=12if α2<yiO∗≤α3 and yiS=1Mif αM<yiO∗≤αM+1 and yiS=1 (4)
(εiSεiO) ∼N2⁢((00),[1ρ⁢σρ⁢σσ2]),

where subscript i indicates the observation, yiO∗ is a latent outcome variable, yiO is a partially observed categorical variable that indicates in which interval yiO∗ lies, M is the number of intervals, α1,…,αM+1 are the boundaries of the intervals (whereas frequently but not necessarily α1=−∞ and αM+1=∞), yiS is a binary variable that indicates whether yiO is observed, yiS∗ is a latent variable that indicates the “tendency” that yiS is one, 𝒙iS and 𝒙iO are (column) vectors of explanatory variables for the selection equation and outcome equation, respectively, εiS and εiO are random disturbance terms that have a joint bivariate normal distribution, and 𝜷S and 𝜷O are (column) vectors and ρ and σ are scalars of unknown model parameters.

2 Log-Likelihood Function

The probability that yiO is unobserved is:

P⁡(yiS=0) =P(yiS∗≤0) (9)
=P⁡(𝜷S′⁢𝒙iS+εiS≤0) (10)
=P⁡(εiS≤−𝜷S′⁢𝒙iS) (11)

The probability that yiO is observed and indicates that yiO∗ lies in the mth interval is:

P⁡(yiS=1∧yiO=m) =P(yiS∗>0∧αm<yiO∗≤αm+1) (12)
=P⁡(𝜷S′⁢𝒙iS+εiS>0∧αm<𝜷O′⁢𝒙iO+εiO≤αm+1) (13)
=P⁡(εiS>−𝜷S′⁢𝒙iS∧αm−𝜷O′⁢𝒙iO<εiO≤αm+1−𝜷O′⁢𝒙iO) (14)

The log-likelihood contribution of the ith observation is:

ℓi= (1−yiS)⁢ln⁡[Φ⁡(−𝜷S′⁢𝒙iS)] (15)
+∑m=1MyiS(yiO=m)ln[Φ2(αm+1−𝜷O′⁢𝒙iOσ,𝜷S′𝒙iS,−ρ)
−Φ2(αm−𝜷O′⁢𝒙iOσ,𝜷S′𝒙iS,−ρ)],

where Φ(.) indicates the cumulative distribution function of the univariate standard normal distribution and Φ2(.) indicates the cumulative distribution function of the bivariate standard normal distribution.

3 Restricting coefficients ρ and σ2

The parameter ρ needs to be in the interval (−1,1). In order to restrict ρ to be in this interval, we estimate arctan⁡(ρ) instead of ρ so that the derived parameter ρ=tan⁡(arctan⁡(ρ)) is always in the interval (−1,1). We use the delta method to calculate approximate standard errors of the derived parameter ρ, whereas the corresponding element of the Jacobian matrix is:

∂tan⁡(arctan⁡(ρ))∂arctan⁡(ρ)=∂ρ∂arctan⁡(ρ)=(1+ρ2) (16)

The parameter σ needs to be strictly positive, i.e. σ>0. In order to restrict σ to be strictly positive, we estimate log⁡(σ) instead of σ or σ2 so that the derived parameters σ=exp⁡(log⁡(σ)) and σ2=exp⁡(2⁢log⁡(σ)) are always strictly positive. We use the delta method to calculate approximate standard errors of the derived parameters σ and σ2, whereas the corresponding elements of the Jacobian matrix are:

∂exp⁡(log⁡(σ))∂log⁡(σ) =exp⁡(log⁡(σ))=σ (17)
∂exp⁡(2⁢log⁡(σ))∂log⁡(σ) =2⁢exp⁡(2⁢log⁡(σ))=2⁢σ2 (18)

4 Gradients of the CDF of the bivariate standard normal distribution

In order to facilitate the calculation of the gradients of the log-likelihood function, we calculate the partial derivatives of the cumulative distribution function (CDF) of the bivariate standard normal distribution:

Φ2⁢(x1,x2,ρ) =∫−∞x2∫−∞x1ϕ2⁢(a1,a2,ρ)⁢d⁢a1⁢d⁢a2, (19)

where ϕ2(.) is the probability density function (PDF) of the bivariate standard normal distribution:

ϕ2⁢(x1,x2,ρ) =12⁢π⁢1−ρ2⋅exp⁡(−x12−2⁢ρ⁢x1⁢x2+x222⁢(1−ρ2)) (20)

In the following, we check equation (20) by a simple numerical example:

> library( "mvtnorm" )> library( "maxLik" )> x1 <- 0.4> x2 <- -0.3> rho <- -0.6> sigma <- matrix( c( 1, rho, rho, 1 ), nrow = 2 )> dens <- dmvnorm( c( x1, x2 ), sigma = sigma )> print( dens )

[1] 0.1831324

> all.equal( dens, ( 2 * pi * sqrt( 1 - rho^2 ) )^(-1) *+ exp( - ( x1^2 - 2 * rho * x1 * x2 + x2^2 ) / ( 2 * ( 1 - rho^2 ) ) ) )

[1] TRUE

4.1 Gradients with respect to the limits (x1 and x2)

∂Φ2⁢(x1,x2,ρ)∂x2 =∫−∞x1ϕ2⁢(a1,x2,ρ)⁢d⁢a1 (21)
=∫−∞x1ϕ⁡(a1|x2,ρ)⁢ϕ⁢(x2)⁢d⁢a1 (22)
=∫−∞x1ϕ~⁢(a1,ρ⁢x2,1−ρ2)⁢ϕ⁢(x2)⁢d⁢a1 (23)
=∫−∞x1ϕ⁡(a1−ρ⁢x21−ρ2)⁢(1−ρ2)−1⁢ϕ⁢(x2)⁢d⁢a1 (24)
=∫−∞x1ϕ⁡(a1−ρ⁢x21−ρ2)⁢(1−ρ2)−1⁢d⁢a1⁢ϕ⁢(x2) (25)
=∫−∞x1−ρ⁢x21−ρ2ϕ⁡(a1)⁢d⁢a1⁢ϕ⁢(x2) (26)
=Φ⁡(x1−ρ⁢x21−ρ2)⁢ϕ⁢(x2), (27)

where ϕ~(,μ,σ2) indicates the density function of a normal distribution with mean μ and variance σ2.

In the following, we use the same simple numerical example as in the beginning of section 4 to check the above derivations. First, we check whether the PDF of the bivariate standard normal distribution, i.e. ϕ2⁢(x1,x2,ρ) (part of equation 21), is equal to ϕ~⁢(x1,ρ⁢x2,1−ρ2)⁢ϕ⁢(x2) (part of equation 23) and equal to ϕ⁡((x1−ρ⁢x2)/(1−ρ2)) (1−ρ2)−1⁢ϕ⁢(x2) (part of equations 24 and 25):

> all.equal( dens, dnorm( x1, rho * x2, sqrt( 1 - rho^2 ) ) * dnorm(x2) )

[1] TRUE

> all.equal( dens, ( dnorm( ( x1 - rho * x2 ) / sqrt( 1 - rho^2 ) ) /+ sqrt( 1 - rho^2 ) ) * dnorm(x2) )

[1] TRUE

In the following, we will numerically calculate the derivative of the cumulative distribution function of the bivaraite normal distribution (equation 19) with respect to x2 and check wehther this partial derivative is equal to the right-hand sides of equations (21), (24), (25), and (27):

> funX2 <- function( a2 ) {+ prob <- pmvnorm( upper = c( x1, a2 ), sigma = sigma )+ return( prob )+ }> grad <- c( numericGradient( funX2, x2 ) )> print( grad )

[1] 0.2320142

> funX1 <- function( a1 ) {+ dens <- rep( NA, length( a1 ) )+ for( i in 1:length( a1 ) ) {+ dens[i] <- dmvnorm( c( a1[i], x2 ), sigma = sigma )+ }+ return( dens )+ }> all.equal( grad, integrate( funX1, lower = -Inf, upper = x1 )$value )

[1] TRUE

> funX1a <- function( a1 ) {+ dens <- rep( NA, length( a1 ) )+ for( i in 1:length( a1 ) ) {+ dens[i] <- ( dnorm( ( a1[i] - rho * x2 ) / sqrt( 1 - rho^2 ) ) /+ sqrt(1-rho^2) ) * dnorm(x2)+ }+ return( dens )+ }> all.equal( grad, integrate( funX1a, lower = -Inf, upper = x1 )$value )

[1] TRUE

> funX1b <- function( a1 ) {+ dens <- rep( NA, length( a1 ) )+ for( i in 1:length( a1 ) ) {+ dens[i] <- dnorm( ( a1[i] - rho * x2 ) / sqrt( 1 - rho^2 ) ) /+ sqrt(1-rho^2)+ }+ return( dens )+ }> all.equal( grad,+ integrate( funX1b, lower = -Inf, upper = x1 )$value * dnorm(x2) )

[1] TRUE

> all.equal( grad,+ pnorm( ( x1 - rho * x2 ) / sqrt( 1 - rho^2 ) ) * dnorm( x2 ) )

[1] TRUE

4.2 Gradients with respect to the coefficient of correlation (ρ)

∂Φ2⁢(x1,x2,ρ)∂ρ (28)
= ∂[∫−∞x1∫−∞x2ϕ2⁢(a1,a2,ρ)⁢d⁢a2⁢d⁢a1]∂ρ (29)
= ∫−∞x1∫−∞x2∂ϕ2⁢(a1,a2,ρ)∂ρ⁢d⁢a2⁢d⁢a1 (30)
= ∫−∞x1∫−∞x2∂∂ρ⁢(exp⁡(−a12−2⁢ρ⁢a1⁢a2+a222⁢(1−ρ2))2⁢π⁢1−ρ2)⁢d⁢a2⁢d⁢a1 (31)
= ∫−∞x1∫−∞x212⁢π⁢∂∂ρ⁢(exp⁡(−a12−2⁢ρ⁢a1⁢a2+a222⁢(1−ρ2))1−ρ2)⁢d⁢a2⁢d⁢a1 (32)
= ∫−∞x1∫−∞x212⁢π⁢(∂∂ρ⁢(exp⁡(−a12−2⁢ρ⁢a1⁢a2+a222⁢(1−ρ2)))⋅1−ρ21−ρ2CLOSE (33)
OPEN−∂∂ρ⁢(1−ρ2)⋅exp⁡(−a12−2⁢ρ⁢a1⁢a2+a222⁢(1−ρ2))1−ρ2)⁢d⁢a2⁢d⁢a1
= ∫−∞x1∫−∞x212⁢π⁢(∂∂ρ⁢(−a12−2⁢ρ⁢a1⁢a2+a222⁢(1−ρ2))⋅exp⁡(−a12−2⁢ρ⁢a1⁢a2+a222⁢(1−ρ2))⋅1−ρ21−ρ2CLOSE (34)
OPEN−(−ρ1−ρ2)⋅exp⁡(−a12−2⁢ρ⁢a1⁢a2+a222⁢(1−ρ2))1−ρ2)⁢d⁢a2⁢d⁢a1
= ∫−∞x1∫−∞x212⁢π⁢((−4⁢ρ⁢(a12−2⁢ρ⁢a1⁢a2+a22)−2⁢(1−ρ2)⁢(−2⁢a1⁢a2))4⁢(1−ρ2)2⋅exp⁡(−a12−2⁢ρ⁢a1⁢a2+a222⁢(1−ρ2))⋅1−ρ21−ρ2CLOSE (35)
OPEN−(−ρ1−ρ2)⋅exp⁡(−a12−2⁢ρ⁢a1⁢a2+a222⁢(1−ρ2))1−ρ2)⁢d⁢a2⁢d⁢a1
= ∫−∞x1∫−∞x212⁢π⁢((−4⁢ρ⁢(a12−2⁢ρ⁢a1⁢a2+a22)−2⁢(1−ρ2)⁢(−2⁢a1⁢a2))4⁢(1−ρ2)2⋅1−ρ21−ρ2−(−ρ1−ρ2)1−ρ2) (36)
⋅exp⁡(−a12−2⁢ρ⁢a1⁢a2+a222⁢(1−ρ2))⁢d⁢a2⁢d⁢a1
= ∫−∞x1∫−∞x212⁢π⁢((−4⁢ρ⁢(a12−2⁢ρ⁢a1⁢a2+a22)−2⁢(1−ρ2)⁢(−2⁢a1⁢a2))4⁢(1−ρ2)53+ρ(1−ρ2)32) (37)
⋅exp⁡(−a12−2⁢ρ⁢a1⁢a2+a222⁢(1−ρ2))⁢d⁢a2⁢d⁢a1
= ∫−∞x1∫−∞x212⁢π⁢((−4⁢ρ⁢(a12−2⁢ρ⁢a1⁢a2+a22)−2⁢(1−ρ2)⁢(−2⁢a1⁢a2))4⁢(1−ρ2)52+ρ(1−ρ2)32) (38)
⋅exp⁡(−a12−2⁢ρ⁢a1⁢a2+a222⁢(1−ρ2))⁢d⁢a2⁢d⁢a1
= ∫−∞x1∫−∞x212⁢π⁢(ρ(1−ρ2)32−ρ⁡(a12−ρ⁢a1⁢a2+a22)−a1⁢a2(1−ρ2)52) (39)
⋅exp⁡(−a12−2⁢ρ⁢a1⁢a2+a222⁢(1−ρ2))⁢d⁢a2⁢d⁢a1
= 12⁢π⁢(1−ρ2)⁢∫−∞x1∫−∞x2(ρ1−ρ2−ρ⁡(a12−ρ⁢a1⁢a2+a22)−a1⁢a2(1−ρ2)2) (40)
⋅exp⁡(−a12−2⁢ρ⁢a1⁢a2+a222⁢(1−ρ2))⁢d⁢a2⁢d⁢a1
= 12⁢π⁢(1−ρ2)⁢∫−∞x1|(−2⁢a1−2⁢ρ⁢a22⁢(1−ρ2))⋅exp⁡(−a12−2⁢ρ⁢a1⁢a2+a222⁢(1−ρ2))|−∞x2⁢d⁢a1 (41)
= 12⁢π⁢(1−ρ2)⁢∫−∞x1((−2⁢a1−2⁢ρ⁢x22⁢(1−ρ2))⋅exp⁡(−a12−2⁢ρ⁢a1⁢x2+x222⁢(1−ρ2))CLOSE (42)
−lima2→−∞12⁢(1−ρ2)−2⁢a1+2⁢ρ⁢a2exp⁡(a12−2⁢ρ⁢a1⁢a2+a222⁢(1−ρ2)))da1

Applying L’Hospital on the last term leads to

= 12⁢π⁢(1−ρ2)⁢∫−∞x1((−2⁢a1−2⁢ρ⁢x22⁢(1−ρ2))⋅exp⁡(−a12−2⁢ρ⁢a1⁢x2+x222⁢(1−ρ2))−0)⁢d⁢a1 (43)
= 12⁢π⁢(1−ρ2)⁢∫−∞x1(−2⁢a1−2⁢ρ⁢x22⁢(1−ρ2))⋅exp⁡(−a12−2⁢ρ⁢a1⁢x2+x222⁢(1−ρ2))⁢d⁢a1 (44)
= 12⁢π⁢(1−ρ2)⁢|exp⁡(−a12−2⁢ρ⁢a1⁢x2+x222⁢(1−ρ2))|−∞x1 (45)
= 12⁢π⁢(1−ρ2)⋅exp⁡(−x12−2⁢ρ⁢x1⁢x2+x222⁢(1−ρ2)) (46)
= ϕ2⁢(x1,x2,ρ) (47)

This result is in line with Sibuya (1960) and Sungur (1990).

In the following, we will numerically calculate the derivative of the cumulative distribution function of the bivariate normal distribution (equation 29) with respect to ρ and check whether this partial derivative is equal to the right-hand sides of equation (47):

> # Numerical gradient of the PDF w.r.t. rho> funrho <- function( p ) {+ prob <- dmvnorm( x = c( x1, x2 ),+ sigma = matrix( c( 1, p, p, 1 ), nrow = 2 ) )+ return( prob )+ }> grad <- c( numericGradient( funrho, rho ) )> print( grad )

[1] -0.1775883

> # Comparison with analytical gradient for rho> efun <- exp(-(x1^2 - 2 * rho * x1 * x2 + x2^2)/(2*(1 - rho^2)))> all.equal( grad,+ ( (-((2*rho*(-2*rho*x1*x2+x1^2+x2^2) - 2*x1*x2*(1-rho^2)) * efun)/+ (2*(1-rho^2)^(3/2) )) ++ ((rho*efun)/(sqrt(1-rho^2))) ) /+ (2*pi*(1-rho^2)) )

[1] TRUE

[1] TRUE

[1] TRUE

[1] TRUE

> # Numerical gradient of the CDF w.r.t. rho> cdfRho <- function( p, xa = x1, xb = x2 ) {+ prob <- pmvnorm( upper = c( xa, xb ),+ sigma = matrix( c( 1, p, p, 1 ), nrow = 2 ) )+ return( prob )+ }> grad <- c( numericGradient( cdfRho, rho ) )> print( grad )

[1] 0.1831324

> # comparison with analytical gradient> all.equal( grad, dmvnorm( x = c( x1, x2 ),+ sigma = matrix( c( 1, rho, rho, 1 ), nrow = 2 ) ) )

[1] TRUE

> # comparisons with other values> compDerivRho <- function( xa, xb, p ) {+ dn <- c( numericGradient( cdfRho, p, xa = xa, xb = xb ) )+ da <- dmvnorm( x = c( xa, xb ),+ sigma = matrix( c( 1, p, p, 1 ), nrow = 2 ) )+ return( all.equal( dn, da ) )+ }> compDerivRho( x1, x2, rho )

[1] TRUE

> compDerivRho( 0.5, x2, rho )

[1] TRUE

> compDerivRho( 2.5, x2, rho )

[1] TRUE

> compDerivRho( x1, -2, rho )

[1] TRUE

> compDerivRho( x1, x2, 0.2 )

[1] TRUE

> compDerivRho( x1, x2, 0.98 )

[1] TRUE

5 Gradients of the Log-Likelihood Function

5.1 Gradients with respect to the parameters of the selection equation (𝜷S)

First, we use equation (27), to determine the derivative of the bivariate standard normal distribution with respect to the parameter 𝜷S as part of the loglikelihood function:

∂Φ2⁢(αm−𝜷O′⁢𝒙iOσ,𝜷S′⁢𝒙iS,−ρ)∂βS=Φ⁡(αm−𝜷O′⁢𝒙iOσ−ρ⁢𝜷S′⁢𝒙iS1−ρ2)⁢ϕ⁢(𝜷S′⁢𝒙iS)⋅∂𝜷S′⁢𝒙iS∂𝜷S (48)
=Φ⁡(αm−𝜷O′⁢𝒙iOσ+ρ⁢𝜷S′⁢𝒙iS1−ρ2)⁢ϕ⁢(𝜷S′⁢𝒙iS)⋅𝒙iS (49)

Using this result we can now derive the gradient for 𝜷S in the log-likelihood function:

∂ℓi∂βS=∂∂βS⁢((1−yiS)⁢ln⁡[Φ⁡(−𝜷S′⁢𝒙iS)]CLOSE (50)
+∑m=1MyiS(yiO=m)ln[Φ2(αm+1−𝜷O′⁢𝒙iOσ,𝜷S′𝒙iS,−ρ)
−Φ2(αm−𝜷O′⁢𝒙iOσ,𝜷S′𝒙iS,−ρ)])
=(1−yiS)⁢∂∂βS⁢(ln⁡[Φ⁡(−𝜷S′⁢𝒙iS)]) (51)
+∑m=1MyiS(yiO=m)∂∂βS(ln[Φ2(αm+1−𝜷O′⁢𝒙iOσ,𝜷S′𝒙iS,−ρ)
−Φ2(αm−𝜷O′⁢𝒙iOσ,𝜷S′𝒙iS,−ρ)])
=(1−yiS)⁢ϕ⁡(−𝜷S′⁢𝒙iS)⋅(−𝒙iS)Φ⁡(−𝜷S′⁢𝒙iS) (52)
+∑m=1MyiS(yiO=m)∂Φ2⁢(αm+1−𝜷O′⁢𝒙iOσ,𝜷S′⁢𝒙iS,−ρ)∂βS−∂Φ2⁢(αm−𝜷O′⁢𝒙iOσ,𝜷S′⁢𝒙iS,−ρ)∂βSΦ2⁢(αm+1−𝜷O′⁢𝒙iOσ,𝜷S′⁢𝒙iS,−ρ)−Φ2⁢(αm−𝜷O′⁢𝒙iOσ,𝜷S′⁢𝒙iS,−ρ)
=(1−yiS)⁢ϕ⁡(−𝜷S′⁢𝒙iS)⋅(−𝒙iS)Φ⁡(−𝜷S′⁢𝒙iS) (53)
+∑m=1MyiS(yiO=m)1Φ2⁢(αm+1−𝜷O′⁢𝒙iOσ,𝜷S′⁢𝒙iS,−ρ)−Φ2⁢(αm−𝜷O′⁢𝒙iOσ,𝜷S′⁢𝒙iS,−ρ)
(Φ⁡(αm+1−𝜷O′⁢𝒙iOσ+ρ⁢𝜷S′⁢𝒙iS1−ρ2)⁢ϕ⁢(𝜷S′⁢𝒙iS)⋅𝒙iSCLOSE
−Φ(OPENαm−𝜷O′⁢𝒙iOσ+ρ⁢𝜷S′⁢𝒙iS)1−ρ2)ϕ(𝜷S′𝒙iS)⋅𝒙iS)
=(1−yiS)⁢ϕ⁡(−𝜷S′⁢𝒙iS)⋅(−𝒙iS)Φ⁡(−𝜷S′⁢𝒙iS) (54)
+∑m=1MyiS(yiO=m)(Φ⁡(αm+1−𝜷O′⁢𝒙iOσ+ρ⁢𝜷S′⁢𝒙iS1−ρ2)−Φ⁡(αm−𝜷O′⁢𝒙iOσ+ρ⁢𝜷S′⁢𝒙iS1−ρ2))⁢ϕ⁢(𝜷S′⁢𝒙iS)⋅𝒙iSΦ2⁢(αm+1−𝜷O′⁢𝒙iOσ,𝜷S′⁢𝒙iS,−ρ)−Φ2⁢(αm−𝜷O′⁢𝒙iOσ,𝜷S′⁢𝒙iS,−ρ)

5.2 Gradients with respect to the parameters in the outcome equation (𝜷O)

Analogous to 𝜷S and by using equation (27) we derive the gradient of 𝜷O:

∂Φ2⁢(αm−𝜷O′⁢𝒙iOσ,𝜷S′⁢𝒙iS,−ρ)∂βO=Φ⁡(𝜷S′⁢𝒙iS+ρ⁢αm−𝜷O′⁢𝒙iOσ1−ρ2)⁢ϕ⁢(αm−𝜷O′⁢𝒙iOσ)⋅(−𝒙iOσ) (55)

Using this result we derive the gradient for the outcome parameter 𝜷O for the log-likelihood function:

∂ℓi∂βO =∂∂βO⁢((1−yiS)⁢ln⁡[Φ⁡(−𝜷S′⁢𝒙iS)]CLOSE (56)
+∑m=1MyiS(yiO=m)ln[Φ2(αm+1−𝜷O′⁢𝒙iOσ,𝜷S′𝒙iS,−ρ)
−Φ2(αm−𝜷O′⁢𝒙iOσ,𝜷S′𝒙iS,−ρ)])
=∑m=1MyiS(yiO=m)∂∂βO(ln[Φ2(αm+1−𝜷O′⁢𝒙iOσ,𝜷S′𝒙iS,−ρ) (57)
−Φ2(αm−𝜷O′⁢𝒙iOσ,𝜷S′𝒙iS,−ρ)])
=∑m=1MyiS⁢(yiO=m)⁢∂Φ2⁢(αm+1−𝜷O′⁢𝒙iOσ,𝜷S′⁢𝒙iS,−ρ)∂βO−∂Φ2⁢(αm−𝜷O′⁢𝒙iOσ,𝜷S′⁢𝒙iS,−ρ)∂βOΦ2⁢(αm+1−𝜷O′⁢𝒙iOσ,𝜷S′⁢𝒙iS,−ρ)−Φ2⁢(αm−𝜷O′⁢𝒙iOσ,𝜷S′⁢𝒙iS,−ρ) (58)
=∑m=1MyiS⁢(yiO=m)⋅1Φ2⁢(αm+1−𝜷O′⁢𝒙iOσ,𝜷S′⁢𝒙iS,−ρ)−Φ2⁢(αm−𝜷O′⁢𝒙iOσ,𝜷S′⁢𝒙iS,−ρ) (59)
(Φ⁡(𝜷S′⁢𝒙iS+ρ⁢αm+1−𝜷O′⁢𝒙iOσ1−ρ2)⁢ϕ⁢(αm+1−𝜷O′⁢𝒙iOσ)⋅(−𝒙iOσ)CLOSE
−Φ(𝜷S′⁢𝒙iS+ρ⁢αm−𝜷O′⁢𝒙iOσ1−ρ2)ϕ(αm−𝜷O′⁢𝒙iOσ)⋅(−𝒙iOσ))
=∑m=1MyiS⁢(yiO=m)⋅1Φ2⁢(αm+1−𝜷O′⁢𝒙iOσ,𝜷S′⁢𝒙iS,−ρ)−Φ2⁢(αm−𝜷O′⁢𝒙iOσ,𝜷S′⁢𝒙iS,−ρ) (60)
(Φ⁡(𝜷S′⁢𝒙iS+ρ⁢αm+1−𝜷O′⁢𝒙iOσ1−ρ2)⁢ϕ⁢(αm+1−𝜷O′⁢𝒙iOσ)CLOSE
OPEN−Φ⁡(𝜷S′⁢𝒙iS+ρ⁢αm−𝜷O′⁢𝒙iOσ1−ρ2)⁢ϕ⁢(αm−𝜷O′⁢𝒙iOσ))⋅(−𝒙iOσ)

5.3 Gradients with respect to the coefficient of correlation (ρ)

Given the result that the derivative of the CDF with respect to ρ is equal to the PDF (see equation 47), we can also derive the gradient of the correlation parameter (ρ):

∂ℓi∂ρ=∂∂ρ⁢((1−yiS)⁢ln⁡[Φ⁡(−𝜷S′⁢𝒙iS)]CLOSE (61)
+∑m=1MyiS(yiO=m)ln[Φ2(αm+1−𝜷O′⁢𝒙iOσ,𝜷S′𝒙iS,−ρ)
−Φ2(αm−𝜷O′⁢𝒙iOσ,𝜷S′𝒙iS,−ρ)])
=∑m=1MyiS(yiO=m)∂∂ρ(ln[Φ2(αm+1−𝜷O′⁢𝒙iOσ,𝜷S′𝒙iS,−ρ) (62)
−Φ2(αm−𝜷O′⁢𝒙iOσ,𝜷S′𝒙iS,−ρ)])
=∑m=1MyiS⁢(yiO=m)⁢ϕ2⁢(αm−𝜷O′⁢𝒙iOσ,𝜷S′⁢𝒙iS,−ρ)−ϕ2⁢(αm+1−𝜷O′⁢𝒙iOσ,𝜷S′⁢𝒙iS,−ρ)Φ2⁢(αm+1−𝜷O′⁢𝒙iOσ,𝜷S′⁢𝒙iS,−ρ)−Φ2⁢(αm−𝜷O′⁢𝒙iOσ,𝜷S′⁢𝒙iS,−ρ) (63)
∂ℓi∂arctanh⁡(ρ) =∂ℓi∂ρ⁢∂ρ∂arctanh⁡(ρ)=∂ℓi∂ρ⁢(1−ρ2) (64)

5.4 Gradients with respect to the standard deviation used for normalisation (σ)

Finally, we derive the gradient for σ in the same way as we did for βS and βO:

∂Φ2⁢(αm−𝜷O′⁢𝒙iOσ,𝜷S′⁢𝒙iS,−ρ)∂σ
=Φ⁡(𝜷S′⁢𝒙iS+ρ⁢αm−𝜷O′⁢𝒙iOσ1−ρ2)⁢ϕ⁢(αm−𝜷O′⁢𝒙iOσ)⋅𝜷O′⁢𝒙iO−αmσ2 (65)
limαm→∞∂Φ2⁢(αm−𝜷O′⁢𝒙iOσ,𝜷S′⁢𝒙iS,−ρ)∂σ
=limαm→∞Φ⁡(𝜷S′⁢𝒙iS+ρ⁢αm−𝜷O′⁢𝒙iOσ1−ρ2)⁢ϕ⁢(αm−𝜷O′⁢𝒙iOσ)⋅𝜷O′⁢𝒙iO−αmσ2 (66)
=Φ⁡(𝜷S′⁢𝒙iS+ρ⁢αm−𝜷O′⁢𝒙iOσ1−ρ2)⁢limαm→∞ϕ⁡(αm−𝜷O′⁢𝒙iOσ)⋅𝜷O′⁢𝒙iO−αmσ2 (67)
=Φ⁡(𝜷S′⁢𝒙iS+ρ⁢αm−𝜷O′⁢𝒙iOσ1−ρ2)⁢limαm→∞𝜷O′⁢𝒙iO−αmσ2(ϕ⁡(αm−𝜷O′⁢𝒙iOσ))−1 (68)
=Φ⁡(𝜷S′⁢𝒙iS+ρ⁢αm−𝜷O′⁢𝒙iOσ1−ρ2) (69)
limαm→∞−1σ2−(ϕ⁡(αm−𝜷O′⁢𝒙iOσ))−2⁢(−αm−𝜷O′⁢𝒙iOσ)⁢ϕ⁢(αm−𝜷O′⁢𝒙iOσ)⁢1σ
=Φ⁡(𝜷S′⁢𝒙iS+ρ⁢αm−𝜷O′⁢𝒙iOσ1−ρ2)⁢limαm→∞−1σ(ϕ⁡(αm−𝜷O′⁢𝒙iOσ))−1⁢αm−𝜷O′⁢𝒙iOσ (70)
=Φ⁡(𝜷S′⁢𝒙iS+ρ⁢αm−𝜷O′⁢𝒙iOσ1−ρ2)⁢limαm→∞−1σ⁢ϕ⁢(αm−𝜷O′⁢𝒙iOσ)αm−𝜷O′⁢𝒙iOσ (71)
=0 (72)

Similarly:

limαm→−∞∂Φ2⁢(αm−𝜷O′⁢𝒙iOσ,𝜷S′⁢𝒙iS,−ρ)∂σ=0 (73)
∂ℓi∂σ =∂∂σ⁢((1−yiS)⁢ln⁡[Φ⁡(−𝜷S′⁢𝒙iS)]CLOSE (74)
+∑m=1MyiS(yiO=m)ln[Φ2(αm+1−𝜷O′⁢𝒙iOσ,𝜷S′𝒙iS,−ρ)
−Φ2(αm−𝜷O′⁢𝒙iOσ,𝜷S′𝒙iS,−ρ)])
=∑m=1MyiS(yiO=m)∂∂σ(ln[Φ2(αm+1−𝜷O′⁢𝒙iOσ,𝜷S′𝒙iS,−ρ) (75)
−Φ2(αm−𝜷O′⁢𝒙iOσ,𝜷S′𝒙iS,−ρ)])
=∑m=1MyiS⁢(yiO=m)⁢∂Φ2⁢(αm+1−𝜷O′⁢𝒙iOσ,𝜷S′⁢𝒙iS,−ρ)∂σ−OPEN∂Φ2⁢(αm−𝜷O′⁢𝒙iOσ,𝜷S′⁢𝒙iS,−ρ))∂σΦ2⁢(αm+1−𝜷O′⁢𝒙iOσ,𝜷S′⁢𝒙iS,−ρ)−Φ2⁢(αm−𝜷O′⁢𝒙iOσ,𝜷S′⁢𝒙iS,−ρ) (76)
=∑m=1MyiS⁢(yiO=m)Φ2⁢(αm+1−𝜷O′⁢𝒙iOσ,𝜷S′⁢𝒙iS,−ρ)−Φ2⁢(αm−𝜷O′⁢𝒙iOσ,𝜷S′⁢𝒙iS,−ρ) (77)
(Φ⁡(𝜷S′⁢𝒙iS+ρ⁢αm+1−𝜷O′⁢𝒙iOσ1−ρ2)⁢ϕ⁢(αm+1−𝜷O′⁢𝒙iOσ)⋅𝜷O′⁢𝒙iO−αm+1σ2CLOSE
−Φ(𝜷S′⁢𝒙iS+ρ⁢αm−𝜷O′⁢𝒙iOσ1−ρ2)ϕ(αm−𝜷O′⁢𝒙iOσ)⋅𝜷O′⁢𝒙iO−αmσ2)
∂ℓi∂log⁡(σ) =∂ℓi∂σ⁢∂σ∂log⁡(σ)=∂ℓi∂σ⁢σ (78)

6 Example with a Generated Dataset

7 Generate Dataset

> library( "mvtnorm" )> # number of observations> nObs <- 300> # parameters> betaS <- c( 1, 1, -1 )> betaO <- c( 10, 4 )> rho <- 0.4> sigma <- 5> # boundaries of the intervals> bound <- c(-Inf,5,15,Inf)> # set 'seed' of the pseudo random number generator> # in order to always generate the same pseudo random numbers> set.seed(123)> # generate variables x1 and x2> dat <- data.frame( x1 = rnorm( nObs ), x2 = rnorm( nObs ) )> # variance-covariance matrix of the two error terms> vcovMat <- matrix( c( 1, rho*sigma, rho*sigma, sigma^2 ), nrow = 2 )> # generate the two error terms> eps <- rmvnorm( nObs, sigma = vcovMat )> dat$epsS <- eps[,1]> dat$epsO <- eps[,2]> # generate the selection variable> dat$yS <- with( dat, betaS[1] + betaS[2] * x1 + betaS[3] * x2 + epsS ) > 0> table( dat$yS )

FALSE TRUE 91 209

> # generate the unobserved/latent outcome variable> dat$yOu <- with( dat, betaO[1] + betaO[2] * x1 + epsO )> dat$yOu[ !dat$yS ] <- NA> # obtain the intervals of the outcome variable> dat$yO <- cut( dat$yOu, bound )> table( dat$yO )

(-Inf,5] (5,15] (15, Inf] 26 130 53

7.1 Estimation of the Model

In the following estimation, the starting values are obtained by a maximum-likelihood (ML) estimation of a tobit-2 model, where the dependent variable of the outcome equation is set to the mid points of the intervals:

> library( "sampleSelection" )> res <- selection( yS ~ x1 + x2, yO ~ x1, data = dat, boundaries = bound )> res

Call: selection(selection = yS ~ x1 + x2, outcome = yO ~ x1, data = dat, boundaries = bound)Coefficients:S:(Intercept) S:x1 S:x2 O:(Intercept) O:x1 0.9821 0.9668 -1.2862 10.2419 2.6586 logSigma atanhRho sigma sigmaSq rho 1.6307 0.2976 5.1076 26.0879 0.2891

> summary( res )

--------------------------------------------Tobit 2 model with interval outcome (sample selection model)Maximum Likelihood estimationBHHH maximisation, 22 iterationsReturn code 8: successive function values within relative tolerance limit (reltol)Log-Likelihood: -275.395300 observations (91 censored and 209 observed)Intervals of the dependent variable of the outcome equation: YO lower upper count1 (-Inf,5] -Inf 5 262 (5,15] 5 15 1303 (15, Inf] 15 Inf 537 free parameters (df = 293)Probit selection equation: Estimate Std. Error t value Pr(>|t|)(Intercept) 0.9821 0.1085 9.050 < 2e-16 ***x1 0.9668 0.1491 6.484 3.78e-10 ***x2 -1.2862 0.1209 -10.636 < 2e-16 ***Outcome equation: Estimate Std. Error t value Pr(>|t|)(Intercept) 10.2419 0.6683 15.325 < 2e-16 ***x1 2.6586 0.5922 4.489 1.03e-05 *** Error terms: Estimate Std. Error t value Pr(>|t|)logSigma 1.63074 0.07472 21.825 < 2e-16 ***atanhRho 0.29757 0.36189 0.822 0.412sigma 5.10763 0.38163 13.384 < 2e-16 ***sigmaSq 26.08791 3.89850 6.692 1.12e-10 ***rho 0.28909 0.33164 0.872 0.384---Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1--------------------------------------------

In the following estimation, the starting values are obtained by a two-step estimation of a tobit-2 model, where the dependent variable of the outcome equation is set to the mid points of the intervals:

> res2 <- selection( yS ~ x1 + x2, yO ~ x1, data = dat, boundaries = bound,+ start = "2step" )> res2

Call: selection(selection = yS ~ x1 + x2, outcome = yO ~ x1, data = dat, start = "2step", boundaries = bound)Coefficients:S:(Intercept) S:x1 S:x2 O:(Intercept) O:x1 0.9821 0.9668 -1.2862 10.2419 2.6586 logSigma atanhRho sigma sigmaSq rho 1.6307 0.2976 5.1076 26.0879 0.2891

> summary( res2 )

--------------------------------------------Tobit 2 model with interval outcome (sample selection model)Maximum Likelihood estimationBHHH maximisation, 22 iterationsReturn code 8: successive function values within relative tolerance limit (reltol)Log-Likelihood: -275.395300 observations (91 censored and 209 observed)Intervals of the dependent variable of the outcome equation: YO lower upper count1 (-Inf,5] -Inf 5 262 (5,15] 5 15 1303 (15, Inf] 15 Inf 537 free parameters (df = 293)Probit selection equation: Estimate Std. Error t value Pr(>|t|)(Intercept) 0.9821 0.1085 9.050 < 2e-16 ***x1 0.9668 0.1491 6.484 3.78e-10 ***x2 -1.2862 0.1209 -10.636 < 2e-16 ***Outcome equation: Estimate Std. Error t value Pr(>|t|)(Intercept) 10.2419 0.6683 15.325 < 2e-16 ***x1 2.6586 0.5922 4.489 1.03e-05 *** Error terms: Estimate Std. Error t value Pr(>|t|)logSigma 1.63074 0.07472 21.825 < 2e-16 ***atanhRho 0.29757 0.36189 0.822 0.412sigma 5.10763 0.38163 13.384 < 2e-16 ***sigmaSq 26.08791 3.89850 6.692 1.12e-10 ***rho 0.28909 0.33164 0.872 0.384---Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1--------------------------------------------

The following commands compare the starting values and the estimated coefficients:

> # compare starting values (small differences)> cbind( res$start, res2$start, res$start - res2$start )

[,1] [,2] [,3](Intercept) 0.9818072 0.9800827 0.001724574x1 0.9663185 0.9686013 -0.002282885x2 -1.2866893 -1.2808419 -0.005847461(Intercept) 10.3990738 10.3516223 0.047451510x1 3.7408112 3.7642665 -0.023455273logSigma 4.2797893 4.2813219 -0.001532652atanhRho 0.2362382 0.2550566 -0.018818413

> # combare estimated coefficients (virtually identical)> cbind( coef( res ), coef( res2 ), coef( res ) - coef( res2 ) )

[,1] [,2] [,3](Intercept) 0.9820618 0.9820617 1.626690e-07x1 0.9668069 0.9668068 1.116395e-07x2 -1.2861861 -1.2861863 1.863040e-07(Intercept) 10.2419399 10.2419334 6.516212e-06x1 2.6585966 2.6586014 -4.744749e-06logSigma 1.6307359 1.6307360 -8.697063e-08atanhRho 0.2975695 0.2975745 -4.901548e-06sigma 5.1076321 5.1076326 -4.442140e-07sigmaSq 26.0879061 26.0879107 -4.537764e-06rho 0.2890868 0.2890913 -4.491913e-06

8 Example with the ‘Smoke’ dataset

The following command loads the dataset:

> data( "Smoke" )

The following command creates a vector with the boundaries of the intervals:

> bounds <- c( 0, 5, 10, 20, 50, Inf )

The following command estimates the model with few explanatory variables:

> SmokeRes1 <- selection( smoker ~ educ + age,+ cigs_intervals ~ educ, data = Smoke, boundaries = bounds )

The following command estimates the model with more explanatory variables:

> SmokeRes2 <- selection( smoker ~ educ + age + restaurn,+ cigs_intervals ~ educ + income + restaurn, data = Smoke,+ boundaries = bounds )

The following commands test whether adding further explanatory variables significantly improves the explanatory power of the model:

> library( "lmtest" )> lrtest( SmokeRes1, SmokeRes2 )

Likelihood ratio testModel 1: selection(selection = smoker ~ educ + age, outcome = cigs_intervals ~ educ, data = Smoke, boundaries = bounds)Model 2: selection(selection = smoker ~ educ + age + restaurn, outcome = cigs_intervals ~ educ + income + restaurn, data = Smoke, boundaries = bounds) #Df LogLik Df Chisq Pr(>Chisq)1 7 -940.542 10 -936.30 3 8.4705 0.03723 *---Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

> waldtest( SmokeRes1, SmokeRes2 )

Wald testModel 1: selection(selection = smoker ~ educ + age, outcome = cigs_intervals ~ educ, data = Smoke, boundaries = bounds)Model 2: selection(selection = smoker ~ educ + age + restaurn, outcome = cigs_intervals ~ educ + income + restaurn, data = Smoke, boundaries = bounds) Res.Df Df Chisq Pr(>Chisq)1 8002 797 3 7.8636 0.04892 *---Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1Both tests indicate that—at 5% significance level—the model with more explanatory variables (SmokeRes2) has significantly higher explanatory power than the model with fewer explanatory variables (SmokeRes1).

References

  • Petersen et al. (2017) S. Petersen, G. Henningsen, and A. Henningsen Can changes in households’ life situations predict participation in energy audits and investments in energy savings?. Note: Unpublished Manuscript. Department of Management Engineering, Technical University of Denmark Cited by: Interval Regression with Sample Selection.
  • Sibuya (1960) M. Sibuya Bivariate extreme statistics, i. Annals of the Institute of Statistical Mathematics 11 (2), pp. 195 – 210. Cited by: §4.2.
  • Sungur (1990) E. Sungur Dependence information in parameterized copulas. Communications in Statistics - Simulation and Computation 19 (4), pp. 1339 – 1360. Cited by: §4.2.