
R version 3.1.0 (2014-04-10) -- "Spring Dance"
Copyright (C) 2014 The R Foundation for Statistical Computing
Platform: i386-w64-mingw32/i386 (32-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

  Natural language support but running in an English locale

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

> library(norm2)
> 
> ## run EM on fake data with no missing values
> set.seed(1234)
> simdata <- data.frame(
+    Y1=rnorm(6), Y2=rnorm(6), Y3=rnorm(6), X1=rnorm(6) )
> emResult <- emNorm( cbind(Y1,Y2,Y3) ~ X1, data=simdata )
> print( summary( emResult ) )
Predictor (X) variables:
                 Mean       SD Observed Missing Pct.Missing
(Intercept) 1.0000000 0.000000        6       0           0
X1          0.2068512 1.178512        6       0           0

Response (Y) variables:
         Mean        SD Observed Missing Pct.Missing
Y1 -0.2092854 1.2953550        6       0           0
Y2 -0.6752401 0.2138685        6       0           0
Y3 -0.2141319 0.6864124        6       0           0

Missingness patterns for response (Y) variables
   (. denotes observed value, m denotes missing value)
   (variable names are displayed vertically)
   (rightmost column is the frequency):
YYY
123
... 6

Method:                             EM
Prior:                              "uniform"
Convergence criterion:              1e-05
Iterations:                         2
Converged:                          TRUE
Max. rel. difference:               0
-2 Loglikelihood:                   -8.3665
-2 Log-posterior density:           -8.3665
Worst fraction missing information: 0

Estimated coefficients (beta):
                    Y1         Y2         Y3
(Intercept) -0.3069981 -0.6785479 -0.2457194
X1           0.4723816  0.0159912  0.1527063

Estimated covariance matrix (sigma):
           Y1         Y2         Y3
Y1 1.14001811 0.06789367 0.13397509
Y2 0.06789367 0.03782049 0.03942553
Y3 0.13397509 0.03942553 0.36564511
> 
> ## impose missing values and run again
> simdata$Y1[3] <- simdata$Y2[4] <- simdata$Y3[4] <- NA
> emResult <- emNorm( cbind(Y1,Y2,Y3) ~ X1, data=simdata )
> print( summary( emResult ) )
Predictor (X) variables:
                 Mean       SD Observed Missing Pct.Missing
(Intercept) 1.0000000 0.000000        6       0           0
X1          0.2068512 1.178512        6       0           0

Response (Y) variables:
         Mean        SD Observed Missing Pct.Missing
Y1 -0.4680307 1.2630567        5       1    16.66667
Y2 -0.6322806 0.2081665        5       1    16.66667
Y3 -0.2349012 0.7653216        5       1    16.66667

Missingness patterns for response (Y) variables
   (. denotes observed value, m denotes missing value)
   (variable names are displayed vertically)
   (rightmost column is the frequency):
YYY
123
... 4
.mm 1
m.. 1

Method:                             EM
Prior:                              "uniform"
Convergence criterion:              1e-05
Iterations:                         114
Converged:                          TRUE
Max. rel. difference:               9.8836e-06
-2 Loglikelihood:                   -10.70465
-2 Log-posterior density:           -10.70465
Worst fraction missing information: 0.9313

Estimated coefficients (beta):
                    Y1          Y2         Y3
(Intercept) -1.0195738 -0.61863204 -0.1874030
X1           0.5166042 -0.01611584  0.1214564

Estimated covariance matrix (sigma):
            Y1          Y2          Y3
Y1  1.80844106 -0.06800034 -0.75502100
Y2 -0.06800034  0.03443290  0.05759078
Y3 -0.75502100  0.05759078  0.41668488
> 
> ## redundant Y-variable
> simdata$Y3 <- simdata$Y1 + simdata$Y2
> emResult <- emNorm( cbind(Y1,Y2,Y3) ~ X1, data=simdata )
Note: Finite-differencing procedure strayed outside
parameter space; solution at or near boundary
OCCURRED IN: estimate_worst_frac in MOD norm_engine
> 
> ## redundant X-variable
> set.seed(987)
> simdata <- data.frame(
+    Y1=rnorm(10), Y2=rnorm(10), Y3=rnorm(10), X1=rnorm(10) )
> simdata$X2 <- simdata$X1 + 4
> emResult <- emNorm( cbind(Y1,Y2,Y3) ~ X1 + X2, data=simdata )
Note: Matrix not positive definite
OCCURRED IN: cholesky_saxpy in MOD matrix_methods
Predictor (X) matrix does not have full rank
OCCURRED IN: make_xtxinv in MOD norm_engine
OCCURRED IN: run_norm_engine_em in MOD norm_engine
> 
> ## run EM for cholesterol data
> data(cholesterol)
> emResult <- emNorm(cholesterol)
> print( summary(emResult) )
Predictor (X) variables:
      Mean SD Observed Missing Pct.Missing
CONST    1  0       28       0           0

Response (Y) variables:
       Mean       SD Observed Missing Pct.Missing
Y1 253.9286 47.71049       28       0     0.00000
Y2 230.6429 46.96745       28       0     0.00000
Y3 221.4737 43.18355       19       9    32.14286

Missingness patterns for response (Y) variables
   (. denotes observed value, m denotes missing value)
   (variable names are displayed vertically)
   (rightmost column is the frequency):
YYY
123
... 19
..m  9

Method:                             EM
Prior:                              "uniform"
Convergence criterion:              1e-05
Iterations:                         15
Converged:                          TRUE
Max. rel. difference:               8.5201e-06
-2 Loglikelihood:                   615.9902
-2 Log-posterior density:           615.9902
Worst fraction missing information: 0.4617

Estimated coefficients (beta):
            Y1       Y2       Y3
CONST 253.9286 230.6429 222.2371

Estimated covariance matrix (sigma):
          Y1       Y2        Y3
Y1 2194.9949 1454.617  835.3973
Y2 1454.6173 2127.158 1515.4584
Y3  835.3973 1515.458 1952.2182
> 
> ## re-run using formula notation
> emResult <- emNorm( cbind(Y1,Y2,Y3) ~ 1, data=cholesterol)
> print( summary(emResult) )
Predictor (X) variables:
            Mean SD Observed Missing Pct.Missing
(Intercept)    1  0       28       0           0

Response (Y) variables:
       Mean       SD Observed Missing Pct.Missing
Y1 253.9286 47.71049       28       0     0.00000
Y2 230.6429 46.96745       28       0     0.00000
Y3 221.4737 43.18355       19       9    32.14286

Missingness patterns for response (Y) variables
   (. denotes observed value, m denotes missing value)
   (variable names are displayed vertically)
   (rightmost column is the frequency):
YYY
123
... 19
..m  9

Method:                             EM
Prior:                              "uniform"
Convergence criterion:              1e-05
Iterations:                         15
Converged:                          TRUE
Max. rel. difference:               8.5201e-06
-2 Loglikelihood:                   615.9902
-2 Log-posterior density:           615.9902
Worst fraction missing information: 0.4617

Estimated coefficients (beta):
                  Y1       Y2       Y3
(Intercept) 253.9286 230.6429 222.2371

Estimated covariance matrix (sigma):
          Y1       Y2        Y3
Y1 2194.9949 1454.617  835.3973
Y2 1454.6173 2127.158 1515.4584
Y3  835.3973 1515.458 1952.2182
> 
> ## re-run using Y1, Y2 as predictors
> emResult <- emNorm( Y3 ~ Y1 + Y2, data=cholesterol)
> print( summary(emResult) )
Predictor (X) variables:
                Mean       SD Observed Missing Pct.Missing
(Intercept)   1.0000  0.00000       28       0           0
Y1          253.9286 47.71049       28       0           0
Y2          230.6429 46.96745       28       0           0

Response (Y) variables:
       Mean       SD Observed Missing Pct.Missing
Y3 221.4737 43.18355       19       9    32.14286

Missingness patterns for response (Y) variables
   (. denotes observed value, m denotes missing value)
   (variable names are displayed vertically)
   (rightmost column is the frequency):
Y
3
. 19
m  9

Method:                             EM
Prior:                              "uniform"
Convergence criterion:              1e-05
Iterations:                         10
Converged:                          TRUE
Max. rel. difference:               4.6597e-06
-2 Loglikelihood:                   146.9102
-2 Log-posterior density:           146.9102
Worst fraction missing information: 0.3214

Estimated coefficients (beta):
                    Y3
(Intercept) 74.0236337
Y1          -0.1673990
Y2           0.8269102

Estimated covariance matrix (sigma):
         Y3
Y3 838.9239
> 
> ## find starting values for trivariate model using lm
> tmp <- lm( cbind(Y1,Y2,Y3) ~ 1, data=cholesterol )
> startval <- list(
+    beta = tmp$coef,
+    sigma = t(tmp$res) %*% tmp$res / tmp$df.res
+    )
> emResult <- emNorm( cbind(Y1,Y2,Y3) ~ 1, data=cholesterol,
+    starting.values = startval )
> print( summary(emResult) )
Predictor (X) variables:
            Mean SD Observed Missing Pct.Missing
(Intercept)    1  0       28       0           0

Response (Y) variables:
       Mean       SD Observed Missing Pct.Missing
Y1 253.9286 47.71049       28       0     0.00000
Y2 230.6429 46.96745       28       0     0.00000
Y3 221.4737 43.18355       19       9    32.14286

Missingness patterns for response (Y) variables
   (. denotes observed value, m denotes missing value)
   (variable names are displayed vertically)
   (rightmost column is the frequency):
YYY
123
... 19
..m  9

Method:                             EM
Prior:                              "uniform"
Convergence criterion:              1e-05
Iterations:                         8
Converged:                          TRUE
Max. rel. difference:               5.7425e-06
-2 Loglikelihood:                   615.9902
-2 Log-posterior density:           615.9902
Worst fraction missing information: 0.3214

Estimated coefficients (beta):
                  Y1       Y2       Y3
(Intercept) 253.9286 230.6429 222.2372

Estimated covariance matrix (sigma):
          Y1       Y2        Y3
Y1 2194.9949 1454.617  835.3979
Y2 1454.6173 2127.158 1515.4672
Y3  835.3979 1515.467 1952.2379
> 
> ## run EM for marijuana data with ridge prior
> data(marijuana)
> emResult <- emNorm(marijuana, prior="ridge", prior.df=0.5)
> print( summary(emResult) )
Predictor (X) variables:
      Mean SD Observed Missing Pct.Missing
CONST    1  0        9       0           0

Response (Y) variables:
             Mean        SD Observed Missing Pct.Missing
Plac.15  8.750000  8.413425        8       1    11.11111
Low.15  16.888889  8.192137        9       0     0.00000
High.15 18.250000 11.877349        8       1    11.11111
Plac.90  1.000000 13.304135        7       2    22.22222
Low.90   7.555556  9.527737        9       0     0.00000
High.90 -3.250000 12.279483        8       1    11.11111

Missingness patterns for response (Y) variables
   (. denotes observed value, m denotes missing value)
   (variable names are displayed vertically)
   (rightmost column is the frequency):
P HP H
lLilLi
aogaog
cwhcwh
......
111999
555000
...... 6
...m.. 1
..mm.. 1
m....m 1

Method:                             EM
Prior:                              "ridge"
Prior df:                           0.5
Convergence criterion:              1e-05
Iterations:                         159
Converged:                          TRUE
Max. rel. difference:               9.7544e-06
-2 Loglikelihood:                   252.7862
-2 Log-posterior density:           434.1799
Worst fraction missing information: 0.9478

Estimated coefficients (beta):
       Plac.15   Low.15  High.15  Plac.90   Low.90   High.90
CONST 7.681907 16.88889 14.77574 5.264824 7.555556 -2.602595

Estimated covariance matrix (sigma):
           Plac.15     Low.15   High.15   Plac.90     Low.90   High.90
Plac.15  38.093845 -10.236764 -34.82589  17.64182  -2.710528 10.994982
Low.15  -10.236764  34.572391  37.93579 -17.88361 -21.117845  7.219839
High.15 -34.825889  37.935786 117.83039 -69.86011 -50.007070 19.152134
Plac.90  17.641817 -17.883607 -69.86011 112.58849  34.945886 12.636713
Low.90   -2.710528 -21.117845 -50.00707  34.94589  46.764310 -3.400230
High.90  10.994982   7.219839  19.15213  12.63671  -3.400230 74.162883
> 
> ## run EM on flas data with uniform prior
> data(flas)
> emResult <- emNorm( flas )
Note: Eigen power method failed to converge
OCCURRED IN: estimate_worst_frac in MOD norm_engine
> 
> ## run EM on flas data with ridge prior
> emResult <- emNorm( flas, prior="ridge", prior.df=0.5 )
Warning message:
In emNorm.default(flas, prior = "ridge", prior.df = 0.5) :
  Algorithm did not converge by iteration 1000
> 
> ## continue until convergence
> emResult <- emNorm( emResult, iter.max=5000 )
> print( summary( emResult, show.params=FALSE ) )
Predictor (X) variables:
      Mean SD Observed Missing Pct.Missing
CONST    1  0      279       0           0

Response (Y) variables:
             Mean         SD Observed Missing Pct.Missing
LAN2   0.27956989  0.4495943      279       0   0.0000000
LAN3   0.40860215  0.4924588      279       0   0.0000000
LAN4   0.07168459  0.2584285      279       0   0.0000000
AGE    1.53731343  0.4995386      268      11   3.9426523
PRI    2.19776119  0.8313129      268      11   3.9426523
SEX    0.45323741  0.4987062      278       1   0.3584229
MLAT  82.48745520 14.0261454      279       0   0.0000000
FLAS  24.25652174  6.2564779      230      49  17.5627240
SATV 501.51428571 91.1623304      245      34  12.1863799
SATM 564.24897959 88.7069640      245      34  12.1863799
ENG   53.95041322 15.4018651      242      37  13.2616487
HGPA   2.75043165  0.6165543      278       1   0.3584229
CGPA   3.29440816  0.4768538      245      34  12.1863799
GRD    1.54310345  0.4992157      232      47  16.8458781

Missingness patterns for response (Y) variables
   (. denotes observed value, m denotes missing value)
   (variable names are displayed vertically)
   (rightmost column is the frequency):
LLL   MFSS HC 
AAAAPSLLAAEGGG
NNNGREAATTNPPR
234EIXTSVMGAAD
.............. 174
.............m  18
..........m...   1
..........m..m   2
........mmm.m.  20
........mmm.mm   7
........mmmmm.   1
.......m......  26
.......m.....m  15
.......mmmm.m.   2
.......mmmm.mm   1
.....m.......m   1
...mm.........   3
...mm........m   1
...mm...mmm.m.   2
...mm..m......   3
...mm..m.....m   1
...mm..mmmm.mm   1

Method:                             EM
Prior:                              "ridge"
Prior df:                           0.5
Convergence criterion:              1e-05
Iterations:                         2148
Converged:                          TRUE
Max. rel. difference:               9.9965e-06
-2 Loglikelihood:                   7315.651
-2 Log-posterior density:           7559.383
Worst fraction missing information: 0.9983
> 
> ## treat completely observed variables as predictors;
> ## uniform prior
> emResult <- emNorm(
+    cbind(AGE,PRI,SEX,FLAS,SATV,SATM,ENG,HGPA,CGPA,GRD) ~ 
+       LAN2 + LAN3 + LAN4 + MLAT, data=flas )
Note: Eigen power method failed to converge
OCCURRED IN: estimate_worst_frac in MOD norm_engine
> print( summary( emResult, show.params=FALSE ) )
Predictor (X) variables:
                   Mean         SD Observed Missing Pct.Missing
(Intercept)  1.00000000  0.0000000      279       0           0
LAN2         0.27956989  0.4495943      279       0           0
LAN3         0.40860215  0.4924588      279       0           0
LAN4         0.07168459  0.2584285      279       0           0
MLAT        82.48745520 14.0261454      279       0           0

Response (Y) variables:
            Mean         SD Observed Missing Pct.Missing
AGE    1.5373134  0.4995386      268      11   3.9426523
PRI    2.1977612  0.8313129      268      11   3.9426523
SEX    0.4532374  0.4987062      278       1   0.3584229
FLAS  24.2565217  6.2564779      230      49  17.5627240
SATV 501.5142857 91.1623304      245      34  12.1863799
SATM 564.2489796 88.7069640      245      34  12.1863799
ENG   53.9504132 15.4018651      242      37  13.2616487
HGPA   2.7504317  0.6165543      278       1   0.3584229
CGPA   3.2944082  0.4768538      245      34  12.1863799
GRD    1.5431034  0.4992157      232      47  16.8458781

Missingness patterns for response (Y) variables
   (. denotes observed value, m denotes missing value)
   (variable names are displayed vertically)
   (rightmost column is the frequency):
   FSS HC 
APSLAAEGGG
GREATTNPPR
EIXSVMGAAD
.......... 174
.........m  18
......m...   1
......m..m   2
....mmm.m.  20
....mmm.mm   7
....mmmmm.   1
...m......  26
...m.....m  15
...mmmm.m.   2
...mmmm.mm   1
..m......m   1
mm........   3
mm.......m   1
mm..mmm.m.   2
mm.m......   3
mm.m.....m   1
mm.mmmm.mm   1

Method:                             EM
Prior:                              "uniform"
Convergence criterion:              1e-05
Iterations:                         19
Converged:                          TRUE
Max. rel. difference:               5.6639e-06
-2 Loglikelihood:                   6466.747
-2 Log-posterior density:           6466.747
Worst fraction missing information: NA

Eigen power method failed to converge
OCCURRED IN: estimate_worst_frac in MOD norm_engine
> 
> ## treat completely observed variables as predictors;
> ## ridge prior
> emResult <- emNorm(
+    cbind(AGE,PRI,SEX,FLAS,SATV,SATM,ENG,HGPA,CGPA,GRD) ~ 
+       LAN2 + LAN3 + LAN4 + MLAT, data=flas,
+    prior="ridge", prior.df=0.5 )
Note: Eigen power method failed to converge
OCCURRED IN: estimate_worst_frac in MOD norm_engine
> print( summary( emResult, show.params=FALSE ) )
Predictor (X) variables:
                   Mean         SD Observed Missing Pct.Missing
(Intercept)  1.00000000  0.0000000      279       0           0
LAN2         0.27956989  0.4495943      279       0           0
LAN3         0.40860215  0.4924588      279       0           0
LAN4         0.07168459  0.2584285      279       0           0
MLAT        82.48745520 14.0261454      279       0           0

Response (Y) variables:
            Mean         SD Observed Missing Pct.Missing
AGE    1.5373134  0.4995386      268      11   3.9426523
PRI    2.1977612  0.8313129      268      11   3.9426523
SEX    0.4532374  0.4987062      278       1   0.3584229
FLAS  24.2565217  6.2564779      230      49  17.5627240
SATV 501.5142857 91.1623304      245      34  12.1863799
SATM 564.2489796 88.7069640      245      34  12.1863799
ENG   53.9504132 15.4018651      242      37  13.2616487
HGPA   2.7504317  0.6165543      278       1   0.3584229
CGPA   3.2944082  0.4768538      245      34  12.1863799
GRD    1.5431034  0.4992157      232      47  16.8458781

Missingness patterns for response (Y) variables
   (. denotes observed value, m denotes missing value)
   (variable names are displayed vertically)
   (rightmost column is the frequency):
   FSS HC 
APSLAAEGGG
GREATTNPPR
EIXSVMGAAD
.......... 174
.........m  18
......m...   1
......m..m   2
....mmm.m.  20
....mmm.mm   7
....mmmmm.   1
...m......  26
...m.....m  15
...mmmm.m.   2
...mmmm.mm   1
..m......m   1
mm........   3
mm.......m   1
mm..mmm.m.   2
mm.m......   3
mm.m.....m   1
mm.mmmm.mm   1

Method:                             EM
Prior:                              "ridge"
Prior df:                           0.5
Convergence criterion:              1e-05
Iterations:                         19
Converged:                          TRUE
Max. rel. difference:               5.7272e-06
-2 Loglikelihood:                   6468.919
-2 Log-posterior density:           6664.591
Worst fraction missing information: NA

Eigen power method failed to converge
OCCURRED IN: estimate_worst_frac in MOD norm_engine
> 
> 
> proc.time()
   user  system elapsed 
   2.12    0.15    2.32 
