Exam 15: Multiple Regression Model Building

Which of the following is used to find a "best" model?

TABLE 15-6 Given below are results from the regression analysis on 40 observations where the dependent variable is the number of weeks a worker is unemployed due to a layoff (Y) and the independent variables are the age of the worker (X₁), the number of years of education received (X₂), the number of years at the previous job (X₃), a dummy variable for marital status (X₄: 1 = married, 0 = otherwise), a dummy variable for head of household (X₅: 1 = yes, 0 = no) and a dummy variable for management position (X₆: 1 = yes, 0 = no). The coefficient of multiple determination $\left( R _ { j } ^ { 2 } \right)$ for the regression model using each of the 6 variables Xⱼ as the dependent variable and all other X variables as independent variables are, respectively, 0.2628, 0.1240, 0.2404, 0.3510, 0.3342 and 0.0993. The partial results from best-subset regression are given below: Model R Square Adj. R Square Std. Error X1X5X6 0.4568 0.4116 18.3534 X1X2X5X6 0.4697 0.4091 18.3919 X1X3X5X6 0.4691 0.4084 18.4023 X1X2X3X5X6 0.4877 0.4123 18.3416 X1X2X3X4X5X6 0.4949 0.4030 18.4861 -Referring to Table 15-6, what is the value of the variance inflationary factor of Married?

An independent variable Xⱼ is considered highly correlated with the other independent variables if

TABLE 15-6 Given below are results from the regression analysis on 40 observations where the dependent variable is the number of weeks a worker is unemployed due to a layoff (Y) and the independent variables are the age of the worker (X₁), the number of years of education received (X₂), the number of years at the previous job (X₃), a dummy variable for marital status (X₄: 1 = married, 0 = otherwise), a dummy variable for head of household (X₅: 1 = yes, 0 = no) and a dummy variable for management position (X₆: 1 = yes, 0 = no). The coefficient of multiple determination $\left( R _ { j } ^ { 2 } \right)$ for the regression model using each of the 6 variables Xⱼ as the dependent variable and all other X variables as independent variables are, respectively, 0.2628, 0.1240, 0.2404, 0.3510, 0.3342 and 0.0993. The partial results from best-subset regression are given below: Model R Square Adj. R Square Std. Error X1X5X6 0.4568 0.4116 18.3534 X1X2X5X6 0.4697 0.4091 18.3919 X1X3X5X6 0.4691 0.4084 18.4023 X1X2X3X5X6 0.4877 0.4123 18.3416 X1X2X3X4X5X6 0.4949 0.4030 18.4861 -Referring to Table 15-6, what is the value of the Mallow's Cp statistic for the model that includes X₁, X₃, X₅ and X₆?

TABLE 15-3 A chemist employed by a pharmaceutical firm has developed a muscle relaxant. She took a sample of 14 people suffering from extreme muscle constriction. She gave each a vial containing a dose (X) of the drug and recorded the time to relief (Y) measured in seconds for each. She fit a "centered" curvilinear model to this data. The results obtained by Microsoft Excel follow, where the dose (X) given has been "centered." SUMMARY OUTPUT Regression Statistics Multiple R 0.747 R Square 0.558 Adjusted R Square 0.478 Standard Error 863.1 Observations 14 ANOVA df SS MS F Signif F Regression 2 10344797 5172399 6.94 0.0110 Residual 11 8193929 744903 Total 13 18538726 Coeff StdFrror t Stat p -value Intercept 1283.0 352.0 3.65 0.0040 CenDose 25.228 8.631 2.92 0.0140 CenDoseSa 0.8604 0.3722 2.31 0.0410 -Referring to Table 15-3, suppose the chemist decides to use a t test to determine if there is a significant difference between a linear model and a curvilinear model that includes a linear term. If she used a level of significance of 0.05, she would decide that the linear model is sufficient.

TABLE 15-1 A certain type of rare gem serves as a status symbol for many of its owners. In theory, for low prices, the demand increases and it decreases as the price of the gem increases. However, experts hypothesize that when the gem is valued at very high prices, the demand increases with price due to the status owners believe they gain in obtaining the gem. Thus, the model proposed to best explain the demand for the gem by its price is the quadratic model: Y = β₀ + β₁X + β₁X² + ε where Y = demand (in thousands) and X = retail price per carat. This model was fit to data collected for a sample of 12 rare gems of this type. A portion of the computer analysis obtained from Microsoft Excel is shown below: SUMMARY OUTPUT Regression Statistics Multiple R 0.994 R Square 0.988 Standard Error 12.42 Observations 12 ANOVA df SS MS F Signif F Regression 2 115145 57573 373 0.0001 Residual 9 1388 154 Total 11 116533 Coeff StdError t Stat p -value Intercept 286.42 9.66 29.64 0.0001 Price -0.31 0.06 -5.14 0.0006 Price Sq 0.000067 0.00007 0.95 0.3647 -Referring to Table 15-1, a more parsimonious simple linear model is likely to be statistically superior to the fitted curvilinear for predicting sale price (Y).

TABLE 15-2 In Hawaii, condemnation proceedings are under way to enable private citizens to own the property that their homes are built on. Until recently, only estates were permitted to own land, and homeowners leased the land from the estate. In order to comply with the new law, a large Hawaiian estate wants to use regression analysis to estimate the fair market value of the land. The following model was fit to data collected for n = 20 properties, 10 of which are located near a cove. Model 1: $Y = \beta _ { 0 } + \beta _ { 1 } X _ { 1 } + \beta _ { 2 } X _ { 2 } + \beta _ { 3 } X _ { 1 } X _ { 2 } + \beta _ { 4 } X _ { 1 } ^ { 2 } + \beta _ { 5 } X _ { 1 } ^ { 2 } X _ { 2 } + \varepsilon$ where $Y =$ Sale price of property in thousands of dollars $X _ { 1 } =$ Size of property in thousands of square feet $X _ { 2 } = 1$ if property located near cove, 0 if not Using the data collected for the 20 properties, the following partial output obtained from Microsoft Excel is shown: SUMMARY OUTPUT Regression Statistics Multiple R 0.985 R Square 0.970 Standard Error 9.5 Observations 20 ANOVA df SS MS F Signif F Regression 5 28324 5664 62.2 0.0001 Residual 14 1279 91 Total 19 29063 Coeff StdError t Stat p -value Intercept -32.1 35.7 -0.90 0.3834 Size 12.2 5.9 2.05 0.0594 Cove -104.3 53.5 -1.95 0.0715 Size Cove 17.0 8.5 1.99 0.0661 SizeSa -0.3 0.2 -1.28 0.2204 SizeSa*Cove -0.3 0.3 -1.13 0.2749 -Referring to Table 15-2, given a quadratic relationship between sale price (Y)and property size (X₁), what test should be used to test whether the curves differ from cove and non-cove properties?

TABLE 15-4 The superintendent of a school district wanted to predict the percentage of students passing a sixth-grade proficiency test. She obtained the data on percentage of students passing the proficiency test (% Passing), daily mean of the percentage of students attending class (% Attendance), mean teacher salary in dollars (Salaries), and instructional spending per pupil in dollars (Spending) of 47 schools in the state. Let Y = % Passing as the dependent variable, X₁ = % Attendance, X₂ = Salaries and X₃ = Spending. The coefficient of multiple determination () of each of the 3 predictors with all the other remaining predictors are, respectively, 0.0338, 0.4669, and 0.4743. The output from the best-subset regressions is given below: Adjusted Model Variables R. Square R. Square Std. Error 1 1 3.05 3 0.6024 0.593 10.5787 2 12 3.66 3 0.6145 0.597 10.535 3 123 4.00 4 0.6288 0.6029 10.4570 4 13 2.00 3 0.6288 0.6119 10.3375 5 2 67.35 3 0.0474 0.026 16.375 6 23 64.30 3 0.0910 0.0497 16.1768 7 3 62.33 3 0.0907 0.0705 15.9984 Following is the residual plot for % Attendance: Following is the output of several multiple regression models: Model (I): Coefficients Standard Error t Stat P-value Lower 95\% Upper 95\% Intercept -753.4225 101.1149 -7.4511 2.88-09 -957.340 -549.5050 \% Attendance 8.5014 1.0771 7.8929 6.73-1 6.3292 10.6735 Salary 6.85-07 0.0006 0.0011 0.9991 -0.0011 0.0013 Spending 0.0060 0.0046 1.2879 0.2047 -0.0034 0.0153 Model (II): Coefficients Standard Error t Stat P-value Intercept -753.4086 99.1451 -7.5991 0.0000 \% Attendance 8.5014 1.0645 7.9862 0.0000 Spending 0.0060 0.0034 1.767 0.0840 Model (III): df SS MS F Significance F Regression 2 8162.9429 4081.4714 39.8709 0.0000 Residual 44 4504.1635 102.3674 Total 46 12667.1064 Coefficients Standard Error t Stat P-value Intercept 6672.8367 3267.7349 2.0420 0.0472 \% Attendance -150.5694 69.9519 -2.1525 0.0369 \% Attendance Squared 0.8532 0.3743 2.2792 0.0276 -Referring to Table 15-4, the quadratic effect of daily average of the percentage of students attending class on percentage of students passing the proficiency test is not significant at a 5% level of significance.

TABLE 15-6 Given below are results from the regression analysis on 40 observations where the dependent variable is the number of weeks a worker is unemployed due to a layoff (Y) and the independent variables are the age of the worker (X₁), the number of years of education received (X₂), the number of years at the previous job (X₃), a dummy variable for marital status (X₄: 1 = married, 0 = otherwise), a dummy variable for head of household (X₅: 1 = yes, 0 = no) and a dummy variable for management position (X₆: 1 = yes, 0 = no). The coefficient of multiple determination $\left( R _ { j } ^ { 2 } \right)$ for the regression model using each of the 6 variables Xⱼ as the dependent variable and all other X variables as independent variables are, respectively, 0.2628, 0.1240, 0.2404, 0.3510, 0.3342 and 0.0993. The partial results from best-subset regression are given below: Model R Square Adj. R Square Std. Error X1X5X6 0.4568 0.4116 18.3534 X1X2X5X6 0.4697 0.4091 18.3919 X1X3X5X6 0.4691 0.4084 18.4023 X1X2X3X5X6 0.4877 0.4123 18.3416 X1X2X3X4X5X6 0.4949 0.4030 18.4861 -Referring to Table 15-6, what is the value of the Mallow's Cp statistic for the model that includes X₁, X₂, X₅ and X₆?

TABLE 15-6 Given below are results from the regression analysis on 40 observations where the dependent variable is the number of weeks a worker is unemployed due to a layoff (Y) and the independent variables are the age of the worker (X₁), the number of years of education received (X₂), the number of years at the previous job (X₃), a dummy variable for marital status (X₄: 1 = married, 0 = otherwise), a dummy variable for head of household (X₅: 1 = yes, 0 = no) and a dummy variable for management position (X₆: 1 = yes, 0 = no). The coefficient of multiple determination $\left( R _ { j } ^ { 2 } \right)$ for the regression model using each of the 6 variables Xⱼ as the dependent variable and all other X variables as independent variables are, respectively, 0.2628, 0.1240, 0.2404, 0.3510, 0.3342 and 0.0993. The partial results from best-subset regression are given below: Model R Square Adj. R Square Std. Error X1X5X6 0.4568 0.4116 18.3534 X1X2X5X6 0.4697 0.4091 18.3919 X1X3X5X6 0.4691 0.4084 18.4023 X1X2X3X5X6 0.4877 0.4123 18.3416 X1X2X3X4X5X6 0.4949 0.4030 18.4861 -Referring to Table 15-6, what is the value of the Mallow's Cp statistic for the model that includes X₁, X₂, X₃, X₅ and X₆?

TABLE 15-6 Given below are results from the regression analysis on 40 observations where the dependent variable is the number of weeks a worker is unemployed due to a layoff (Y) and the independent variables are the age of the worker (X₁), the number of years of education received (X₂), the number of years at the previous job (X₃), a dummy variable for marital status (X₄: 1 = married, 0 = otherwise), a dummy variable for head of household (X₅: 1 = yes, 0 = no) and a dummy variable for management position (X₆: 1 = yes, 0 = no). The coefficient of multiple determination $\left( R _ { j } ^ { 2 } \right)$ for the regression model using each of the 6 variables Xⱼ as the dependent variable and all other X variables as independent variables are, respectively, 0.2628, 0.1240, 0.2404, 0.3510, 0.3342 and 0.0993. The partial results from best-subset regression are given below: Model R Square Adj. R Square Std. Error X1X5X6 0.4568 0.4116 18.3534 X1X2X5X6 0.4697 0.4091 18.3919 X1X3X5X6 0.4691 0.4084 18.4023 X1X2X3X5X6 0.4877 0.4123 18.3416 X1X2X3X4X5X6 0.4949 0.4030 18.4861 -Referring to Table 15-6, the model that includes X?, X? and X? should be among the appropriate models using the Mallow's Cp statistic.

TABLE 15-4 The superintendent of a school district wanted to predict the percentage of students passing a sixth-grade proficiency test. She obtained the data on percentage of students passing the proficiency test (% Passing), daily mean of the percentage of students attending class (% Attendance), mean teacher salary in dollars (Salaries), and instructional spending per pupil in dollars (Spending) of 47 schools in the state. Let Y = % Passing as the dependent variable, X₁ = % Attendance, X₂ = Salaries and X₃ = Spending. The coefficient of multiple determination () of each of the 3 predictors with all the other remaining predictors are, respectively, 0.0338, 0.4669, and 0.4743. The output from the best-subset regressions is given below: Adjusted Model Variables R. Square R. Square Std. Error 1 1 3.05 3 0.6024 0.593 10.5787 2 12 3.66 3 0.6145 0.597 10.535 3 123 4.00 4 0.6288 0.6029 10.4570 4 13 2.00 3 0.6288 0.6119 10.3375 5 2 67.35 3 0.0474 0.026 16.375 6 23 64.30 3 0.0910 0.0497 16.1768 7 3 62.33 3 0.0907 0.0705 15.9984 Following is the residual plot for % Attendance: Following is the output of several multiple regression models: Model (I): Coefficients Standard Error t Stat P-value Lower 95\% Upper 95\% Intercept -753.4225 101.1149 -7.4511 2.88-09 -957.340 -549.5050 \% Attendance 8.5014 1.0771 7.8929 6.73-1 6.3292 10.6735 Salary 6.85-07 0.0006 0.0011 0.9991 -0.0011 0.0013 Spending 0.0060 0.0046 1.2879 0.2047 -0.0034 0.0153 Model (II): Coefficients Standard Error t Stat P-value Intercept -753.4086 99.1451 -7.5991 0.0000 \% Attendance 8.5014 1.0645 7.9862 0.0000 Spending 0.0060 0.0034 1.767 0.0840 Model (III): df SS MS F Significance F Regression 2 8162.9429 4081.4714 39.8709 0.0000 Residual 44 4504.1635 102.3674 Total 46 12667.1064 Coefficients Standard Error t Stat P-value Intercept 6672.8367 3267.7349 2.0420 0.0472 \% Attendance -150.5694 69.9519 -2.1525 0.0369 \% Attendance Squared 0.8532 0.3743 2.2792 0.0276 -Referring to Table 15-4, what is the value of the test statistic to determine whether the quadratic effect of daily average of the percentage of students attending class on percentage of students passing the proficiency test is significant at a 5% level of significance?

In data mining where huge data sets are being explored to discover relationships among a large number of variables, the best-subsets approach is more practical than the stepwise regression approach.

TABLE 15-6 Given below are results from the regression analysis on 40 observations where the dependent variable is the number of weeks a worker is unemployed due to a layoff (Y) and the independent variables are the age of the worker (X₁), the number of years of education received (X₂), the number of years at the previous job (X₃), a dummy variable for marital status (X₄: 1 = married, 0 = otherwise), a dummy variable for head of household (X₅: 1 = yes, 0 = no) and a dummy variable for management position (X₆: 1 = yes, 0 = no). The coefficient of multiple determination $\left( R _ { j } ^ { 2 } \right)$ for the regression model using each of the 6 variables Xⱼ as the dependent variable and all other X variables as independent variables are, respectively, 0.2628, 0.1240, 0.2404, 0.3510, 0.3342 and 0.0993. The partial results from best-subset regression are given below: Model R Square Adj. R Square Std. Error X1X5X6 0.4568 0.4116 18.3534 X1X2X5X6 0.4697 0.4091 18.3919 X1X3X5X6 0.4691 0.4084 18.4023 X1X2X3X5X6 0.4877 0.4123 18.3416 X1X2X3X4X5X6 0.4949 0.4030 18.4861 -Referring to Table 15-6, what is the value of the variance inflationary factor of Job Yr?

TABLE 15-3 A chemist employed by a pharmaceutical firm has developed a muscle relaxant. She took a sample of 14 people suffering from extreme muscle constriction. She gave each a vial containing a dose (X) of the drug and recorded the time to relief (Y) measured in seconds for each. She fit a "centered" curvilinear model to this data. The results obtained by Microsoft Excel follow, where the dose (X) given has been "centered." SUMMARY OUTPUT Regression Statistics Multiple R 0.747 R Square 0.558 Adjusted R Square 0.478 Standard Error 863.1 Observations 14 ANOVA df SS MS F Signif F Regression 2 10344797 5172399 6.94 0.0110 Residual 11 8193929 744903 Total 13 18538726 Coeff StdFrror t Stat p -value Intercept 1283.0 352.0 3.65 0.0040 CenDose 25.228 8.631 2.92 0.0140 CenDoseSa 0.8604 0.3722 2.31 0.0410 -Referring to Table 15-3, suppose the chemist decides to use a t test to determine if there is a significant difference between a linear model and a curvilinear model that includes a linear term. If she used a level of significance of 0.01, she would decide that the linear model is sufficient.

Collinearity is present when there is a high degree of correlation between the dependent variable and any of the independent variables.

TABLE 15-5 What are the factors that determine the acceleration time (in sec.) from 0 to 60 miles per hour of a car? Data on the following variables for 171 different vehicle models were collected: Accel Time: Acceleration time in sec. Cargo Vol: Cargo volume in cu. ft. HP: Horsepower MPG: Miles per gallon SUV: 1 if the vehicle model is an SUV with Coupe as the base when SUV and Sedan are both 0 Sedan: 1 if the vehicle model is a sedan with Coupe as the base when SUV and Sedan are both 0 The coefficient of multiple determination $\left( R _ { j } ^ { 2 } \right)$ for the regression model using each of the 5 variables Xⱼ as the dependent variable and all other X variables as independent variables are, respectively, 0.7461, 0.5676, 0.6764, 0.8582, 0.6632. -Referring to Table 15-5, what is the value of the variance inflationary factor of HP?

If independent variables are not significant individually but are significant as a group at a specified level of significance, this is most likely due to

TABLE 15-6 Given below are results from the regression analysis on 40 observations where the dependent variable is the number of weeks a worker is unemployed due to a layoff (Y) and the independent variables are the age of the worker (X₁), the number of years of education received (X₂), the number of years at the previous job (X₃), a dummy variable for marital status (X₄: 1 = married, 0 = otherwise), a dummy variable for head of household (X₅: 1 = yes, 0 = no) and a dummy variable for management position (X₆: 1 = yes, 0 = no). The coefficient of multiple determination $\left( R _ { j } ^ { 2 } \right)$ for the regression model using each of the 6 variables Xⱼ as the dependent variable and all other X variables as independent variables are, respectively, 0.2628, 0.1240, 0.2404, 0.3510, 0.3342 and 0.0993. The partial results from best-subset regression are given below: Model R Square Adj. R Square Std. Error X1X5X6 0.4568 0.4116 18.3534 X1X2X5X6 0.4697 0.4091 18.3919 X1X3X5X6 0.4691 0.4084 18.4023 X1X2X3X5X6 0.4877 0.4123 18.3416 X1X2X3X4X5X6 0.4949 0.4030 18.4861 -Referring to Table 15-6, the model that includes X?, X?, X? and X? should be among the appropriate models using the Mallow's Cp statistic.

TABLE 15-4 The superintendent of a school district wanted to predict the percentage of students passing a sixth-grade proficiency test. She obtained the data on percentage of students passing the proficiency test (% Passing), daily mean of the percentage of students attending class (% Attendance), mean teacher salary in dollars (Salaries), and instructional spending per pupil in dollars (Spending) of 47 schools in the state. Let Y = % Passing as the dependent variable, X₁ = % Attendance, X₂ = Salaries and X₃ = Spending. The coefficient of multiple determination () of each of the 3 predictors with all the other remaining predictors are, respectively, 0.0338, 0.4669, and 0.4743. The output from the best-subset regressions is given below: Adjusted Model Variables R. Square R. Square Std. Error 1 1 3.05 3 0.6024 0.593 10.5787 2 12 3.66 3 0.6145 0.597 10.535 3 123 4.00 4 0.6288 0.6029 10.4570 4 13 2.00 3 0.6288 0.6119 10.3375 5 2 67.35 3 0.0474 0.026 16.375 6 23 64.30 3 0.0910 0.0497 16.1768 7 3 62.33 3 0.0907 0.0705 15.9984 Following is the residual plot for % Attendance: Following is the output of several multiple regression models: Model (I): Coefficients Standard Error t Stat P-value Lower 95\% Upper 95\% Intercept -753.4225 101.1149 -7.4511 2.88-09 -957.340 -549.5050 \% Attendance 8.5014 1.0771 7.8929 6.73-1 6.3292 10.6735 Salary 6.85-07 0.0006 0.0011 0.9991 -0.0011 0.0013 Spending 0.0060 0.0046 1.2879 0.2047 -0.0034 0.0153 Model (II): Coefficients Standard Error t Stat P-value Intercept -753.4086 99.1451 -7.5991 0.0000 \% Attendance 8.5014 1.0645 7.9862 0.0000 Spending 0.0060 0.0034 1.767 0.0840 Model (III): df SS MS F Significance F Regression 2 8162.9429 4081.4714 39.8709 0.0000 Residual 44 4504.1635 102.3674 Total 46 12667.1064 Coefficients Standard Error t Stat P-value Intercept 6672.8367 3267.7349 2.0420 0.0472 \% Attendance -150.5694 69.9519 -2.1525 0.0369 \% Attendance Squared 0.8532 0.3743 2.2792 0.0276 -Referring to Table 15-4, which of the following predictors should first be dropped to remove collinearity?