Predicting Property Sale Values in Ames, Iowa

Cleaning

The data comprises information collected from the Ames City Assessor’s OfficeRather than from traditional MLS data sources during 2006—2010 with variables being nominal, ordinal, continuous, and discrete in nature. As a real-world dataset, it required extensive cleaning and feature engineering to be able to garner insights and create models.

We dropped one duplicate observation and removed two outliersWhere GrLivArea > 4,000 feet² as per the recommendation of the dataset authorDean De Cock. Journal of Statistics Education Volume 19, Number 3. Another approach would have been to define outliers as properties whose sale prices were more than, say, four standard deviations from the mean. This would have dropped five points in our case.

Imputation

We imputed missing values^⊕For implementation details, see clean.ipynb for features by taking the mean, median, or mode as applicable of the feature category.

Sometimes, imputation was done by making reasonable assumptions. For instance, a missing GarageYrBlt can be imputed by the year the property was built. Finally, we imputed None (e.g., MasVnrType) or 0 (e.g., MasVnrArea) for values that might not exist or for which an educated guess could not be made.

Market Analysis

In addition to exploring the dataset itself, we wanted to focus on the Ames housing market dynamics. We came in to this with a lot of assumptions—the 2008 crash must have caused house prices to fall. Seasonality patterns in the housing market must change too. Surely, single-family homes would be more resilient than multi-dwelling units to the depressed market conditions?

We used a SARIMA model to look at average SalePrice for the dataset. We split up the properties by quintile to see if there was a difference between expensive and cheap houses.

We can see that in 2006-2010, the house prices by quintile stayed relatively constant. This is counterintuitive, since the housing market collapsed in 2008, and we do not see that reflected in any of the quintiles. Given that we lived through this recession and knew house prices to be falling where we grew up, we were perplexed at what was going on in Ames. We conducted a Ljung-Box testSee code to see if we could reject H₀—whether the data are independently distributed. I.e., the correlations in the population from which the sample is taken are zero, so that any observed correlations in the data result from the randomness of the sampling process.

# Ljung-Box Test—We test to reject H_0, looking for p<.05
test1 <- Box.test(pur.ts,type="Ljung-Box", lag= log(nrow(pur.ts))) # p = .99
test2 <- Box.test(o.ts,type="Ljung-Box", lag= log(nrow(o.ts)))     # p = .14
test3 <- Box.test(t.ts,type="Ljung-Box", lag= log(nrow(t.ts)))     # p = .34
test4 <- Box.test(th.ts,type="Ljung-Box", lag= log(nrow(th.ts)))   # p = .94
test5 <- Box.test(fo.ts,type="Ljung-Box", lag= log(nrow(fo.ts)))   # p = .33
test6 <- Box.test(fi.ts,type="Ljung-Box", lag= log(nrow(fi.ts)))   # p = .31

The Ljung-Box test for each quintile reports a p-value that is over the threshold of 0.05, meaning these values are white noise distributions around an average value. This is an extremely stationary dataset—remarkably so.

However, looking at the U.S. Federal Housing Finance Agency data from 2006-2010, we can see that in Iowa, the prices stayed remarkably constant even through the housing market crisis. This accounts for how unintuitive the results we’ve seen were–housing in Iowa itself behaved in a counterintuitive way.

Looking on to the left, we can see that the whole of the average number of houses sold at each date oscillates with a regular seasonality through the year.

However, when we split housing stock into the upper- and lower-half, we can see that, before 2008, the more expensive prices were sold more often. After the crash, the cheaper houses were sold more often. However, we start to see the original pattern return around 2010, the year which the housing market began to significantly recover.

This is important for Regression Realty’s realtors, as they know that they should now switch back to focusing on expensive houses and that the market might be recovering.

Finally, we use a SARIMA model^⊕

SARIMA

Seasonal Autoregressive Integrated Moving Average: An extension to ARIMA that supports the direct modeling of the seasonal component of the time series. It is a statistical model that uses Seasonality, Autocorrelation, Differencing, and Moving Averages to predict future data given only past data.

Model Scoring

Although there were multiple complicated models that we applied to the dataset, they did not drastically outperform linear regression in predictive power. This dataset is very conducive towards using linear regression—unsurprising as it was designed to be a regression end-of-semester project.

Some models take a long time to code for a data science team and others take a long time to train, so there are trade-offs to be made. For our case, we wanted a model that served our three goals.

Below is a comparison of our models.

In addition, we used a time series model to predict average SalePrice for the entire Ames housing market. This would be a more robust with more observations, such as in a larger city or if we had more complete data for Ames.

Statistical Validation

Applying linear regression blindly is folly. For the Ames dataset, the four assumptions of linear regression are

1. Linearity–that there exists a linear relationship between the SalePrice and predictors (features)
2. Independence–the residuals are independent
3. Homoscedasticity–the residuals have constant variance
4. Normality–the residuals are normally distributed

The following shows ways we checked the assumptions of multiple linear regression. We paid special attention to not inflate VIF by introducing multicollinearity when adding new features during feature selection.

• First, from the correlation matrixThe image is illustrative, we did not use these exact features in our linear model., we see that there is low multicollinearity between features.

• We see that the residuals are normally distributed.

• We see that the residuals are linear.

• We see that the residuals are linear.

• The Variance Inflation Factor (VIF) quantifies the severity of multicollinearity in regression analysis. It provides an estimate of how much the variance of a regression coefficient is increased because of collinearity. The table below shows a low generalized VIF for our predictorsWe use the R car package's vif function.

• By looking at the predictor slopes and their confidence intervals, we see that some encompass 0. HasPool was thus removed as a predictor.

Future Development

If we had more time, we would have liked to

• Use log-log transformations for our models,
• Create a mobile app for our fake realtors at Regression Realty
• Incorporate household income and MLS data into our models
• Investigate the impact of distance from Iowa State University or other locations
  • This is not promising as we can see from the maps that YearBuilt would trump distance from a major location
• Add interaction and polynomial features

Engineering Process

We tackled this project together with team members taking on different models. The data pipeline is shown below. Note that we would have liked to have a train-test split that was the same across all models, but did not have time to implement that. As a result, the model results are not directly comparable.