Boston Housing (Regression)

A good dataset to practice Regression techniques, we can load the Boston Housing Dataset saved directly to Scikitlearn using the dataset submodule.

Loading the Data

from sklearn.datasets import load_boston

data = load_boston()

Doing so gives us a Bunch object

type(data)

sklearn.utils.Bunch

Which is basically a dictionary, but with some other stuff

data.__class__.__bases__

(dict,)

Inspecting the Data

Let’s look at the keys

data.keys()

dict_keys(['data', 'target', 'feature_names', 'DESCR'])

The data and target keys are just numpy arrays

print(type(data['data']), data['data'].shape)
print(type(data['target']), data['target'].shape)

<class 'numpy.ndarray'> (506, 13)
<class 'numpy.ndarray'> (506,)

Whereas feature_names are just that

print(data['feature_names'])

['CRIM' 'ZN' 'INDUS' 'CHAS' 'NOX' 'RM' 'AGE' 'DIS' 'RAD' 'TAX' 'PTRATIO'
 'B' 'LSTAT']

However, making sense of those requires some inspection of the documentation (see the bottom)

Using the Data

Sklearn

Data’s already broken up by X and y so let’s assign it as such.

X = data['data']
y = data['target']

Done deal.

Pandas

A bit trickier, basically, we want to merge our X and y together

import numpy as np

values = np.c_[X, y]

Then stuff those into a DataFrame

import pandas as pd

df = pd.DataFrame(values)
df.head()

And label the data accordingly

# MEDV, the median value of owner-occupied homes in $1000's
cols = list(data['feature_names']) + ['MEDV']

df.columns = cols
df.head()

Some More Description

The DESCR key gives a pretty good overview of what we’re dealing with

print(data['DESCR'])

Boston House Prices dataset
===========================

Notes
------
Data Set Characteristics:  

    :Number of Instances: 506 

    :Number of Attributes: 13 numeric/categorical predictive

    :Median Value (attribute 14) is usually the target

    :Attribute Information (in order):
        - CRIM     per capita crime rate by town
        - ZN       proportion of residential land zoned for lots over 25,000 sq.ft.
        - INDUS    proportion of non-retail business acres per town
        - CHAS     Charles River dummy variable (= 1 if tract bounds river; 0 otherwise)
        - NOX      nitric oxides concentration (parts per 10 million)
        - RM       average number of rooms per dwelling
        - AGE      proportion of owner-occupied units built prior to 1940
        - DIS      weighted distances to five Boston employment centres
        - RAD      index of accessibility to radial highways
        - TAX      full-value property-tax rate per $10,000
        - PTRATIO  pupil-teacher ratio by town
        - B        1000(Bk - 0.63)^2 where Bk is the proportion of blacks by town
        - LSTAT    % lower status of the population
        - MEDV     Median value of owner-occupied homes in $1000's

    :Missing Attribute Values: None

    :Creator: Harrison, D. and Rubinfeld, D.L.

This is a copy of UCI ML housing dataset.
http://archive.ics.uci.edu/ml/datasets/Housing


This dataset was taken from the StatLib library which is maintained at Carnegie Mellon University.

The Boston house-price data of Harrison, D. and Rubinfeld, D.L. 'Hedonic
prices and the demand for clean air', J. Environ. Economics & Management,
vol.5, 81-102, 1978.   Used in Belsley, Kuh & Welsch, 'Regression diagnostics
...', Wiley, 1980.   N.B. Various transformations are used in the table on
pages 244-261 of the latter.

The Boston house-price data has been used in many machine learning papers that address regression
problems.   

**References**

   - Belsley, Kuh & Welsch, 'Regression diagnostics: Identifying Influential Data and Sources of Collinearity', Wiley, 1980. 244-261.
   - Quinlan,R. (1993). Combining Instance-Based and Model-Based Learning. In Proceedings on the Tenth International Conference of Machine Learning, 236-243, University of Massachusetts, Amherst. Morgan Kaufmann.
   - many more! (see http://archive.ics.uci.edu/ml/datasets/Housing)