Merging results from model.predict() with original pandas DataFrame? : <p>I am trying to merge the results of a <code>predict</code> method back with the original data in a <code>pandas.DataFrame</code> object.</p>

<pre><code>from sklearn.datasets import load_iris
from sklearn.cross_validation import train_test_split
from sklearn.tree import DecisionTreeClassifier
import pandas as pd
import numpy as np

data = load_iris()

# bear with me for the next few steps... I'm trying to walk you through
# how my data object landscape looks... i.e. how I get from raw data 
# to matrices with the actual data I have, not the iris dataset
# put feature matrix into columnar format in dataframe
df = pd.DataFrame(data = data.data)

# add outcome variable
df['class'] = data.target

X = np.matrix(df.loc[:, [0, 1, 2, 3]])
y = np.array(df['class'])

# finally, split into train-test
X_train, X_test, y_train, y_test = train_test_split(X, y, train_size = 0.8)

model = DecisionTreeClassifier()

model.fit(X_train, y_train)

# I've got my predictions now
y_hats = model.predict(X_test)
</code></pre>

<p>To merge these predictions back with the original <code>df</code>, I try this:</p>

<pre><code>df['y_hats'] = y_hats
</code></pre>

<p>But that raises:</p>

<blockquote>
  <p>ValueError: Length of values does not match length of index</p>
</blockquote>

<p>I know I could split the <code>df</code> into <code>train_df</code> and <code>test_df</code> and this problem would be solved, but in reality I need to follow the path above to create the matrices <code>X</code> and <code>y</code> (my actual problem is a text classification problem in which I normalize the <em>entire</em> feature matrix before splitting into train and test). How can I align these predicted values with the appropriate rows in my <code>df</code>, since the <code>y_hats</code> array is zero-indexed and seemingly all information about <em>which</em> rows were included in the <code>X_test</code> and <code>y_test</code> is lost? Or will I be relegated to splitting dataframes into train-test first, and then building feature matrices? I'd like to just fill the rows included in <code>train</code> with <code>np.nan</code> values in the dataframe.</p>
