[빅분기 실기] 앙상블 스태킹 (Stacking)

앙상블 스태킹 (Stacking)

데이터셋이 아니라 여러 학습기에서 예측한 예측값으로
다시 학습 데이터를 만들어 일반화된 최종 모델을 구성하는 방법

 

 

[주요 하이퍼파라미터]

- estimators

 

 

 

 

---

 

 

Part 1. 분류 (Classification)

 

1. 분석 데이터 준비

import pandas as pd

# 유방암 예측 분류 데이터
data1=pd.read_csv('breast-cancer-wisconsin.csv', encoding='utf-8')

X=data1[data1.columns[1:10]]
y=data1[["Class"]]

 

 

 

1-2. train-test 데이터셋 나누기

from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test=train_test_split(X, y, stratify=y, random_state=42)

 

 

 

1-3. Min-Max 정규화

from sklearn.preprocessing import MinMaxScaler

scaler=MinMaxScaler()
scaler.fit(X_train) 

X_scaled_train=scaler.transform(X_train)
X_scaled_test=scaler.transform(X_test)

 

 

 

 

2. 기본모델 적용

 

2-1. 훈련 데이터

from sklearn.ensemble import RandomForestClassifier
from sklearn.svm import SVC
from sklearn.linear_model import LogisticRegression
from sklearn.ensemble import StackingClassifier

estimators = [('rf', RandomForestClassifier(n_estimators=10, random_state=42)),
              ('svr', SVC(random_state=42))]
model = StackingClassifier(estimators=estimators, final_estimator=LogisticRegression())
model.fit(X_scaled_train, y_train)

pred_train=model.predict(X_scaled_train)
model.score(X_scaled_train, y_train)  # 0.986328125

 

 

 

① 오차행렬 (confusion matrix)

from sklearn.metrics import confusion_matrix

confusion_train=confusion_matrix(y_train, pred_train)
print("훈련데이터 오차행렬:\n", confusion_train)

정상(0) 중 3명이 오분류, 환자(1) 중 4명이 오분류되었다.

 

 

 

② 분류예측 레포트 (classification report)

from sklearn.metrics import classification_report

cfreport_train=classification_report(y_train, pred_train)
print("분류예측 레포트:\n", cfreport_train)

정밀도(precision) = 0.99, 재현율(recall) = 0.98

 

 

 

2-2. 테스트 데이터

pred_test=model.predict(X_scaled_test)
model.score(X_scaled_test, y_test)  # 0.9649122807017544

 

 

 

① 오차행렬 (confusion matrix)

confusion_test=confusion_matrix(y_test, pred_test)
print("테스트데이터 오차행렬:\n", confusion_test)

정상(0) 중 5명이 오분류, 환자(1) 중 1명이 오분류되었다.

 

 

 

② 분류예측 레포트 (classification report)

cfreport_test=classification_report(y_test, pred_test)
print("분류예측 레포트:\n", cfreport_test)

정밀도(precision) = 0.96, 재현율(recall) = 0.97

 

 

 

 

---

 

 

Part 2. 회귀 (Regression)

 

1. 분석 데이터 준비

# 주택 가격 데이터
data2=pd.read_csv('house_price.csv', encoding='utf-8')

X=data2[data2.columns[1:5]]
y=data2[["house_value"]]

 

 

 

1-2. train-test 데이터셋 나누기

from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test=train_test_split(X, y, random_state=42)

 

 

 

1-3. Min-Max 정규화

from sklearn.preprocessing import MinMaxScaler

scaler=MinMaxScaler()
scaler.fit(X_train)

X_scaled_train=scaler.transform(X_train)
X_scaled_test=scaler.transform(X_test)

 

 

 

 

2. 기본모델 적용

 

2-1. 훈련 데이터

from sklearn.linear_model import LinearRegression
from sklearn.neighbors import KNeighborsRegressor
from sklearn.ensemble import RandomForestRegressor
from sklearn.ensemble import StackingRegressor

estimators = [('lr', LinearRegression()), 
              ('knn', KNeighborsRegressor())]
model = StackingRegressor(estimators=estimators, 
                          final_estimator=RandomForestRegressor(n_estimators=10, random_state=42))
model.fit(X_scaled_train, y_train)

pred_train=model.predict(X_scaled_train)
model.score(X_scaled_train, y_train)  # 0.543404932348004

 

 

 

2-2. 테스트 데이터

pred_test=model.predict(X_scaled_test)
model.score(X_scaled_test, y_test)  # 0.4781188528801523

 

 

 

① RMSE (Root Mean Squared Error)

import numpy as np
from sklearn.metrics import mean_squared_error 

MSE_train = mean_squared_error(y_train, pred_train)
MSE_test = mean_squared_error(y_test, pred_test)

print("훈련   데이터 RMSE:", np.sqrt(MSE_train))
print("테스트 데이터 RMSE:", np.sqrt(MSE_test))