机器学习 决策树篇——解决离散变量的分类问题

2021年9月23日 8点热度 0条评论 来源: Cyrus_May

机器学习 决策树篇——解决离散变量的分类问题

摘要

本文通过python实现了熵增益熵增益率的计算、实现了离散变量的决策树模型,并将代码进行了封装,方便读者调用。

熵增益和熵增益率计算

此对象用于计算离散变量的熵、条件熵、熵增益(互信息)和熵增益率
.cal_entropy():计算熵的函数
.cal_conditional_entropy():计算条件熵的函数
.cal_entropy_gain():计算熵增益(互信息)的函数
.cal_entropy_gain_ratio():计算熵增益率的函数
用法:先传入特征和标注创建对象,再调用相关函数计算就行
特征和标注的类型最好转入DataFrame、Series或者list格式
若想计算单个变量的熵,则特征和标注传同一个值就行

import numpy as np
import pandas as pd
import copy
from sklearn.preprocessing import LabelEncoder
from sklearn.datasets import  load_wine,load_breast_cancer

class CyrusEntropy(object):
    """
    此对象用于计算离散变量的熵、条件熵、熵增益(互信息)和熵增益率
    .cal_entropy():计算熵的函数
    .cal_conditional_entropy():计算条件熵的函数
    .cal_entropy_gain:计算熵增益(互信息)的函数
    .cal_entropy_gain_ratio():计算熵增益率的函数
    用法:先传入特征和标注创建对象,再调用相关函数计算就行
          特征和标注的类型最好转入DataFrame、Series或者list格式
          若想计算单个变量的熵,则特征和标注传同一个值就行
    """
    def __init__(self,x,y):
        # 特征进行标签编码
        x = pd.DataFrame(x)
        y = pd.Series(y)
        x0 = copy.copy(x)
        y = copy.copy(y)
        for i in range(x.shape[1]):
            x0.iloc[:,i] = LabelEncoder().fit_transform(x.iloc[:,i])
        self.X = x0
        self.Y = pd.Series(LabelEncoder().fit_transform(y))
        
    def cal_entropy(self):
        x_entropy = []
        for i in range(self.X.shape[1]):
            number = np.array(self.X.iloc[:,i].value_counts())
            p = number/number.sum()
            x_entropy.append(np.sum(-p*np.log2(p)))
        number = np.array(self.Y.value_counts())
        p = number/number.sum()
        y_entropy = np.sum(-p*np.log2(p))
        return x_entropy,y_entropy
    
    def cal_conditional_entropy(self):
        y_x_conditional_entropy = []
        for i in range(self.X.shape[1]):
            dict_flag = { }
            list_flag = []
            for j in range(self.X.shape[0]):
                dict_flag[self.X.iloc[j,i]] = dict_flag.get(self.X.iloc[j,i],list_flag) + [self.Y.iloc[j]]
            condition_value = 0
            for y_value in dict_flag.values():
                number = np.array(pd.Series(y_value).value_counts())
                p = number/number.sum()
                condition_value += np.sum(-p*np.log2(p))*len(y_value)/(self.Y.shape[0])
            y_x_conditional_entropy.append(condition_value)
        return y_x_conditional_entropy
                
    def cal_entropy_gain(self):
        return list(np.array(self.cal_entropy()[1])-np.array(self.cal_conditional_entropy()))
        
    def cal_entropy_gain_ratio(self):
        return list(np.array(self.cal_entropy_gain())/np.array(self.cal_entropy()[0]))

熵增益和熵增益率运行结果

使用kaggle上的一份离散变量数据进行模型验证,以下是kaggle上的数据描述:
The Lifetime reality television show and social experiment, Married at First Sight, features men and women who sign up to marry a complete stranger they’ve never met before. Experts pair couples based on tests and interviews. After marriage, couples have only a few short weeks together to decide if they want to stay married or get a divorce. There have been 10 full seasons so far which provides interesting data to look at what factors may or may not play a role in their decisions at the end of eight weeks as well as longer-term outcomes since the show aired.

if __name__ == "__main__":
    data = pd.read_csv("./mafs.csv",header=0)
    Y = data.Status
    X = data.drop(labels="Couple",axis=1)
    X = X.drop(labels="Status",axis=1)
    print(X.head(2))

建立求取信息熵对象
求取各特征和标注的信息熵

# 建立求取信息熵对象
entropy_model = CyrusEntropy(X,Y)
# 求取各特征和标注的信息熵
entropy = entropy_model.cal_entropy()
([3.29646716508619, 3.1199965768508955, 6.087462841250342, 3.520444587294042, 1.0, 6.087462841250342, 0.8739810481273578, 0.0, 0.833764907210665, 0.833764907210665, 0.833764907210665, 0.833764907210665, 0.6722948170756379, 0.8739810481273578, 0.833764907210665], 0.833764907210665)

求取标注相对各特征的条件熵

# 求取标注相对各特征的条件熵
conditon_entropy = entropy_model.cal_conditional_entropy()
print(conditon_entropy)
[0.6655644259732555, 0.699248162082863, 0.0, 0.7352336969711815, 0.833764907210665, 0.0, 0.67371811971174, 0.833764907210665, 0.7982018075321516, 0.7982018075321516, 0.7982018075321516, 0.7982018075321516, 0.8255150132281116, 0.8067159627055736, 0.8276667497383372]

求取标注相对于各特征的信息增益(互信息)

# 求取标注相对于各特征的信息增益(互信息)
entropy_gain = entropy_model.cal_entropy_gain()
print(entropy_gain)
[0.1682004812374095, 0.13451674512780198, 0.833764907210665, 0.09853121023948352, 0.0, 0.833764907210665, 0.16004678749892498, 0.0, 0.035563099678513344, 0.035563099678513344, 0.035563099678513344, 0.035563099678513344, 0.00824989398255338, 0.027048944505091432, 0.006098157472327781]

求取标注相对于各特征的信息增益率

# 求取标注相对于各特征的信息增益率
entropy_gain_rate = entropy_model.cal_entropy_gain_ratio()
print(entropy_gain_rate)
[0.05102446735064376, 0.04311438869063559, 0.1369642704939145, 0.02798828608042902, 0.0, 0.1369642704939145, 0.183123865033287, nan, 0.04265362978335057, 0.04265362978335057, 0.04265362978335057, 0.04265362978335057, 0.012271244360381867, 0.03094912019322165, 0.007314001128602282]

离散变量的决策树模型

此对象为针对离散变量的分类问题建立决策树模型适用的。
.fit():拟合及训练模型的函数
.predict():模型预测函数
.tree_net:决策树网络
用法:先调用类创建实例对象,再调用fit函数训练模型,
再调用predict函数进行预测,且可通过tree_net属性查看决策树网络。
特征和标注的类型最好转入DataFrame、Series或者list格式

class CyrusDecisionTreeDiscrete(object):
    """
    此对象为针对离散变量的分类问题建立决策树模型适用的。
    .fit():拟合及训练模型的函数
    .predict():模型预测函数
    .tree_net:决策树网络
    用法:先调用类创建实例对象,再调用fit函数训练模型,
          再调用predict函数进行预测,且可通过tree_net属性查看决策树网络。
          特征和标注的类型最好转入DataFrame、Series或者list格式
    """
    X = None
    Y = None
    def __init__(self,algorithm = "ID3"):
        self.method = algorithm
        self.tree_net = { }
    def tree(self,x,y,dict_):
        entropy_model = CyrusEntropy(x,y)
        index = np.argmax(entropy_model.cal_entropy_gain())
        dict_[index] = { }
        dict_x_flag = { }
        dict_y_flag = { }
        for i in range(x.shape[0]):
            dict_x_flag[x.iloc[i,index]] = dict_x_flag.get(x.iloc[i,index],[]) + [list(x.iloc[i,:])]
            dict_y_flag[x.iloc[i,index]] = dict_y_flag.get(x.iloc[i,index],[]) + [(y.iloc[i])]
        key_list = []
        for key,value in dict_x_flag.items():
            if pd.Series(dict_y_flag[key]).value_counts().shape[0] == 1:
                dict_[index][key] = dict_y_flag[key][0]
            else:
                key_list.append(key)
                dict_[index][key] = { }
        code = ""
        if len(key_list) != 0:  
            for key in key_list:
                code += "self.tree(pd.DataFrame(dict_x_flag['{}']),pd.Series(dict_y_flag['{}']),dict_[{}]['{}']),".format(key,key,index,key)
            code = code[:-1]
            return eval(code)
    def fit(self,x,y):
        self.X = pd.DataFrame(x)
        self.Y = pd.Series(y)
        self.tree(self.X,self.Y,self.tree_net)
    def cal_label(self,x,dict_):
        index = list(dict_.keys())[0]
        if str(type(dict_[index][x[index]])) != "<class 'dict'>":
            return dict_[index][x[index]]
        else:
            return self.cal_label(x,dict_[index][x[index]])
        
    def predict(self,x):
        x = pd.DataFrame(x)
        y = []
        for i in range(x.shape[0]):
            se = pd.Series(x.iloc[i,:])
            y.append(self.cal_label(se,self.tree_net))
        return y
    

决策树模型运行结果

建立决策树模型
训练并拟合模型
模型预测

# 建立决策树模型
tree_model = CyrusDecisionTreeDiscrete()

# 训练并拟合模型
tree_model.fit(X,Y)

# 模型预测
y_pre = tree_model.predict(X)
print(y_pre)
['Married', 'Married', 'Divorced', 'Divorced', 'Divorced', 'Divorced', 'Divorced', 'Divorced', 'Divorced', 'Divorced', 'Divorced', 'Divorced', 'Divorced', 'Divorced', 'Divorced', 'Divorced', 'Divorced', 'Divorced', 'Divorced', 'Divorced', 'Divorced', 'Divorced', 'Divorced', 'Divorced', 'Divorced', 'Divorced', 'Divorced', 'Divorced', 'Married', 'Married', 'Married', 'Married', 'Divorced', 'Divorced', 'Divorced', 'Divorced', 'Divorced', 'Divorced', 'Married', 'Married', 'Divorced', 'Divorced', 'Married', 'Married', 'Divorced', 'Divorced', 'Divorced', 'Divorced', 'Married', 'Married', 'Divorced', 'Divorced', 'Married', 'Married', 'Married', 'Married', 'Divorced', 'Divorced', 'Divorced', 'Divorced', 'Married', 'Married', 'Divorced', 'Divorced', 'Divorced', 'Divorced', 'Divorced', 'Divorced']

准确率检测

# 准确率检测
result = [1 if y_pre[i] == Y[i] else 0 for i in range(len(y_pre))]
print("准确率为:",np.array(result).sum()/len(result))
准确率为: 1.0

by CyrusMay 2020 05 20

时间如果可以倒流
我想我还是
会卯起来蹉跎
反正就这样吧
我知道我
努力过
——————五月天(一颗苹果)——————

    原文作者:Cyrus_May
    原文地址: https://blog.csdn.net/Cyrus_May/article/details/106225293
    本文转自网络文章,转载此文章仅为分享知识,如有侵权,请联系管理员进行删除。