ML之catboost:catboost模型中常用的Pool类型数据结构源代码解读、案例应用之详细攻略

ML之catboost:catboost模型中常用的Pool类型数据结构源代码解读、案例应用之详细攻略

 

 

 

 

目录

catboost模型中常用的Pool类型数据源结构代码解读

Pool简介

Pool的案例应用

Pool源代码解读


 

 

 

catboost模型中常用的Pool类型数据源结构代码解读

Pool简介

Pool 是在CatBoost中用作训练模型的数据结构

 

 

Pool的案例应用

train_dataPool = Pool(data=[[12, 14, 16, 18], [23, 25, 27, 29], [32, 34, 36, 38]],
                  label=[10, 20, 30],
                  weight=[0.1, 0.2, 0.3])    # weight各自权重默认设置为1

 

 

 

Pool源代码解读

class Pool Found at: catboost.core

class Pool(_PoolBase):
    """
    Pool used in CatBoost as a data structure to train model from.
    """
    def __init__(
        self, 
        data, 
        label=None, 
        cat_features=None, 
        text_features=None, 
        embedding_features=None, 
        column_description=None, 
        pairs=None, 
        delimiter='\t', 
        has_header=False, 
        ignore_csv_quoting=False, 
        weight=None, 
        group_id=None, 
        group_weight=None, 
        subgroup_id=None, 
        pairs_weight=None, 
        baseline=None, 
        feature_names=None, 
        thread_count=-1):
        """
        Pool is an internal data structure that is used by CatBoost.
        You can construct Pool from list, numpy.ndarray, pandas.
         DataFrame, pandas.Series.

        Parameters
        ----------

Pool 是在CatBoost中用作训练模型的数据结构

        data : list or numpy.ndarray or pandas.DataFrame or  pandas.Series or FeaturesData or string. Data source of Pool. If list or numpy.ndarrays or pandas.DataFrame or pandas.  Series, giving 2 dimensional array like data.  If FeaturesData - see FeaturesData description for  details, 'cat_features' and 'feature_names' parameters must be equal to None in this case. If string, giving the path to the file with data in catboost  format. If path starts with "quantized://", the file has to contain  quantized dataset saved with Pool.save().

        label : list or numpy.ndarrays or pandas.DataFrame or  pandas.Series, optional (default=None). Label of the training data. If not None, giving 1 or 2 dimensional array like data   with floats. If data is a file, then label must be in the file, that is label must be equals to None  cat_features : list or numpy.ndarray, optional  (default=None). If not None, giving the list of Categ features indices or names. If it contains feature names, Pool's feature names must be defined: either by passing 'feature_names' parameter or if data is pandas.DataFrame (feature  names are initialized from it's column names). Must be None if 'data' parameter has FeaturesData type

        text_features : list or numpy.ndarray, optional . (default=None), If not None, giving the list of Text features indices or  names. If it contains feature names, Pool's feature names must 
             be defined: either by passing 'feature_names'  parameter or if data is pandas.DataFrame (feature names are initialized from it's column names).  Must be None if 'data' parameter has FeaturesData type

        embedding_features : list or numpy.ndarray, optional . (default=None). If not None, giving the list of Embedding features  indices or names. If it contains feature names, Pool's feature names must  be defined: either by passing 'feature_names' parameter or if data is pandas.DataFrame (feature names are initialized from it's column names). Must be None if 'data' parameter has FeaturesData type

        column_description : string, optional (default=None). ColumnsDescription parameter. There are several columns description types: Label,   Categ, Num, Auxiliary, DocId, Weight, Baseline, GroupId,  Timestamp.  All columns are Num as default, it's not necessary to   specify  this type of columns. Default Label column index is 0   (zero).  If None, Label column is 0 (zero) as default, all data   columns are Num as default. If string, giving the path to the file with  ColumnsDescription in column_description format.

        pairs : list or numpy.ndarray or pandas.DataFrame or string.The pairs description.  If list or numpy.ndarrays or pandas.DataFrame, giving 2  dimensional. The shape should be Nx2, where N is the pairs' count.  The first element of the pair is  the index of winner object in the training set. The  second element of the pair is the index of loser object in the training set. If string, giving the path to the file with pairs description.

        delimiter : string, optional (default='\t'). Delimiter to use for separate features in file. Should be only one symbol, otherwise would be taken  only the first character of the string.

        has_header : bool optional (default=False). If True, read column names from first line.

        ignore_csv_quoting : bool optional (default=False). If True ignore quoting '"'.

        weight : list or numpy.ndarray, optional (default=None). Weight for each instance. If not None, giving 1 dimensional array like data.

        group_id : list or numpy.ndarray, optional (default=None). group id for each instance.  If not None, giving 1 dimensional array like data.

        group_weight : list or numpy.ndarray, optional .(default=None).Group weight for each instance.  If not None, giving 1 dimensional array like data.

        subgroup_id : list or numpy.ndarray, optional . (default=None) .subgroup id for each instance. If not None, giving 1 dimensional array like data.

        pairs_weight : list or numpy.ndarray, optional .(default=None).  Weight for each pair. If not None, giving 1 dimensional array like pairs.

        baseline : list or numpy.ndarray, optional (default=None).Baseline for each instance. If not None, giving 2 dimensional array like data.

        feature_names : list or string, optional (default=None). If list - list of names for each given data_feature. If string - path with scheme for feature names data to  load. If this parameter is None and 'data' is pandas.DataFrame   feature names will be initialized  from DataFrame's column names. Must be None if 'data' parameter has FeaturesData type

        thread_count : int, optional (default=-1).Thread count for data processing. If -1, then the number of threads is set to the number of  CPU cores.  

data : 列表或numpy。ndarray或pandas.DataFrame或pandas.Series或特性数据或字符串。池的数据源。如果列表或numpy。ndarrays或pandas.DataFrame或pandas.Series,给出类似数据的二维数组。如果FeaturesData -参见FeaturesData描述的详细信息,在这种情况下'cat_features'和'feature_names'参数必须等于None。如果字符串,以catboost格式提供文件的路径。如果path以"quantized://"开头,则文件必须包含保存在Pool.save()中的量化数据集。

label : 列表或numpy。ndarrays或pandas.DataFrame或pandas.Series,可选(默认= None)。训练数据的标签。如果不是None,则给出1维或2维的数组,如带有浮点数的data。如果data是一个文件,那么label必须在文件中,即label必须等于None cat_features: list或numpy。ndarray,可选(默认= None)。如果不是None,则给出类别特征索引或名称的列表。如果包含特性名称,则必须定义池的特性名称:通过传递'feature_names'参数或如果data是pandas。DataFrame(特性名称从它的列名初始化)。如果“data”参数具有FeaturesData类型,必须为None

text_features : 列表或numpy。ndarray,可选的。(default=None),如果不是None,则给出文本特性的索引或名称。如果它包含特性名称,则池的特性名称必须
be defined: 通过传递'feature_names'参数或如果数据是pandas.DataFrame特性名称从它的列名初始化)。如果“data”参数具有FeaturesData类型,必须为None

embedding_features : list或numpy。ndarray,可选的。(默认= None)。如果不是None,则给出嵌入特性的索引或名称。如果包含特性名称,则必须定义池的特性名称:通过传递'feature_names'参数或如果data是pandas。DataFrame(特性名称从它的列名初始化)。如果“data”参数具有FeaturesData类型,必须为None

column_description : 字符串,可选(默认为None)。ColumnsDescription参数。有几个列描述类型:Label, Categ, Num, Auxiliary, DocId, Weight, Baseline, GroupId, Timestamp。默认情况下,所有列都是Num,没有必要指定这种类型的列。默认标签列索引为0(零)。如果为None,则Label column默认为0(0),所有数据列默认为Num。如果是字符串,则以column_description格式给出该文件的路径。

pairs : 列表或numpy。ndarray或pandas.DataFrame或字符串。对描述。如果列表或numpy。ndarrays或pandas.DataFrame,给出二维。形状应该是Nx2,其中N是对的计数。pair的第一个元素是训练集中赢家对象的索引。对的第二个元素是输家对象在训练集中的索引。如果是字符串,则给出文件的路径和对的描述。

delimiter : 字符串,可选(默认='\t')。用于文件中单独特性的分隔符。应该只有一个符号,否则将只取字符串的第一个字符。

has_header : bool可选(默认为False)。如果为True,则从第一行读取列名。

ignore_csv_quoting :bool可选(默认为False)。如果为真,请忽略“”。

weight :列表或numpy。ndarray,可选(默认= None)。每个实例的权重。如果不是None,则给出类似data的一维数组。

group_id : list或numpy。ndarray,可选(默认= None)。每个实例的组id。如果不是None,则给出类似data的一维数组。

group_weight : list或numpy。ndarray,可选的。(默认= None)。每个实例的组权重。如果不是None,则给出类似data的一维数组。

subgroup_id : list或numpy。ndarray,可选的。(default=None) .subgroup每个实例的id。如果不是None,则给出类似data的一维数组。

pairs_weight :列表或numpy。ndarray,可选的。(默认= None)。每双的重量。如果不是None,则给出一个一维数组。

baseline:列表或numpy。ndarray,可选(默认= None)。每个实例的基线。如果不是None,则给出像data这样的二维数组。

feature_names : 列表或字符串,可选(默认为None)。If list -每个给定data_feature的名称列表。If string - path with scheme for feature names要加载的数据。如果此参数为None且'data'为pandas.DataFrame特性名称将从DataFrame的列名初始化。如果“data”参数具有FeaturesData类型,必须为None

thread_count : nt,可选(默认=-1)。用于数据处理的线程计数。如果-1,则线程数设置为CPU核数。

        if data is not None:
            self._check_data_type(data)
            self._check_data_empty(data)
            if pairs is not None and isinstance(data, STRING_TYPES) !
             = isinstance(pairs, STRING_TYPES):
                raise CatBoostError("data and pairs parameters should 
                 be the same types.")
            if column_description is not None and not isinstance
             (data, STRING_TYPES):
                raise CatBoostError("data should be the string type if 
                 column_description parameter is specified.")
            if isinstance(data, STRING_TYPES):
                if any(v is not None for v in [cat_features, 
                 text_features, embedding_features, weight, group_id, 
                 group_weight, 
                        subgroup_id, pairs_weight, baseline, label]):
                    raise CatBoostError(
                        "cat_features, text_features, embedding_features, 
                         weight, group_id, group_weight, subgroup_id, pairs_weight, "
                        "baseline, label should have the None type when 
                         the pool is read from the file.")
                if (feature_names is not None) and (not isinstance
                 (feature_names, STRING_TYPES)):
                    raise CatBoostError(
                        "feature_names should have None or string type 
                         when the pool is read from the file.")
                self._read(data, column_description, pairs, 
                 feature_names, delimiter, has_header, ignore_csv_quoting, 
                 thread_count)
            else:
                if isinstance(data, FeaturesData):
                    if any(v is not None for v in [cat_features, 
                     text_features, embedding_features, feature_names]):
                        raise CatBoostError(
                            "cat_features, text_features, 
                             embedding_features, feature_names should have the None 
                             type"
                            " when 'data' parameter has FeaturesData type")
                elif isinstance(data, np.ndarray):
                    if (data.dtype.kind == 'f') and (cat_features is not 
                     None) and (len(cat_features) > 0):
                        raise CatBoostError("'data' is numpy array of 
                         floating point numerical type, it means no categorical features,"
                            " but 'cat_features' parameter specifies nonzero 
                             number of categorical features")
                    if (data.dtype.kind == 'f') and (text_features is not 
                     None) and (len(text_features) > 0):
                        raise CatBoostError(
                            "'data' is numpy array of floating point 
                             numerical type, it means no text features,"
                            " but 'text_features' parameter specifies 
                             nonzero number of text features")
                    if (data.dtype.kind != 'O') and (embedding_features 
                     is not None) and (len(embedding_features) > 0):
                        raise CatBoostError(
                            "'data' is numpy array of non-object type, it 
                             means no embedding features,"
                            " but 'embedding_features' parameter specifies 
                             nonzero number of embedding features")
                elif isinstance(data, scipy.sparse.spmatrix):
                    if (data.dtype.kind == 'f') and (cat_features is not 
                     None) and (len(cat_features) > 0):
                        raise CatBoostError("'data' is scipy.sparse.
                         spmatrix of floating point numerical type, it means no 
                         categorical features,"
                            " but 'cat_features' parameter specifies nonzero 
                             number of categorical features")
                    if (text_features is not None) and (len(text_features) 
                     > 0):
                        raise CatBoostError(
                            "'data' is scipy.sparse.spmatrix, it means no text 
                             features,"
                            " but 'text_features' parameter specifies 
                             nonzero number of text features")
                    if (embedding_features is not None) and (len
                     (embedding_features) > 0):
                        raise CatBoostError(
                            "'data' is scipy.sparse.spmatrix, it means no 
                             embedding features,"
                            " but 'embedding_features' parameter specifies 
                             nonzero number of embedding features")
                if isinstance(feature_names, STRING_TYPES):
                    raise CatBoostError(
                        "feature_names must be None or have non-string 
                         type when the pool is created from "
                        "python objects.")
                self._init(data, label, cat_features, text_features, 
                 embedding_features, pairs, weight, group_id, group_weight, 
                 subgroup_id, pairs_weight, baseline, feature_names, 
                 thread_count)
        super(Pool, self).__init__()
    
 

    def _check_files(self, data, column_description, pairs):
        """
        Check files existence.
        """
        if data.find('://') == -1 and not os.path.isfile(data):
            raise CatBoostError("Invalid data path='{}': file does not 
             exist.".format(data))
        if column_description is not None and column_description.
         find('://') == -1 and not os.path.isfile(column_description):
            raise CatBoostError("Invalid column_description 
             path='{}': file does not exist.".format(column_description))
        if pairs is not None and pairs.find('://') == -1 and not os.
         path.isfile(pairs):
            raise CatBoostError("Invalid pairs path='{}': file does not 
             exist.".format(pairs))
    
    def _check_delimiter(self, delimiter):
        if not isinstance(delimiter, STRING_TYPES):
            raise CatBoostError("Invalid delimiter type={} : must be 
             str().".format(type(delimiter)))
        if len(delimiter) < 1:
            raise CatBoostError("Invalid delimiter length={} : must 
             be > 0.".format(len(delimiter)))
    
    def _check_column_description_type(self, 
     column_description):
        """
        Check type of column_description parameter.
        """
        if not isinstance(column_description, STRING_TYPES):
            raise CatBoostError("Invalid column_description type={}: 
             must be str().".format(type(column_description)))
    
    def _check_string_feature_type(self, features, features_name):
        """
        Check type of cat_feature parameter.
        """
        if not isinstance(features, (list, np.ndarray)):
            raise CatBoostError("Invalid {} type={}: must be list() or 
             np.ndarray().".format(features_name, type(features)))
    
    def _check_string_feature_value(self, features, 
     features_count, features_name):
        """
        Check values in cat_feature parameter. Must be int indices.
        """
        for indx, feature in enumerate(features):
            if not isinstance(feature, INTEGER_TYPES):
                raise CatBoostError("Invalid {}[{}] = {} value type={}: 
                 must be int().".format(features_name, indx, feature, type
                 (feature)))
            if feature >= features_count:
                raise CatBoostError("Invalid {}[{}] = {} value: index must 
                 be < {}.".format(features_name, indx, feature, features_count))
    
    def _check_pairs_type(self, pairs):
        """
        Check type of pairs parameter.
        """
        if not isinstance(pairs, (list, np.ndarray, DataFrame)):
            raise CatBoostError("Invalid pairs type={}: must be list(), 
             np.ndarray() or pd.DataFrame.".format(type(pairs)))
    
    def _check_pairs_value(self, pairs):
        """
        Check values in pairs parameter. Must be int indices.
        """
        for pair_id, pair in enumerate(pairs):
            if (len(pair) != 2):
                raise CatBoostError("Length of pairs[{}] isn't equal to 
                 2.".format(pair_id))
            for i, index in enumerate(pair):
                if not isinstance(index, INTEGER_TYPES):
                    raise CatBoostError("Invalid pairs[{}][{}] = {} value 
                     type={}: must be int().".format(pair_id, i, index, type(index)))
    
    def _check_data_type(self, data):
        """
        Check type of data.
        """
        if not isinstance(data, (STRING_TYPES, ARRAY_TYPES, 
         SPARSE_MATRIX_TYPES, FeaturesData)):
            raise CatBoostError(
                "Invalid data type={}: data must be list(), np.ndarray(), 
                 DataFrame(), Series(), FeaturesData " + " scipy.sparse matrix or 
                 filename str().".format(type(data)))
    
    def _check_data_empty(self, data):
        """
        Check that data is not empty (0 objects is ok).
        note: already checked if data is FeatureType, so no need 
         to check again
        """
        if isinstance(data, STRING_TYPES):
            if not data:
                raise CatBoostError("Features filename is empty.")
        elif isinstance(data, (ARRAY_TYPES, 
         SPARSE_MATRIX_TYPES)):
            data_shape = np.shape(data)
            if len(data_shape) == 1 and data_shape[0] > 0:
                if isinstance(data[0], Iterable):
                    data_shape = tuple(data_shape + tuple([len(data
                     [0])]))
                else:
                    data_shape = tuple(data_shape + tuple([1]))
            if not len(data_shape) == 2:
                raise CatBoostError("Input data has invalid shape: {}. 
                 Must be 2 dimensional".format(data_shape))
            if data_shape[1] == 0:
                raise CatBoostError("Input data must have at least one 
                 feature")
    
    def _check_label_type(self, label):
        """
        Check type of label.
        """
        if not isinstance(label, ARRAY_TYPES):
            raise CatBoostError("Invalid label type={}: must be array 
             like.".format(type(label)))
    
    def _check_label_empty(self, label):
        """
        Check label is not empty.
        """
        if len(label) == 0:
            raise CatBoostError("Labels variable is empty.")
    
    def _check_label_shape(self, label, samples_count):
        """
        Check label length and dimension.
        """
        if len(label) != samples_count:
            raise CatBoostError("Length of label={} and length of 
             data={} is different.".format(len(label), samples_count))
    
    def _check_baseline_type(self, baseline):
        """
        Check type of baseline parameter.
        """
        if not isinstance(baseline, ARRAY_TYPES):
            raise CatBoostError("Invalid baseline type={}: must be 
             array like.".format(type(baseline)))
    
    def _check_baseline_shape(self, baseline, samples_count):
        """
        Check baseline length and dimension.
        """
        if len(baseline) != samples_count:
            raise CatBoostError("Length of baseline={} and length of 
             data={} are different.".format(len(baseline), samples_count))
        if not isinstance(baseline[0], Iterable) or isinstance(baseline
         [0], STRING_TYPES):
            raise CatBoostError("Baseline must be 2 dimensional 
             data, 1 column for each class.")
        try:
            if np.array(baseline).dtype not in (np.dtype('float'), np.
             dtype('float32'), np.dtype('int')):
                raise CatBoostError()
        except CatBoostError:
            raise CatBoostError("Invalid baseline value type={}: must 
             be float or int.".format(np.array(baseline).dtype))
    
    def _check_weight_type(self, weight):
        """
        Check type of weight parameter.
        """
        if not isinstance(weight, ARRAY_TYPES):
            raise CatBoostError("Invalid weight type={}: must be 
             array like.".format(type(weight)))
    
    def _check_weight_shape(self, weight, samples_count):
        """
        Check weight length.
        """
        if len(weight) != samples_count:
            raise CatBoostError("Length of weight={} and length of 
             data={} are different.".format(len(weight), samples_count))
        if not isinstance(weight[0], (INTEGER_TYPES, 
         FLOAT_TYPES)):
            raise CatBoostError("Invalid weight value type={}: must 
             be 1 dimensional data with int, float or long types.".format(type
             (weight[0])))
    
    def _check_group_id_type(self, group_id):
        """
        Check type of group_id parameter.
        """
        if not isinstance(group_id, ARRAY_TYPES):
            raise CatBoostError("Invalid group_id type={}: must be 
             array like.".format(type(group_id)))
    
    def _check_group_id_shape(self, group_id, samples_count):
        """
        Check group_id length.
        """
        if len(group_id) != samples_count:
            raise CatBoostError("Length of group_id={} and length 
             of data={} are different.".format(len(group_id), samples_count))
    
    def _check_group_weight_type(self, group_weight):
        """
        Check type of group_weight parameter.
        """
        if not isinstance(group_weight, ARRAY_TYPES):
            raise CatBoostError("Invalid group_weight type={}: must 
             be array like.".format(type(group_weight)))
    
    def _check_group_weight_shape(self, group_weight, 
     samples_count):
        """
        Check group_weight length.
        """
        if len(group_weight) != samples_count:
            raise CatBoostError("Length of group_weight={} and 
             length of data={} are different.".format(len(group_weight), 
             samples_count))
        if not isinstance(group_weight[0], (FLOAT_TYPES)):
            raise CatBoostError("Invalid group_weight value type={}: 
             must be 1 dimensional data with float types.".format(type
             (group_weight[0])))
    
    def _check_subgroup_id_type(self, subgroup_id):
        """
        Check type of subgroup_id parameter.
        """
        if not isinstance(subgroup_id, ARRAY_TYPES):
            raise CatBoostError("Invalid subgroup_id type={}: must 
             be array like.".format(type(subgroup_id)))
    
    def _check_subgroup_id_shape(self, subgroup_id, 
     samples_count):
        """
        Check subgroup_id length.
        """
        if len(subgroup_id) != samples_count:
            raise CatBoostError("Length of subgroup_id={} and 
             length of data={} are different.".format(len(subgroup_id), 
             samples_count))
    
    def _check_feature_names(self, feature_names, 
     num_col=None):
        if num_col is None:
            num_col = self.num_col()
        if not isinstance(feature_names, Sequence):
            raise CatBoostError("Invalid feature_names type={} : 
             must be list".format(type(feature_names)))
        if len(feature_names) != num_col:
            raise CatBoostError("Invalid length of feature_names={} : 
             must be equal to the number of columns in data={}".format(len
             (feature_names), num_col))
    
    def _check_thread_count(self, thread_count):
        if not isinstance(thread_count, INTEGER_TYPES):
            raise CatBoostError("Invalid thread_count type={} : must 
             be int".format(type(thread_count)))

 

    def slice(self, rindex):
        if not isinstance(rindex, ARRAY_TYPES):
            raise CatBoostError("Invalid rindex type={} : must be list 
             or numpy.ndarray".format(type(rindex)))
        slicedPool = Pool(None)
        slicedPool._take_slice(self, rindex)
        return slicedPool
    
    def set_pairs(self, pairs):
        self._check_pairs_type(pairs)
        if isinstance(pairs, DataFrame):
            pairs = pairs.values
        self._check_pairs_value(pairs)
        self._set_pairs(pairs)
        return self
    
    def set_feature_names(self, feature_names):
        self._check_feature_names(feature_names)
        self._set_feature_names(feature_names)
        return self
    
    def set_baseline(self, baseline):
        self._check_baseline_type(baseline)
        baseline = self._if_pandas_to_numpy(baseline)
        baseline = np.reshape(baseline, (self.num_row(), -1))
        self._check_baseline_shape(baseline, self.num_row())
        self._set_baseline(baseline)
        return self
    
    def set_weight(self, weight):
        self._check_weight_type(weight)
        weight = self._if_pandas_to_numpy(weight)
        self._check_weight_shape(weight, self.num_row())
        self._set_weight(weight)
        return self
    
    def set_group_id(self, group_id):
        self._check_group_id_type(group_id)
        group_id = self._if_pandas_to_numpy(group_id)
        self._check_group_id_shape(group_id, self.num_row())
        self._set_group_id(group_id)
        return self
    
    def set_group_weight(self, group_weight):
        self._check_group_weight_type(group_weight)
        group_weight = self._if_pandas_to_numpy(group_weight)
        self._check_group_weight_shape(group_weight, self.
         num_row())
        self._set_group_weight(group_weight)
        return self
    
    def set_subgroup_id(self, subgroup_id):
        self._check_subgroup_id_type(subgroup_id)
        subgroup_id = self._if_pandas_to_numpy(subgroup_id)
        self._check_subgroup_id_shape(subgroup_id, self.
         num_row())
        self._set_subgroup_id(subgroup_id)
        return self
    
    def set_pairs_weight(self, pairs_weight):
        self._check_weight_type(pairs_weight)
        pairs_weight = self._if_pandas_to_numpy(pairs_weight)
        self._check_weight_shape(pairs_weight, self.num_pairs())
        self._set_pairs_weight(pairs_weight)
        return self
    
    def save(self, fname):
        """
        Save the quantized pool to a file.        Parameters
        ----------
        fname : string
            Output file name.
        """
        if not self.is_quantized():
            raise CatBoostError('Pool is not quantized')
        if not isinstance(fname, STRING_TYPES):
            raise CatBoostError("Invalid fname type={}: must be 
             str().".format(type(fname)))
        self._save(fname)

 

    def quantize(self, ignored_features=None, 
     per_float_feature_quantization=None, border_count=None, 
        max_bin=None, feature_border_type=None, 
         sparse_features_conflict_fraction=None, 
        nan_mode=None, input_borders=None, task_type=None, 
         used_ram_limit=None, random_seed=None, **kwargs):
        """
        Quantize this pool

        Parameters
        ----------
        pool : catboost.Pool
            Dataset to quantize.

        ignored_features : list, [default=None]
            Indices or names of features that should be excluded 
             when training.

        per_float_feature_quantization : list of strings, 
         [default=None]
            List of float binarization descriptions.
            Format : described in documentation on catboost.ai
            Example 1: ['0:1024'] means that feature 0 will have 1024 
             borders.
            Example 2: ['0:border_count=1024', '1:
             border_count=1024', ...] means that two first features have 
             1024 borders.
            Example 3: ['0:nan_mode=Forbidden,border_count=32,
             border_type=GreedyLogSum',
                        '1:nan_mode=Forbidden,border_count=32,
                         border_type=GreedyLogSum'] - defines more quantization 
                         properties for first two features.

        border_count : int, [default = 254 for training on CPU or 
         128 for training on GPU]
            The number of partitions in numeric features 
             binarization. Used in the preliminary calculation.
            range: [1,65535] on CPU, [1,255] on GPU

        max_bin : float, synonym for border_count.

        feature_border_type : string, [default='GreedyLogSum']
            The binarization mode in numeric features binarization. 
             Used in the preliminary calculation.
            Possible values:
                - 'Median'
                - 'Uniform'
                - 'UniformAndQuantiles'
                - 'GreedyLogSum'
                - 'MaxLogSum'
                - 'MinEntropy'

        sparse_features_conflict_fraction : float, [default=0.0]
            CPU only. Maximum allowed fraction of conflicting non-
             default values for features in exclusive features bundle.
            Should be a real value in [0, 1) interval.

        nan_mode : string, [default=None]
            Way to process missing values for numeric features.
            Possible values:
                - 'Forbidden' - raises an exception if there is a missing 
                 value for a numeric feature in a dataset.
                - 'Min' - each missing value will be processed as the 
                 minimum numerical value.
                - 'Max' - each missing value will be processed as the 
                 maximum numerical value.
            If None, then nan_mode=Min.

        input_borders : string, [default=None]
            input file with borders used in numeric features 
             binarization.

        task_type : string, [default=None]
            The calcer type used to train the model.
            Possible values:
                - 'CPU'
                - 'GPU'

        used_ram_limit=None

        random_seed : int, [default=None]
            The random seed used for data sampling.
            If None, 0 is used.
        """
        if self.is_quantized():
            raise CatBoostError('Pool is already quantized')
        params = {}
        _process_synonyms(params)
        if border_count is None:
            border_count = max_bin
        dev_efb_max_buckets = kwargs.pop
         ('dev_efb_max_buckets', None)
        dev_max_subset_size_for_build_borders = kwargs.pop
         ('dev_max_subset_size_for_build_borders', None)
        if kwargs:
            raise CatBoostError("got an unexpected keyword 
             arguments: {}".format(kwargs.keys()))
        _update_params_quantize_part(params, ignored_features, 
         per_float_feature_quantization, border_count, 
         feature_border_type, sparse_features_conflict_fraction, 
         dev_efb_max_buckets, nan_mode, input_borders, task_type, 
         used_ram_limit, random_seed, 
         dev_max_subset_size_for_build_borders)
        self._quantize(params)
    
    def _if_pandas_to_numpy(self, array):
        if isinstance(array, Series):
            array = array.values
        if isinstance(array, DataFrame):
            array = np.transpose(array.values)[0]
        return array
    
    def _label_if_pandas_to_numpy(self, label):
        if isinstance(label, Series):
            label = label.values
        if isinstance(label, DataFrame):
            label = label.values
        return label
    
    def _read(
        self, 
        pool_file, 
        column_description, 
        pairs, 
        feature_names_path, 
        delimiter, 
        has_header, 
        ignore_csv_quoting, 
        thread_count, 
        quantization_params=None):
        """
        Read Pool from file.
        """
        with log_fixup():
            self._check_files(pool_file, column_description, pairs)
            self._check_delimiter(delimiter)
            if column_description is None:
                column_description = ''
            else:
                self._check_column_description_type
                 (column_description)
            if pairs is None:
                pairs = ''
            if feature_names_path is None:
                feature_names_path = ''
            self._check_thread_count(thread_count)
            self._read_pool(pool_file, column_description, pairs, 
             feature_names_path, delimiter[0], has_header, 
             ignore_csv_quoting, thread_count, quantization_params)
    
    def _init(
        self, 
        data, 
        label, 
        cat_features, 
        text_features, 
        embedding_features, 
        pairs, weight, 
        group_id, 
        group_weight, 
        subgroup_id, 
        pairs_weight, 
        baseline, 
        feature_names, 
        thread_count):
        """
        Initialize Pool from array like data.
        """
        if isinstance(data, DataFrame):
            if feature_names is None:
                feature_names = list(data.columns)
        if isinstance(data, Series):
            data = data.values.tolist()
        if isinstance(data, FeaturesData):
            samples_count = data.get_object_count()
            features_count = data.get_feature_count()
        else:
            if len(np.shape(data)) == 1:
                data = np.expand_dims(data, 1)
            samples_count, features_count = np.shape(data)
        pairs_len = 0
        if label is not None:
            self._check_label_type(label)
            self._check_label_empty(label)
            label = self._label_if_pandas_to_numpy(label)
            if len(np.shape(label)) == 1:
                label = np.expand_dims(label, 1)
            self._check_label_shape(label, samples_count)
        if feature_names is not None:
            self._check_feature_names(feature_names, 
             features_count)
        if cat_features is not None:
            cat_features = _get_features_indices(cat_features, 
             feature_names)
            self._check_string_feature_type(cat_features, 
             'cat_features')
            self._check_string_feature_value(cat_features, 
             features_count, 'cat_features')
        if text_features is not None:
            text_features = _get_features_indices(text_features, 
             feature_names)
            self._check_string_feature_type(text_features, 
             'text_features')
            self._check_string_feature_value(text_features, 
             features_count, 'text_features')
        if embedding_features is not None:
            embedding_features = _get_features_indices
             (embedding_features, feature_names)
            self._check_string_feature_type(embedding_features, 
             'embedding_features')
            self._check_string_feature_value(embedding_features, 
             features_count, 'embedding_features')
        if pairs is not None:
            self._check_pairs_type(pairs)
            if isinstance(pairs, DataFrame):
                pairs = pairs.values
            self._check_pairs_value(pairs)
            pairs_len = np.shape(pairs)[0]
        if weight is not None:
            self._check_weight_type(weight)
            weight = self._if_pandas_to_numpy(weight)
            self._check_weight_shape(weight, samples_count)
        if group_id is not None:
            self._check_group_id_type(group_id)
            group_id = self._if_pandas_to_numpy(group_id)
            self._check_group_id_shape(group_id, samples_count)
        if group_weight is not None:
            self._check_group_weight_type(group_weight)
            group_weight = self._if_pandas_to_numpy
             (group_weight)
            self._check_group_weight_shape(group_weight, 
             samples_count)
        if subgroup_id is not None:
            self._check_subgroup_id_type(subgroup_id)
            subgroup_id = self._if_pandas_to_numpy(subgroup_id)
            self._check_subgroup_id_shape(subgroup_id, 
             samples_count)
        if pairs_weight is not None:
            self._check_weight_type(pairs_weight)
            pairs_weight = self._if_pandas_to_numpy(pairs_weight)
            self._check_weight_shape(pairs_weight, pairs_len)
        if baseline is not None:
            self._check_baseline_type(baseline)
            baseline = self._if_pandas_to_numpy(baseline)
            baseline = np.reshape(baseline, (samples_count, -1))
            self._check_baseline_shape(baseline, samples_count)
        self._init_pool(data, label, cat_features, text_features, 
         embedding_features, pairs, weight, group_id, group_weight, 
         subgroup_id, pairs_weight, baseline, feature_names, 
         thread_count)

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

一个处女座的程序猿 CSDN认证博客专家 华为杯研电赛一等 华为研数模一等奖 国内外AI竞十
人工智能硕博生,拥有十多项发明专利(6项)和软件著作权(9项),多个国家级证书(2个国三级、3个国四级),先后获得国内外“人工智能算法”竞赛(包括国家级、省市级等,一等奖5项、二等奖4项、三等奖2项)相关证书十多个,以上均以第一作者身份,并拥有省市校级个人荣誉证书十多项。正在撰写《人工智算法最新实战》一书,目前已37万字。
已标记关键词 清除标记
相关推荐
©️2020 CSDN 皮肤主题: 代码科技 设计师:Amelia_0503 返回首页
实付 29.90元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、C币套餐、付费专栏及课程。

余额充值