Question: Suggestion: Add support for unpenalized logistic regression provides unpenalized OLS, and , which supports , also supports . But if you want plain old unpenalized logistic regression, you have to fake it by setting in to a large number, or use from instead.

prefer = 'processes'         fold_coefs_ = Parallel(n_jobs=self.n_jobs, verbose=self.verbose,                                **_joblib_parallel_args(prefer=prefer))(            path_func(X, y, pos_class=class_, Cs=[self.C],            path_func(X, y, pos_class=class_, Cs=[C_],                       l1_ratio=self.l1_ratio, fit_intercept=self.fit_intercept,                       tol=self.tol, verbose=self.verbose, solver=solver,                       multi_class=multi_class, max_iter=self.max_iter,                       class_weight=self.class_weight, check_input=False,                       random_state=self.random_state, coef=warm_start_coef_,                      penalty=self.penalty, max_squared_sum=max_squared_sum,                      penalty=penalty, max_squared_sum=max_squared_sum,                       sample_weight=sample_weight)             for class_, warm_start_coef_ in zip(classes_, warm_start_coef))


Question: Suggestion: Add support for unpenalized logistic regression provides unpenalized OLS, and , which supports , also supports . But if you want plain old unpenalized logistic regression, you have to fake it by setting in to a large number, or use from instead.

l1_ratios_ = [None] <ins>        if self.penalty == 'none':            raise ValueError(                "penalty='none' is not useful and not supported by "                "LogisticRegressionCV."            )</ins>         X, y = check_X_y(X, y, accept_sparse='csr', dtype=np.float64,                          order="C",                          accept_large_sparse=solver != 'liblinear')


Question: AttributeError: 'PrettyPrinter' object has no attribute 'indentat_name' There's a failing example in , and here's a piece of code causing it: With the stack trace:

self._format(v, stream, indent + len(rep) + len(middle), allowance,                      context, level)     _dispatch = pprint.PrettyPrinter._dispatch    # Note: need to copy _dispatch to prevent instances of the builtin    # PrettyPrinter class to call methods of _EstimatorPrettyPrinter (see issue    # 12906)    _dispatch = pprint.PrettyPrinter._dispatch.copy()     _dispatch[BaseEstimator.__repr__] = _pprint_estimator     _dispatch[KeyValTuple.__repr__] = _pprint_key_val_tuple


Question: ValueError not enough values to unpack (expected 5, got 0) I'm using with in sklearn v 0.20.1. MacOS Mojave. The issue traces to these two lines below, with Setting did not work, still hitting the same issue. From the trackback, I suspect that the line below could be returning None. This doesn't always happen, but only sometimes. My param search was able to run for a few rounds, and then hit this issue. The problem could be that lightgbm did not return valid results (have not verified), but either way, this error could be handled better. Does anyone have more insight into the potential cause? Thanks.

in product(candidate_params,                                           cv.split(X, y, groups))) <ins>                if len(out) < 1:                    raise ValueError('No fits were performed. '                                     'Was the CV iterator empty? '                                     'Were there no candidates?')                elif len(out) != n_candidates * n_splits:                    raise ValueError('cv.split and cv.get_n_splits returned '                                     'inconsistent results. Expected {} '                                     'splits, got {}'                                     .format(n_splits,                                             len(out) // n_candidates))</ins>                 all_candidate_params.extend(candidate_params)                 all_out.extend(out)


Question: LassoLarsIC: unintuitive copyX behaviour Hi, I would like to report what seems to be a bug in the treatment of the parameter of the class. Because it's a simple bug, it's much easier to see in the code directly than in the execution, so I am not posting steps to reproduce it. As you can see here, LassoLarsIC accepts a copyX parameter. However, it also takes a copyX parameter a few lines below, in the definition of . Now there are two values (potentially contradicting each other) for copyX and each one is used once. Therefore can have a mixed behaviour. Even worse, this can be completely invisible to the user, since copyX has a default value of True. Let's assume that I'd like it to be False, and have set it to False in the initialization, . I then call and my choice will be silently overwritten. Ideally I think that copy_X should be removed as an argument in . No other estimator seems to have a duplication in class parameters and fit arguments (I've checked more than ten in the linear models module). However, this would break existing code. Therefore I propose that takes a default value of and only overwrites the existing value if the user has explicitly passed it as an argument to . I will submit a PR to that effect.

self.eps = eps         self.fit_path = True     def fit(self, X, y, copy_X=True):    def fit(self, X, y, copy_X=None):         """Fit the model using X, y as training data.          Parameters


Question: LassoLarsIC: unintuitive copyX behaviour Hi, I would like to report what seems to be a bug in the treatment of the parameter of the class. Because it's a simple bug, it's much easier to see in the code directly than in the execution, so I am not posting steps to reproduce it. As you can see here, LassoLarsIC accepts a copyX parameter. However, it also takes a copyX parameter a few lines below, in the definition of . Now there are two values (potentially contradicting each other) for copyX and each one is used once. Therefore can have a mixed behaviour. Even worse, this can be completely invisible to the user, since copyX has a default value of True. Let's assume that I'd like it to be False, and have set it to False in the initialization, . I then call and my choice will be silently overwritten. Ideally I think that copy_X should be removed as an argument in . No other estimator seems to have a duplication in class parameters and fit arguments (I've checked more than ten in the linear models module). However, this would break existing code. Therefore I propose that takes a default value of and only overwrites the existing value if the user has explicitly passed it as an argument to . I will submit a PR to that effect.

y : array-like, shape (n_samples,)             target values. Will be cast to X's dtype if necessary         copy_X : boolean, optional, default True        copy_X : boolean, optional, default None            If provided, this parameter will override the choice            of copy_X made at instance creation.             If ``True``, X will be copied; else, it may be overwritten.          Returns


Question: LassoLarsIC: unintuitive copyX behaviour Hi, I would like to report what seems to be a bug in the treatment of the parameter of the class. Because it's a simple bug, it's much easier to see in the code directly than in the execution, so I am not posting steps to reproduce it. As you can see here, LassoLarsIC accepts a copyX parameter. However, it also takes a copyX parameter a few lines below, in the definition of . Now there are two values (potentially contradicting each other) for copyX and each one is used once. Therefore can have a mixed behaviour. Even worse, this can be completely invisible to the user, since copyX has a default value of True. Let's assume that I'd like it to be False, and have set it to False in the initialization, . I then call and my choice will be silently overwritten. Ideally I think that copy_X should be removed as an argument in . No other estimator seems to have a duplication in class parameters and fit arguments (I've checked more than ten in the linear models module). However, this would break existing code. Therefore I propose that takes a default value of and only overwrites the existing value if the user has explicitly passed it as an argument to . I will submit a PR to that effect.

self : object             returns an instance of self.         """        if copy_X is None:            copy_X = self.copy_X         X, y = check_X_y(X, y, y_numeric=True)          X, y, Xmean, ymean, Xstd = LinearModel._preprocess_data(            X, y, self.fit_intercept, self.normalize, self.copy_X)            X, y, self.fit_intercept, self.normalize, copy_X)         max_iter = self.max_iter          Gram = self.precompute


Question: and have inconsistent default init <!-- If your issue is a usage question, submit it here instead: StackOverflow with the scikit-learn tag: Mailing List: For more information, see User Questions: http://scikit- --<!-- Instructions For Filing a Bug: --<!-- Example: Joblib Error thrown when calling fit on LatentDirichletAllocation with evaluateevery 0--and have inconsistent default init. has while has . See As suggested, we could change the default in with a deprecation process. <!-- Example: If the code is too long, feel free to put it in a public gist and link it in the issue: --<!-- Example: No error is thrown. Please paste or describe the expected results.--<!-- Please paste or specifically describe the actual output or traceback. --<!-- Please run the following snippet and paste the output below. For scikit-learn = 0.20: import sklearn; sklearn.showversions() For scikit-learn < 0.20: import platform; print(platform.platform()) import sys; print("Python", sys.version) import numpy; print("NumPy", numpy.version) import scipy; print("SciPy", scipy.version) import sklearn; print("Scikit-Learn", sklearn.version) --<!-- Thanks for contributing! --

init :  None | 'random' | 'nndsvd' | 'nndsvda' | 'nndsvdar'         Method used to initialize the procedure.        Default: 'nndsvd' if n_components < n_features, otherwise 'random'.        Default: None.         Valid options:         - None: 'nndsvd' if n_components < n_features, otherwise 'random'.         - 'random': non-negative random matrices, scaled with:             sqrt(X.mean() / n_components)


Question: and have inconsistent default init <!-- If your issue is a usage question, submit it here instead: StackOverflow with the scikit-learn tag: Mailing List: For more information, see User Questions: http://scikit- --<!-- Instructions For Filing a Bug: --<!-- Example: Joblib Error thrown when calling fit on LatentDirichletAllocation with evaluateevery 0--and have inconsistent default init. has while has . See As suggested, we could change the default in with a deprecation process. <!-- Example: If the code is too long, feel free to put it in a public gist and link it in the issue: --<!-- Example: No error is thrown. Please paste or describe the expected results.--<!-- Please paste or specifically describe the actual output or traceback. --<!-- Please run the following snippet and paste the output below. For scikit-learn = 0.20: import sklearn; sklearn.showversions() For scikit-learn < 0.20: import platform; print(platform.platform()) import sys; print("Python", sys.version) import numpy; print("NumPy", numpy.version) import scipy; print("SciPy", scipy.version) import sklearn; print("Scikit-Learn", sklearn.version) --<!-- Thanks for contributing! --

def non_negative_factorization(X, W=None, H=None, n_components=None,                               init='random', update_H=True, solver='cd',                               init='warn', update_H=True, solver='cd',                                beta_loss='frobenius', tol=1e-4,                                max_iter=200, alpha=0., l1_ratio=0.,                                regularization=None, random_state=None,


Question: and have inconsistent default init <!-- If your issue is a usage question, submit it here instead: StackOverflow with the scikit-learn tag: Mailing List: For more information, see User Questions: http://scikit- --<!-- Instructions For Filing a Bug: --<!-- Example: Joblib Error thrown when calling fit on LatentDirichletAllocation with evaluateevery 0--and have inconsistent default init. has while has . See As suggested, we could change the default in with a deprecation process. <!-- Example: If the code is too long, feel free to put it in a public gist and link it in the issue: --<!-- Example: No error is thrown. Please paste or describe the expected results.--<!-- Please paste or specifically describe the actual output or traceback. --<!-- Please run the following snippet and paste the output below. For scikit-learn = 0.20: import sklearn; sklearn.showversions() For scikit-learn < 0.20: import platform; print(platform.platform()) import sys; print("Python", sys.version) import numpy; print("NumPy", numpy.version) import scipy; print("SciPy", scipy.version) import sklearn; print("Scikit-Learn", sklearn.version) --<!-- Thanks for contributing! --

Number of components, if n_components is not set all features         are kept.     init :  None | 'random' | 'nndsvd' | 'nndsvda' | 'nndsvdar' | 'custom'    init : None | 'random' | 'nndsvd' | 'nndsvda' | 'nndsvdar' | 'custom'         Method used to initialize the procedure.         Default: 'random'.        The default value will change from 'random' to None in version 0.23        to make it consistent with decomposition.NMF.         Valid options:         - None: 'nndsvd' if n_components < n_features, otherwise 'random'.         - 'random': non-negative random matrices, scaled with:             sqrt(X.mean() / n_components)


Question: and have inconsistent default init <!-- If your issue is a usage question, submit it here instead: StackOverflow with the scikit-learn tag: Mailing List: For more information, see User Questions: http://scikit- --<!-- Instructions For Filing a Bug: --<!-- Example: Joblib Error thrown when calling fit on LatentDirichletAllocation with evaluateevery 0--and have inconsistent default init. has while has . See As suggested, we could change the default in with a deprecation process. <!-- Example: If the code is too long, feel free to put it in a public gist and link it in the issue: --<!-- Example: No error is thrown. Please paste or describe the expected results.--<!-- Please paste or specifically describe the actual output or traceback. --<!-- Please run the following snippet and paste the output below. For scikit-learn = 0.20: import sklearn; sklearn.showversions() For scikit-learn < 0.20: import platform; print(platform.platform()) import sys; print("Python", sys.version) import numpy; print("NumPy", numpy.version) import scipy; print("SciPy", scipy.version) import sklearn; print("Scikit-Learn", sklearn.version) --<!-- Thanks for contributing! --

raise ValueError("Tolerance for stopping criteria must be "                          "positive; got (tol=%r)" % tol) <ins>    if init == "warn":        if n_components < n_features:            warnings.warn("The default value of init will change from "                          "random to None in 0.23 to make it consistent "                          "with decomposition.NMF.", FutureWarning)        init = "random"</ins>     # check W and H, or initialize them     if init == 'custom' and update_H:         _check_init(H, (n_components, n_features), "NMF (input H)")


Question: and have inconsistent default init <!-- If your issue is a usage question, submit it here instead: StackOverflow with the scikit-learn tag: Mailing List: For more information, see User Questions: http://scikit- --<!-- Instructions For Filing a Bug: --<!-- Example: Joblib Error thrown when calling fit on LatentDirichletAllocation with evaluateevery 0--and have inconsistent default init. has while has . See As suggested, we could change the default in with a deprecation process. <!-- Example: If the code is too long, feel free to put it in a public gist and link it in the issue: --<!-- Example: No error is thrown. Please paste or describe the expected results.--<!-- Please paste or specifically describe the actual output or traceback. --<!-- Please run the following snippet and paste the output below. For scikit-learn = 0.20: import sklearn; sklearn.showversions() For scikit-learn < 0.20: import platform; print(platform.platform()) import sys; print("Python", sys.version) import numpy; print("NumPy", numpy.version) import scipy; print("SciPy", scipy.version) import sklearn; print("Scikit-Learn", sklearn.version) --<!-- Thanks for contributing! --

Number of components, if n_components is not set all features         are kept.     init :  'random' | 'nndsvd' |  'nndsvda' | 'nndsvdar' | 'custom'    init : None | 'random' | 'nndsvd' |  'nndsvda' | 'nndsvdar' | 'custom'         Method used to initialize the procedure.        Default: 'nndsvd' if n_components < n_features, otherwise random.        Default: None.         Valid options:         - None: 'nndsvd' if n_components < n_features, otherwise random.         - 'random': non-negative random matrices, scaled with:             sqrt(X.mean() / n_components)


Question: StandardScaler fit overflows on float16 When using StandardScaler on a large float16 numpy array the mean and std calculation overflows. I can convert the array to a larger precision but when working with a larger dataset the memory saved by using float16 on smaller numbers kind of matter. The error is mostly on numpy. Adding the dtype on the mean/std calculation does it but I'm not sure if that how people here would like to do it. The normalized array

return X  <ins># Use at least float64 for the accumulating functions to avoid precision issue# see https://github.com/numpy/numpy/issues/9393. The float64 is also retained# as it is in case the float overflowsdef _safe_accumulator_op(op, x, *args, **kwargs):    """    This function provides numpy accumulator functions with a float64 dtype    when used on a floating point input. This prevents accumulator overflow on    smaller floating point dtypes.    Parameters    ----------    op : function        A numpy accumulator function such as np.mean or np.sum    x : numpy array        A numpy array to apply the accumulator function    *args : positional arguments        Positional arguments passed to the accumulator function after the        input x    **kwargs : keyword arguments        Keyword arguments passed to the accumulator function    Returns    -------    result : The output of the accumulator function passed to this function    """    if np.issubdtype(x.dtype, np.floating) and x.dtype.itemsize < 8:        result = op(x, *args, **kwargs, dtype=np.float64)    else:        result = op(x, *args, **kwargs)    return result</ins> def _incremental_mean_and_var(X, last_mean, last_variance, last_sample_count):     """Calculate mean update and a Youngs and Cramer variance update.


Question: StandardScaler fit overflows on float16 When using StandardScaler on a large float16 numpy array the mean and std calculation overflows. I can convert the array to a larger precision but when working with a larger dataset the memory saved by using float16 on smaller numbers kind of matter. The error is mostly on numpy. Adding the dtype on the mean/std calculation does it but I'm not sure if that how people here would like to do it. The normalized array

# new = the current increment     # updated = the aggregated stats     last_sum = last_mean * last_sample_count    if np.issubdtype(X.dtype, np.floating) and X.dtype.itemsize < 8:        # Use at least float64 for the accumulator to avoid precision issues;        # see https://github.com/numpy/numpy/issues/9393        new_sum = np.nansum(X, axis=0, dtype=np.float64).astype(X.dtype)    else:        new_sum = np.nansum(X, axis=0)    new_sum = _safe_accumulator_op(np.nansum, X, axis=0)      new_sample_count = np.sum(~np.isnan(X), axis=0)     updated_sample_count = last_sample_count + new_sample_count


Question: StandardScaler fit overflows on float16 When using StandardScaler on a large float16 numpy array the mean and std calculation overflows. I can convert the array to a larger precision but when working with a larger dataset the memory saved by using float16 on smaller numbers kind of matter. The error is mostly on numpy. Adding the dtype on the mean/std calculation does it but I'm not sure if that how people here would like to do it. The normalized array

if last_variance is None:         updated_variance = None     else:        new_unnormalized_variance = np.nanvar(X, axis=0) * new_sample_count        new_unnormalized_variance = (            _safe_accumulator_op(np.nanvar, X, axis=0) * new_sample_count)         last_unnormalized_variance = last_variance * last_sample_count          with np.errstate(divide='ignore', invalid='ignore'):diff --git a/sklearn/utils/validation.py b/sklearn/utils/validation.py-- a/sklearn/utils/validation.py++ b/sklearn/utils/validation.py


Question: StandardScaler fit overflows on float16 When using StandardScaler on a large float16 numpy array the mean and std calculation overflows. I can convert the array to a larger precision but when working with a larger dataset the memory saved by using float16 on smaller numbers kind of matter. The error is mostly on numpy. Adding the dtype on the mean/std calculation does it but I'm not sure if that how people here would like to do it. The normalized array

def _assert_all_finite(X, allow_nan=False):     """Like assert_all_finite, but only for ndarray."""    # validation is also imported in extmath    from .extmath import _safe_accumulator_op     if _get_config()['assume_finite']:         return     X = np.asanyarray(X)     # First try an O(n) time, O(1) space solution for the common case that     # everything is finite; fall back to O(n) space np.isfinite to prevent    # false positives from overflow in sum method.    # false positives from overflow in sum method. The sum is also calculated    # safely to reduce dtype induced overflows.     is_float = X.dtype.kind in 'fc'    if is_float and np.isfinite(X.sum()):    if is_float and (np.isfinite(_safe_accumulator_op(np.sum, X))):         pass     elif is_float:         msg_err = "Input contains {} or a value too large for {!r}."


Question: ColumnTransformer behavior for negative column indexes <!-- If your issue is a usage question, submit it here instead: StackOverflow with the scikit-learn tag: Mailing List: For more information, see User Questions: http://scikit- --<!-- Instructions For Filing a Bug: --The behavior of when negative integers are passed as column indexes is not clear. The first transformer should either raise an error or give the same result as the second transformer

"""     n_columns = X.shape[1]     if _check_key_type(key, int):        if isinstance(key, int):            return [key]        elif isinstance(key, slice):            return list(range(n_columns)[key])        else:            return list(key)    if (_check_key_type(key, int)            or hasattr(key, 'dtype') and np.issubdtype(key.dtype, np.bool_)):        # Convert key into positive indexes        idx = np.arange(n_columns)[key]        return np.atleast_1d(idx).tolist()     elif _check_key_type(key, str):         try:             all_columns = list(X.columns)


Question: ColumnTransformer behavior for negative column indexes <!-- If your issue is a usage question, submit it here instead: StackOverflow with the scikit-learn tag: Mailing List: For more information, see User Questions: http://scikit- --<!-- Instructions For Filing a Bug: --The behavior of when negative integers are passed as column indexes is not clear. The first transformer should either raise an error or give the same result as the second transformer

columns = list(key)          return [all_columns.index(col) for col in columns]<del>    elif hasattr(key, 'dtype') and np.issubdtype(key.dtype, np.bool_):        # boolean mask        return list(np.arange(n_columns)[key])</del>     else:         raise ValueError("No valid specification of the columns. Only a "                          "scalar, list or slice of all integers or all "


Question: MissingIndicator failed with non-numeric inputs fails with string and object type numpy arrays

return imputer_mask, features_with_missing <ins>    def _validate_input(self, X):        if not is_scalar_nan(self.missing_values):            force_all_finite = True        else:            force_all_finite = "allow-nan"        X = check_array(X, accept_sparse=('csc', 'csr'), dtype=None,                        force_all_finite=force_all_finite)        _check_inputs_dtype(X, self.missing_values)        if X.dtype.kind not in ("i", "u", "f", "O"):            raise ValueError("MissingIndicator does not support data with "                             "dtype {0}. Please provide either a numeric array"                             " (with a floating point or integer dtype) or "                             "categorical data represented either as an array "                             "with integer dtype or an array of string values "                             "with an object dtype.".format(X.dtype))        return X</ins>     def fit(self, X, y=None):         """Fit the transformer on X.


Question: MissingIndicator failed with non-numeric inputs fails with string and object type numpy arrays

self : object             Returns self.         """        if not is_scalar_nan(self.missing_values):            force_all_finite = True        else:            force_all_finite = "allow-nan"        X = check_array(X, accept_sparse=('csc', 'csr'),                        force_all_finite=force_all_finite)        _check_inputs_dtype(X, self.missing_values)        X = self._validate_input(X)         self._n_features = X.shape[1]          if self.features not in ('missing-only', 'all'):


Question: MissingIndicator failed with non-numeric inputs fails with string and object type numpy arrays

"""         check_is_fitted(self, "features_")        if not is_scalar_nan(self.missing_values):            force_all_finite = True        else:            force_all_finite = "allow-nan"        X = check_array(X, accept_sparse=('csc', 'csr'),                        force_all_finite=force_all_finite)        _check_inputs_dtype(X, self.missing_values)        X = self._validate_input(X)          if X.shape[1] != self._n_features:             raise ValueError("X has a different number of features "diff --git a/sklearn/utils/estimator_checks.py b/sklearn/utils/estimator_checks.py-- a/sklearn/utils/estimator_checks.py++ b/sklearn/utils/estimator_checks.py


Question: MissingIndicator failed with non-numeric inputs fails with string and object type numpy arrays

'OrthogonalMatchingPursuit', 'PLSCanonical', 'PLSRegression',                 'RANSACRegressor', 'RadiusNeighborsRegressor',                 'RandomForestRegressor', 'Ridge', 'RidgeCV'] ALLOW_NAN = ['Imputer', 'SimpleImputer', 'MissingIndicator',              'MaxAbsScaler', 'MinMaxScaler', 'RobustScaler', 'StandardScaler',              'PowerTransformer', 'QuantileTransformer']SUPPORT_STRING = ['SimpleImputer', 'MissingIndicator']   def _yield_non_meta_checks(name, estimator):


Question: MissingIndicator failed with non-numeric inputs fails with string and object type numpy arrays

if "Unknown label type" not in str(e):             raise     X[0, 0] = {'foo': 'bar'}    msg = "argument must be a string or a number"    assert_raises_regex(TypeError, msg, estimator.fit, X, y)    if name not in SUPPORT_STRING:        X[0, 0] = {'foo': 'bar'}        msg = "argument must be a string or a number"        assert_raises_regex(TypeError, msg, estimator.fit, X, y)    else:        # Estimators supporting string will not call np.asarray to convert the        # data to numeric and therefore, the error will not be raised.        # Checking for each element dtype in the input array will be costly.        # Refer to #11401 for full discussion.        estimator.fit(X, y)   def check_complex_data(name, estimator_orig):


Question: Feature request: support for arbitrary bin spacing in calibration.calibrationcurve I was using , and it currently accepts an parameter to specify the number of bins to evenly partition the probability space between 0 and 1. However, I am using this in combination with a gradient boosting model in which the probabilities are very uncalibrated, and most of the predictions are close to 0. When I use the calibrated classifier, the result is very noisy because there are many data points in some bins and few, if any, in others (see example below). In the code below, I made a work-around to do what I want and show a plot of my output (in semilog space because of the skewed distribution). I haven't contributed to a large open-source project before, but if there's agreement this would be a useful feature, I would be happy to try to draft up a PR. Notice the large disparity in results between the different bins chosen. For this reason, I think the user should be able to choose the bin edges, as in numpy's or matplotlib's functions. !

return expit(-(self.a_ * T + self.b_))  def calibration_curve(y_true, y_prob, normalize=False, n_bins=5):def calibration_curve(y_true, y_prob, normalize=False, n_bins=5,                      strategy='uniform'):     """Compute true and predicted probabilities for a calibration curve.       The method assumes the inputs come from a binary classifier.


Question: Feature request: support for arbitrary bin spacing in calibration.calibrationcurve I was using , and it currently accepts an parameter to specify the number of bins to evenly partition the probability space between 0 and 1. However, I am using this in combination with a gradient boosting model in which the probabilities are very uncalibrated, and most of the predictions are close to 0. When I use the calibrated classifier, the result is very noisy because there are many data points in some bins and few, if any, in others (see example below). In the code below, I made a work-around to do what I want and show a plot of my output (in semilog space because of the skewed distribution). I haven't contributed to a large open-source project before, but if there's agreement this would be a useful feature, I would be happy to try to draft up a PR. Notice the large disparity in results between the different bins chosen. For this reason, I think the user should be able to choose the bin edges, as in numpy's or matplotlib's functions. !

points (i.e. without corresponding values in y_prob) will not be         returned, thus there may be fewer than n_bins in the return value. <ins>    strategy : {'uniform', 'quantile'}, (default='uniform')        Strategy used to define the widths of the bins.        uniform            All bins have identical widths.        quantile            All bins have the same number of points.</ins>     Returns     -------     prob_true : array, shape (n_bins,) or smaller


Question: Feature request: support for arbitrary bin spacing in calibration.calibrationcurve I was using , and it currently accepts an parameter to specify the number of bins to evenly partition the probability space between 0 and 1. However, I am using this in combination with a gradient boosting model in which the probabilities are very uncalibrated, and most of the predictions are close to 0. When I use the calibrated classifier, the result is very noisy because there are many data points in some bins and few, if any, in others (see example below). In the code below, I made a work-around to do what I want and show a plot of my output (in semilog space because of the skewed distribution). I haven't contributed to a large open-source project before, but if there's agreement this would be a useful feature, I would be happy to try to draft up a PR. Notice the large disparity in results between the different bins chosen. For this reason, I think the user should be able to choose the bin edges, as in numpy's or matplotlib's functions. !

y_true = _check_binary_probabilistic_predictions(y_true, y_prob)     bins = np.linspace(0., 1. + 1e-8, n_bins + 1)    if strategy == 'quantile':  # Determine bin edges by distribution of data        quantiles = np.linspace(0, 1, n_bins + 1)        bins = np.percentile(y_prob, quantiles * 100)        bins[-1] = bins[-1] + 1e-8    elif strategy == 'uniform':        bins = np.linspace(0., 1. + 1e-8, n_bins + 1)    else:        raise ValueError("Invalid entry to 'strategy' input. Strategy "                         "must be either 'quantile' or 'uniform'.")     binids = np.digitize(y_prob, bins) - 1      bin_sums = np.bincount(binids, weights=y_prob, minlength=len(bins))


Question: sklearn.modelselection.StratifiedKFold either shuffling is wrong or documentation is misleading <!-- If your issue is a usage question, submit it here instead: StackOverflow with the scikit-learn tag: Mailing List: For more information, see User Questions: http://scikit- --<!-- Instructions For Filing a Bug: --Regarding the shuffle parameter, the documentation states: "Whether to shuffle each stratification of the data before splitting into batches". However, instead of shuffling samples within each stratum, the order of batches is shuffled. As you can see in the output below, 1 is always paired with 11, 2 with 12, 3 with 13, etc. regardless whether shuffle parameter is True or False. When shuffle=True, the batches are always the same for any randomstate, but appear in a different order. When cross-validation is performed, the results from each batch are summed and then divided by the number of batches. Changing the order of batches does not change the result. The way shuffle works now is completely useless from cross-validation perspective. import numpy as np from sklearn.modelselection import StratifiedKFold RANDOMSEED = 1 samplesperclass = 10 X = np.linspace(0, samplesperclass2-1, samplesperclass 2) y = np.concatenate(((samplesperclass), (samplesperclass)), axis=0) print(X, '\n', y, '\n') print('\nshuffle = False\n') kfold = StratifiedKFold(nsplits=10, shuffle=False, randomstate=RANDOMSEED) result = 0 for foldn, (trainidx, testidx) in enumerate((X, y)): print(trainidx, '\n', testidx) print('\nshuffle = True, Random seed =', RANDOMSEED, '\n') kfold = StratifiedKFold(nsplits=10, shuffle=True, randomstate=RANDOMSEED) result = 0 for foldn, (trainidx, testidx) in enumerate((X, y)): print(trainidx, '\n', testidx) RANDOMSEED += 1 print('\nshuffle = True, Random seed =', RANDOMSEED, '\n') kfold = StratifiedKFold(nsplits=10, shuffle=False, randomstate=RANDOMSEED) result = 0 for foldn, (trainidx, testidx) in enumerate((X, y)): print(trainidx, '\n', testidx) <!-- Example: No error is thrown. Please paste or describe the expected results.--I expect batches to be different when Shuffle is turned on for different randomstate seeds. But they are the same <!-- Please paste or specifically describe the actual output or traceback. --[ 0. 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 19.] [1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.] shuffle = False [ 1 2 3 4 5 6 7 8 9 11 12 13 14 15 16 17 18 19] [ 0 10] [ 0 2 3 4 5 6 7 8 9 10 12 13 14 15 16 17 18 19] [ 1 11] [ 0 1 3 4 5 6 7 8 9 10 11 13 14 15 16 17 18 19] [ 2 12] [ 0 1 2 4 5 6 7 8 9 10 11 12 14 15 16 17 18 19] [ 3 13] [ 0 1 2 3 5 6 7 8 9 10 11 12 13 15 16 17 18 19] [ 4 14] [ 0 1 2 3 4 6 7 8 9 10 11 12 13 14 16 17 18 19] [ 5 15] [ 0 1 2 3 4 5 7 8 9 10 11 12 13 14 15 17 18 19] [ 6 16] [ 0 1 2 3 4 5 6 8 9 10 11 12 13 14 15 16 18 19] [ 7 17] [ 0 1 2 3 4 5 6 7 9 10 11 12 13 14 15 16 17 19] [ 8 18] [ 0 1 2 3 4 5 6 7 8 10 11 12 13 14 15 16 17 18] [ 9 19] shuffle = True, Random seed = 1 [ 0 1 3 4 5 6 7 8 9 10 11 13 14 15 16 17 18 19] [ 2 12] [ 0 1 2 3 4 5 6 7 8 10 11 12 13 14 15 16 17 18] [ 9 19] [ 0 1 2 3 4 5 7 8 9 10 11 12 13 14 15 17 18 19] [ 6 16] [ 0 1 2 3 5 6 7 8 9 10 11 12 13 15 16 17 18 19] [ 4 14] [ 1 2 3 4 5 6 7 8 9 11 12 13 14 15 16 17 18 19] [ 0 10] [ 0 1 2 4 5 6 7 8 9 10 11 12 14 15 16 17 18 19] [ 3 13] [ 0 2 3 4 5 6 7 8 9 10 12 13 14 15 16 17 18 19] [ 1 11] [ 0 1 2 3 4 5 6 8 9 10 11 12 13 14 15 16 18 19] [ 7 17] [ 0 1 2 3 4 5 6 7 9 10 11 12 13 14 15 16 17 19] [ 8 18] [ 0 1 2 3 4 6 7 8 9 10 11 12 13 14 16 17 18 19] [ 5 15] shuffle = True, Random seed = 2 [ 1 2 3 4 5 6 7 8 9 11 12 13 14 15 16 17 18 19] [ 0 10] [ 0 2 3 4 5 6 7 8 9 10 12 13 14 15 16 17 18 19] [ 1 11] [ 0 1 3 4 5 6 7 8 9 10 11 13 14 15 16 17 18 19] [ 2 12] [ 0 1 2 4 5 6 7 8 9 10 11 12 14 15 16 17 18 19] [ 3 13] [ 0 1 2 3 5 6 7 8 9 10 11 12 13 15 16 17 18 19] [ 4 14] [ 0 1 2 3 4 6 7 8 9 10 11 12 13 14 16 17 18 19] [ 5 15] [ 0 1 2 3 4 5 7 8 9 10 11 12 13 14 15 17 18 19] [ 6 16] [ 0 1 2 3 4 5 6 8 9 10 11 12 13 14 15 16 18 19] [ 7 17] [ 0 1 2 3 4 5 6 7 9 10 11 12 13 14 15 16 17 19] [ 8 18] [ 0 1 2 3 4 5 6 7 8 10 11 12 13 14 15 16 17 18] [ 9 19] System: python: 3.7.2 (default, Jan 13 2019, 12:50:01) [Clang 10.0.0 (clang-1000.11.45.5)] executable: /usr/local/opt/python/bin/python3.7 machine: Darwin-18.2.0-x8664-i386-64bit BLAS: macros: NOATLASINFO=3, HAVECBLAS=None libdirs: cblaslibs: cblas Python deps: pip: 18.1 setuptools: 40.6.3 sklearn: 0.20.2 numpy: 1.15.2 scipy: 1.2.0 Cython: None pandas: 0.23.4 <!-- Thanks for contributing! --

``n_splits`` default value will change from 3 to 5 in v0.22.      shuffle : boolean, optional        Whether to shuffle each stratification of the data before splitting        into batches.        Whether to shuffle each class's samples before splitting into batches.      random_state : int, RandomState instance or None, optional, default=None         If int, random_state is the seed used by the random number generator;


Question: sklearn.modelselection.StratifiedKFold either shuffling is wrong or documentation is misleading <!-- If your issue is a usage question, submit it here instead: StackOverflow with the scikit-learn tag: Mailing List: For more information, see User Questions: http://scikit- --<!-- Instructions For Filing a Bug: --Regarding the shuffle parameter, the documentation states: "Whether to shuffle each stratification of the data before splitting into batches". However, instead of shuffling samples within each stratum, the order of batches is shuffled. As you can see in the output below, 1 is always paired with 11, 2 with 12, 3 with 13, etc. regardless whether shuffle parameter is True or False. When shuffle=True, the batches are always the same for any randomstate, but appear in a different order. When cross-validation is performed, the results from each batch are summed and then divided by the number of batches. Changing the order of batches does not change the result. The way shuffle works now is completely useless from cross-validation perspective. import numpy as np from sklearn.modelselection import StratifiedKFold RANDOMSEED = 1 samplesperclass = 10 X = np.linspace(0, samplesperclass2-1, samplesperclass 2) y = np.concatenate(((samplesperclass), (samplesperclass)), axis=0) print(X, '\n', y, '\n') print('\nshuffle = False\n') kfold = StratifiedKFold(nsplits=10, shuffle=False, randomstate=RANDOMSEED) result = 0 for foldn, (trainidx, testidx) in enumerate((X, y)): print(trainidx, '\n', testidx) print('\nshuffle = True, Random seed =', RANDOMSEED, '\n') kfold = StratifiedKFold(nsplits=10, shuffle=True, randomstate=RANDOMSEED) result = 0 for foldn, (trainidx, testidx) in enumerate((X, y)): print(trainidx, '\n', testidx) RANDOMSEED += 1 print('\nshuffle = True, Random seed =', RANDOMSEED, '\n') kfold = StratifiedKFold(nsplits=10, shuffle=False, randomstate=RANDOMSEED) result = 0 for foldn, (trainidx, testidx) in enumerate((X, y)): print(trainidx, '\n', testidx) <!-- Example: No error is thrown. Please paste or describe the expected results.--I expect batches to be different when Shuffle is turned on for different randomstate seeds. But they are the same <!-- Please paste or specifically describe the actual output or traceback. --[ 0. 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 19.] [1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.] shuffle = False [ 1 2 3 4 5 6 7 8 9 11 12 13 14 15 16 17 18 19] [ 0 10] [ 0 2 3 4 5 6 7 8 9 10 12 13 14 15 16 17 18 19] [ 1 11] [ 0 1 3 4 5 6 7 8 9 10 11 13 14 15 16 17 18 19] [ 2 12] [ 0 1 2 4 5 6 7 8 9 10 11 12 14 15 16 17 18 19] [ 3 13] [ 0 1 2 3 5 6 7 8 9 10 11 12 13 15 16 17 18 19] [ 4 14] [ 0 1 2 3 4 6 7 8 9 10 11 12 13 14 16 17 18 19] [ 5 15] [ 0 1 2 3 4 5 7 8 9 10 11 12 13 14 15 17 18 19] [ 6 16] [ 0 1 2 3 4 5 6 8 9 10 11 12 13 14 15 16 18 19] [ 7 17] [ 0 1 2 3 4 5 6 7 9 10 11 12 13 14 15 16 17 19] [ 8 18] [ 0 1 2 3 4 5 6 7 8 10 11 12 13 14 15 16 17 18] [ 9 19] shuffle = True, Random seed = 1 [ 0 1 3 4 5 6 7 8 9 10 11 13 14 15 16 17 18 19] [ 2 12] [ 0 1 2 3 4 5 6 7 8 10 11 12 13 14 15 16 17 18] [ 9 19] [ 0 1 2 3 4 5 7 8 9 10 11 12 13 14 15 17 18 19] [ 6 16] [ 0 1 2 3 5 6 7 8 9 10 11 12 13 15 16 17 18 19] [ 4 14] [ 1 2 3 4 5 6 7 8 9 11 12 13 14 15 16 17 18 19] [ 0 10] [ 0 1 2 4 5 6 7 8 9 10 11 12 14 15 16 17 18 19] [ 3 13] [ 0 2 3 4 5 6 7 8 9 10 12 13 14 15 16 17 18 19] [ 1 11] [ 0 1 2 3 4 5 6 8 9 10 11 12 13 14 15 16 18 19] [ 7 17] [ 0 1 2 3 4 5 6 7 9 10 11 12 13 14 15 16 17 19] [ 8 18] [ 0 1 2 3 4 6 7 8 9 10 11 12 13 14 16 17 18 19] [ 5 15] shuffle = True, Random seed = 2 [ 1 2 3 4 5 6 7 8 9 11 12 13 14 15 16 17 18 19] [ 0 10] [ 0 2 3 4 5 6 7 8 9 10 12 13 14 15 16 17 18 19] [ 1 11] [ 0 1 3 4 5 6 7 8 9 10 11 13 14 15 16 17 18 19] [ 2 12] [ 0 1 2 4 5 6 7 8 9 10 11 12 14 15 16 17 18 19] [ 3 13] [ 0 1 2 3 5 6 7 8 9 10 11 12 13 15 16 17 18 19] [ 4 14] [ 0 1 2 3 4 6 7 8 9 10 11 12 13 14 16 17 18 19] [ 5 15] [ 0 1 2 3 4 5 7 8 9 10 11 12 13 14 15 17 18 19] [ 6 16] [ 0 1 2 3 4 5 6 8 9 10 11 12 13 14 15 16 18 19] [ 7 17] [ 0 1 2 3 4 5 6 7 9 10 11 12 13 14 15 16 17 19] [ 8 18] [ 0 1 2 3 4 5 6 7 8 10 11 12 13 14 15 16 17 18] [ 9 19] System: python: 3.7.2 (default, Jan 13 2019, 12:50:01) [Clang 10.0.0 (clang-1000.11.45.5)] executable: /usr/local/opt/python/bin/python3.7 machine: Darwin-18.2.0-x8664-i386-64bit BLAS: macros: NOATLASINFO=3, HAVECBLAS=None libdirs: cblaslibs: cblas Python deps: pip: 18.1 setuptools: 40.6.3 sklearn: 0.20.2 numpy: 1.15.2 scipy: 1.2.0 Cython: None pandas: 0.23.4 <!-- Thanks for contributing! --

super().__init__(n_splits, shuffle, random_state)      def _make_test_folds(self, X, y=None):        rng = self.random_state        rng = check_random_state(self.random_state)         y = np.asarray(y)         type_of_target_y = type_of_target(y)         allowed_target_types = ('binary', 'multiclass')


Question: KBinsDiscretizer: kmeans fails due to unsorted binedges with fails in certain situations, due to centers and consequently binedges being unsorted, which is fatal for np.digitize. A very simple way to reproduce this is to set nbins in the existing testnonuniformstrategies from to a higher value (here 5 instead of 3). In this simple example it seems like an edge case to set nbins to almost the number of data points. However I've seen this happen in productive situations with very reasonable number of bins of order log2(number of unique values of X). No error is thrown. <!-- Thanks for contributing! --KBinsDiscretizer: kmeans fails due to unsorted binedges with fails in certain situations, due to centers and consequently binedges being unsorted, which is fatal for np.digitize. A very simple way to reproduce this is to set nbins in the existing testnonuniformstrategies from to a higher value (here 5 instead of 3). In this simple example it seems like an edge case to set nbins to almost the number of data points. However I've seen this happen in productive situations with very reasonable number of bins of order log2(number of unique values of X). No error is thrown. <!-- Thanks for contributing! --

# 1D k-means procedure                 km = KMeans(n_clusters=n_bins[jj], init=init, n_init=1)                 centers = km.fit(column[:, None]).cluster_centers_[:, 0]<ins>                # Must sort, centers may be unsorted even with sorted init                centers.sort()</ins>                 bin_edges[jj] = (centers[1:] + centers[:-1]) * 0.5                 bin_edges[jj] = np.r_[col_min, bin_edges[jj], col_max]


Question: GaussianMixture predict and fitpredict disagree when ninit1 When is specified in GaussianMixture, the results of fitpredict(X) and predict(X) are often different. The unit test doesn't catch this because it does not set . No exceptions.

best_params = self._get_parameters()                 best_n_iter = n_iter <del>        # Always do a final e-step to guarantee that the labels returned by        # fit_predict(X) are always consistent with fit(X).predict(X)        # for any value of max_iter and tol (and any random_state).        _, log_resp = self._e_step(X)</del>         if not self.converged_:             warnings.warn('Initialization %d did not converge. '                           'Try different init parameters, '


Question: GaussianMixture predict and fitpredict disagree when ninit1 When is specified in GaussianMixture, the results of fitpredict(X) and predict(X) are often different. The unit test doesn't catch this because it does not set . No exceptions.

self.n_iter_ = best_n_iter         self.lower_bound_ = max_lower_bound <ins>        # Always do a final e-step to guarantee that the labels returned by        # fit_predict(X) are always consistent with fit(X).predict(X)        # for any value of max_iter and tol (and any random_state).        _, log_resp = self._e_step(X)</ins>         return log_resp.argmax(axis=1)      def _e_step(self, X):


Question: precisionscore shows incorrect value precisionscore shows incorrect value ([[0,0,1],[0,1,0],[0,0,1]]) B=A precisionscore(A,B, average=None) array([ 0., 1., 1.]) array([ 1., 1., 1.]) array([ 0., 1., 1.]) import platform; print(platform.platform()) Darwin-14.5.0-x8664-i386-64bit import sys; print("Python", sys.version) ('Python', '2.7.10 (default, Jul 14 2015, 19:46:27) \n[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.39)]') import numpy; print("NumPy", numpy.version) ('NumPy', '1.13.3') import scipy; print("SciPy", scipy.version) ('SciPy', '1.0.0') import sklearn; print("Scikit-Learn", sklearn.version) ('Scikit-Learn', '0.18.1')

>>> f1_score(y_true, y_pred, average=None)     array([0.8, 0. , 0. ]) <ins>    Notes    -----    When ``true positive + false positive == 0`` or    ``true positive + false negative == 0``, f-score returns 0 and raises    ``UndefinedMetricWarning``.</ins>     """     return fbeta_score(y_true, y_pred, 1, labels=labels,                        pos_label=pos_label, average=average,


Question: precisionscore shows incorrect value precisionscore shows incorrect value ([[0,0,1],[0,1,0],[0,0,1]]) B=A precisionscore(A,B, average=None) array([ 0., 1., 1.]) array([ 1., 1., 1.]) array([ 0., 1., 1.]) import platform; print(platform.platform()) Darwin-14.5.0-x8664-i386-64bit import sys; print("Python", sys.version) ('Python', '2.7.10 (default, Jul 14 2015, 19:46:27) \n[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.39)]') import numpy; print("NumPy", numpy.version) ('NumPy', '1.13.3') import scipy; print("SciPy", scipy.version) ('SciPy', '1.0.0') import sklearn; print("Scikit-Learn", sklearn.version) ('Scikit-Learn', '0.18.1')

... # doctest: +ELLIPSIS     array([0.71..., 0.        , 0.        ]) <ins>    Notes    -----    When ``true positive + false positive == 0`` or    ``true positive + false negative == 0``, f-score returns 0 and raises    ``UndefinedMetricWarning``.</ins>     """     _, _, f, _ = precision_recall_fscore_support(y_true, y_pred,                                                  beta=beta,


Question: precisionscore shows incorrect value precisionscore shows incorrect value ([[0,0,1],[0,1,0],[0,0,1]]) B=A precisionscore(A,B, average=None) array([ 0., 1., 1.]) array([ 1., 1., 1.]) array([ 0., 1., 1.]) import platform; print(platform.platform()) Darwin-14.5.0-x8664-i386-64bit import sys; print("Python", sys.version) ('Python', '2.7.10 (default, Jul 14 2015, 19:46:27) \n[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.39)]') import numpy; print("NumPy", numpy.version) ('NumPy', '1.13.3') import scipy; print("SciPy", scipy.version) ('SciPy', '1.0.0') import sklearn; print("Scikit-Learn", sklearn.version) ('Scikit-Learn', '0.18.1')

array([0., 0., 1.]), array([0. , 0. , 0.8]),      array([2, 2, 2])) <ins>    Notes    -----    When ``true positive + false positive == 0``, precision is undefined;    When ``true positive + false negative == 0``, recall is undefined.    In such cases, the metric will be set to 0, as will f-score, and    ``UndefinedMetricWarning`` will be raised.</ins>     """     average_options = (None, 'micro', 'macro', 'weighted', 'samples')     if average not in average_options and average != 'binary':


Question: precisionscore shows incorrect value precisionscore shows incorrect value ([[0,0,1],[0,1,0],[0,0,1]]) B=A precisionscore(A,B, average=None) array([ 0., 1., 1.]) array([ 1., 1., 1.]) array([ 0., 1., 1.]) import platform; print(platform.platform()) Darwin-14.5.0-x8664-i386-64bit import sys; print("Python", sys.version) ('Python', '2.7.10 (default, Jul 14 2015, 19:46:27) \n[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.39)]') import numpy; print("NumPy", numpy.version) ('NumPy', '1.13.3') import scipy; print("SciPy", scipy.version) ('SciPy', '1.0.0') import sklearn; print("Scikit-Learn", sklearn.version) ('Scikit-Learn', '0.18.1')

if average == 'binary':         if y_type == 'binary':            if pos_label not in present_labels:                if len(present_labels) < 2:                    # Only negative labels                    return (0., 0., 0., 0)                else:                    raise ValueError("pos_label=%r is not a valid label: %r" %                                     (pos_label, present_labels))            if pos_label not in present_labels and len(present_labels) >= 2:                raise ValueError("pos_label=%r is not a valid label: %r" %                                 (pos_label, present_labels))             labels = [pos_label]         else:             raise ValueError("Target is %s but average='binary'. Please "


Question: precisionscore shows incorrect value precisionscore shows incorrect value ([[0,0,1],[0,1,0],[0,0,1]]) B=A precisionscore(A,B, average=None) array([ 0., 1., 1.]) array([ 1., 1., 1.]) array([ 0., 1., 1.]) import platform; print(platform.platform()) Darwin-14.5.0-x8664-i386-64bit import sys; print("Python", sys.version) ('Python', '2.7.10 (default, Jul 14 2015, 19:46:27) \n[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.39)]') import numpy; print("NumPy", numpy.version) ('NumPy', '1.13.3') import scipy; print("SciPy", scipy.version) ('SciPy', '1.0.0') import sklearn; print("Scikit-Learn", sklearn.version) ('Scikit-Learn', '0.18.1')

true_sum = np.array([true_sum.sum()])      # Finally, we have all our sufficient statistics. Divide! #<del></del>     beta2 = beta ** 2     with np.errstate(divide='ignore', invalid='ignore'):         # Divide, and on zero-division, set scores to 0 and warn:


Question: precisionscore shows incorrect value precisionscore shows incorrect value ([[0,0,1],[0,1,0],[0,0,1]]) B=A precisionscore(A,B, average=None) array([ 0., 1., 1.]) array([ 1., 1., 1.]) array([ 0., 1., 1.]) import platform; print(platform.platform()) Darwin-14.5.0-x8664-i386-64bit import sys; print("Python", sys.version) ('Python', '2.7.10 (default, Jul 14 2015, 19:46:27) \n[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.39)]') import numpy; print("NumPy", numpy.version) ('NumPy', '1.13.3') import scipy; print("SciPy", scipy.version) ('SciPy', '1.0.0') import sklearn; print("Scikit-Learn", sklearn.version) ('Scikit-Learn', '0.18.1')

f_score[tp_sum == 0] = 0.0      # Average the results<del></del>     if average == 'weighted':         weights = true_sum         if weights.sum() == 0:


Question: precisionscore shows incorrect value precisionscore shows incorrect value ([[0,0,1],[0,1,0],[0,0,1]]) B=A precisionscore(A,B, average=None) array([ 0., 1., 1.]) array([ 1., 1., 1.]) array([ 0., 1., 1.]) import platform; print(platform.platform()) Darwin-14.5.0-x8664-i386-64bit import sys; print("Python", sys.version) ('Python', '2.7.10 (default, Jul 14 2015, 19:46:27) \n[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.39)]') import numpy; print("NumPy", numpy.version) ('NumPy', '1.13.3') import scipy; print("SciPy", scipy.version) ('SciPy', '1.0.0') import sklearn; print("Scikit-Learn", sklearn.version) ('Scikit-Learn', '0.18.1')

>>> precision_score(y_true, y_pred, average=None)  # doctest: +ELLIPSIS     array([0.66..., 0.        , 0.        ]) <ins>    Notes    -----    When ``true positive + false positive == 0``, precision returns 0 and    raises ``UndefinedMetricWarning``.</ins>     """     p, _, _, _ = precision_recall_fscore_support(y_true, y_pred,                                                  labels=labels,


Question: precisionscore shows incorrect value precisionscore shows incorrect value ([[0,0,1],[0,1,0],[0,0,1]]) B=A precisionscore(A,B, average=None) array([ 0., 1., 1.]) array([ 1., 1., 1.]) array([ 0., 1., 1.]) import platform; print(platform.platform()) Darwin-14.5.0-x8664-i386-64bit import sys; print("Python", sys.version) ('Python', '2.7.10 (default, Jul 14 2015, 19:46:27) \n[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.39)]') import numpy; print("NumPy", numpy.version) ('NumPy', '1.13.3') import scipy; print("SciPy", scipy.version) ('SciPy', '1.0.0') import sklearn; print("Scikit-Learn", sklearn.version) ('Scikit-Learn', '0.18.1')

>>> recall_score(y_true, y_pred, average=None)     array([1., 0., 0.]) <ins>    Notes    -----    When ``true positive + false negative == 0``, recall returns 0 and raises    ``UndefinedMetricWarning``.</ins>     """     _, r, _, _ = precision_recall_fscore_support(y_true, y_pred,                                                  labels=labels,


Question: Different r2score multioutput default in r2score and base.RegressorMixin We've changed multioutput default in r2score to "uniformaverage" in 0.19, but in base.RegressorMixin, we still use ` (). Also see the strange things below:

-------         score : float             R^2 of self.predict(X) wrt. y.        Notes        -----        The R2 score used when calling ``score`` on a regressor will use        ``multioutput='uniform_average'`` from version 0.23 to keep consistent        with `metrics.r2_score`. This will influence the ``score`` method of        all the multioutput regressors (except for        `multioutput.MultiOutputRegressor`). To use the new default, please        either call `metrics.r2_score` directly or make a custom scorer with        `metric.make_scorer`.         """          from .metrics import r2_score        return r2_score(y, self.predict(X), sample_weight=sample_weight,        from .metrics.regression import _check_reg_targets        y_pred = self.predict(X)        # XXX: Remove the check in 0.23        y_type, _, _, _ = _check_reg_targets(y, y_pred, None)        if y_type == 'continuous-multioutput':            warnings.warn("The default value of multioutput (not exposed in "                          "score method) will change from 'variance_weighted' "                          "to 'uniform_average' in 0.23 to keep consistent "                          "with 'metrics.r2_score'. To use the new default, "                          "please either call 'metrics.r2_score' directly or "                          "make a custom scorer with 'metric.make_scorer'.",                          FutureWarning)        return r2_score(y, y_pred, sample_weight=sample_weight,                         multioutput='variance_weighted')  diff --git a/sklearn/linear_model/coordinate_descent.py b/sklearn/linear_model/coordinate_descent.py-- a/sklearn/linear_model/coordinate_descent.py++ b/sklearn/linear_model/coordinate_descent.py


Question: Different r2score multioutput default in r2score and base.RegressorMixin We've changed multioutput default in r2score to "uniformaverage" in 0.19, but in base.RegressorMixin, we still use ` (). Also see the strange things below:

--------     >>> from sklearn.linear_model import MultiTaskLassoCV     >>> from sklearn.datasets import make_regression    >>> from sklearn.metrics import r2_score     >>> X, y = make_regression(n_targets=2, noise=4, random_state=0)     >>> reg = MultiTaskLassoCV(cv=5, random_state=0).fit(X, y)    >>> reg.score(X, y) # doctest: +ELLIPSIS    >>> r2_score(y, reg.predict(X)) # doctest: +ELLIPSIS     0.9994...     >>> reg.alpha_     0.5713...diff --git a/sklearn/multioutput.py b/sklearn/multioutput.py-- a/sklearn/multioutput.py++ b/sklearn/multioutput.py


Question: Different r2score multioutput default in r2score and base.RegressorMixin We've changed multioutput default in r2score to "uniformaverage" in 0.19, but in base.RegressorMixin, we still use ` (). Also see the strange things below:

super().partial_fit(             X, y, sample_weight=sample_weight) <ins>    # XXX Remove this method in 0.23</ins>     def score(self, X, y, sample_weight=None):         """Returns the coefficient of determination R^2 of the prediction.


Question: Ensure monotonic bin edges for KBinsDiscretizer strategy quantile The percentiles returned from np.percentile are monotonic up to possible numeric instabilities. Monotonicity is enforced by applying a simple maximum on subsequent values to deal with this case and increase robustness. The additional line is a no-op in almost all cases. This is unfortunate, but since there is essentially no performance impact, I guess robustness is worth the effort.

Attributes     ----------     n_bins_ : int array, shape (n_features,)        Number of bins per feature.        Number of bins per feature. Bins whose width are too small        (i.e., <= 1e-8) are removed with a warning.      bin_edges_ : array of arrays, shape (n_features, )         The edges of each bin. Contain arrays of varying shapes ``(n_bins_, )``


Question: Ensure monotonic bin edges for KBinsDiscretizer strategy quantile The percentiles returned from np.percentile are monotonic up to possible numeric instabilities. Monotonicity is enforced by applying a simple maximum on subsequent values to deal with this case and increase robustness. The additional line is a no-op in almost all cases. This is unfortunate, but since there is essentially no performance impact, I guess robustness is worth the effort.

:class:`sklearn.compose.ColumnTransformer` if you only want to preprocess     part of the features. <ins>    ``KBinsDiscretizer`` might produce constant features (e.g., when    ``encode = 'onehot'`` and certain bins do not contain any data).    These features can be removed with feature selection algorithms    (e.g., :class:`sklearn.feature_selection.VarianceThreshold`).</ins>     See also     --------      sklearn.preprocessing.Binarizer : class used to bin values as ``0`` or


Question: Ensure monotonic bin edges for KBinsDiscretizer strategy quantile The percentiles returned from np.percentile are monotonic up to possible numeric instabilities. Monotonicity is enforced by applying a simple maximum on subsequent values to deal with this case and increase robustness. The additional line is a no-op in almost all cases. This is unfortunate, but since there is essentially no performance impact, I guess robustness is worth the effort.

bin_edges[jj] = (centers[1:] + centers[:-1]) * 0.5                 bin_edges[jj] = np.r_[col_min, bin_edges[jj], col_max] <ins>            # Remove bins whose width are too small (i.e., <= 1e-8)            if self.strategy in ('quantile', 'kmeans'):                mask = np.ediff1d(bin_edges[jj], to_begin=np.inf) > 1e-8                bin_edges[jj] = bin_edges[jj][mask]                if len(bin_edges[jj]) - 1 != n_bins[jj]:                    warnings.warn('Bins whose width are too small (i.e., <= '                                  '1e-8) in feature %d are removed. Consider '                                  'decreasing the number of bins.' % jj)                    n_bins[jj] = len(bin_edges[jj]) - 1</ins>         self.bin_edges_ = bin_edges         self.n_bins_ = n_bins


Question: gamma='scale' in SVC I believe that setting in is not meeting its intended purpose of being invariant to the scale of . Currently, is set to . However, I believe it should be . Rationale: if you scale by 10 you need to scale by 1/100, not 1/10, to achieve the same results. See the definition of the RBF kernel : the "units" of are 1/x^2, not 1/x. I also tested this empirically: scaling by 10 and scaling by 1/100 gives the same result as the original, whereas scaling by 10 and scaling by 1/10 gives a different result. Here is some code: Note that will become the default setting for in version 0.22. Related: ,

if self.gamma in ('scale', 'auto_deprecated'):             if sparse:                # std = sqrt(E[X^2] - E[X]^2)                X_std = np.sqrt((X.multiply(X)).mean() - (X.mean())**2)                # var = E[X^2] - E[X]^2                X_var = (X.multiply(X)).mean() - (X.mean()) ** 2             else:                X_std = X.std()                X_var = X.var()             if self.gamma == 'scale':                if X_std != 0:                    self._gamma = 1.0 / (X.shape[1] * X_std)                if X_var != 0:                    self._gamma = 1.0 / (X.shape[1] * X_var)                 else:                     self._gamma = 1.0             else:                 kernel_uses_gamma = (not callable(self.kernel) and self.kernel                                      not in ('linear', 'precomputed'))                if kernel_uses_gamma and not np.isclose(X_std, 1.0):                if kernel_uses_gamma and not np.isclose(X_var, 1.0):                     # NOTE: when deprecation ends we need to remove explicitly                     # setting `gamma` in examples (also in tests). See                     # https://github.com/scikit-learn/scikit-learn/pull/10331diff --git a/sklearn/svm/classes.py b/sklearn/svm/classes.py-- a/sklearn/svm/classes.py++ b/sklearn/svm/classes.py


Question: gamma='scale' in SVC I believe that setting in is not meeting its intended purpose of being invariant to the scale of . Currently, is set to . However, I believe it should be . Rationale: if you scale by 10 you need to scale by 1/100, not 1/10, to achieve the same results. See the definition of the RBF kernel : the "units" of are 1/x^2, not 1/x. I also tested this empirically: scaling by 10 and scaling by 1/100 gives the same result as the original, whereas scaling by 10 and scaling by 1/10 gives a different result. Here is some code: Note that will become the default setting for in version 0.22. Related: ,

Kernel coefficient for 'rbf', 'poly' and 'sigmoid'.          Current default is 'auto' which uses 1 / n_features,        if ``gamma='scale'`` is passed then it uses 1 / (n_features * X.std())        if ``gamma='scale'`` is passed then it uses 1 / (n_features * X.var())         as value of gamma. The current default of gamma, 'auto', will change         to 'scale' in version 0.22. 'auto_deprecated', a deprecated version of         'auto' is used as a default indicating that no explicit value of gamma


Question: gamma='scale' in SVC I believe that setting in is not meeting its intended purpose of being invariant to the scale of . Currently, is set to . However, I believe it should be . Rationale: if you scale by 10 you need to scale by 1/100, not 1/10, to achieve the same results. See the definition of the RBF kernel : the "units" of are 1/x^2, not 1/x. I also tested this empirically: scaling by 10 and scaling by 1/100 gives the same result as the original, whereas scaling by 10 and scaling by 1/10 gives a different result. Here is some code: Note that will become the default setting for in version 0.22. Related: ,


Question: gamma='scale' in SVC I believe that setting in is not meeting its intended purpose of being invariant to the scale of . Currently, is set to . However, I believe it should be . Rationale: if you scale by 10 you need to scale by 1/100, not 1/10, to achieve the same results. See the definition of the RBF kernel : the "units" of are 1/x^2, not 1/x. I also tested this empirically: scaling by 10 and scaling by 1/100 gives the same result as the original, whereas scaling by 10 and scaling by 1/10 gives a different result. Here is some code: Note that will become the default setting for in version 0.22. Related: ,


Question: gamma='scale' in SVC I believe that setting in is not meeting its intended purpose of being invariant to the scale of . Currently, is set to . However, I believe it should be . Rationale: if you scale by 10 you need to scale by 1/100, not 1/10, to achieve the same results. See the definition of the RBF kernel : the "units" of are 1/x^2, not 1/x. I also tested this empirically: scaling by 10 and scaling by 1/100 gives the same result as the original, whereas scaling by 10 and scaling by 1/10 gives a different result. Here is some code: Note that will become the default setting for in version 0.22. Related: ,


Question: gamma='scale' in SVC I believe that setting in is not meeting its intended purpose of being invariant to the scale of . Currently, is set to . However, I believe it should be . Rationale: if you scale by 10 you need to scale by 1/100, not 1/10, to achieve the same results. See the definition of the RBF kernel : the "units" of are 1/x^2, not 1/x. I also tested this empirically: scaling by 10 and scaling by 1/100 gives the same result as the original, whereas scaling by 10 and scaling by 1/10 gives a different result. Here is some code: Note that will become the default setting for in version 0.22. Related: ,


Question: Differences among the results of KernelPCA with rbf kernel Hi there, I met with a problem: When I run KernelPCA for dimension reduction for the same datasets, the results are different in signs. Just to reduce the dimension to 7 with rbf kernel: pca = KernelPCA(ncomponents=7, kernel='rbf', copyX=False, njobs=-1) pca.fittransform(X) The same result. The results are the same except for their signs:( [[-0. -0. -0. 0. -0. -0. 0.]] [[ 0. 0. 0. -0. -0. -0. -0.]] [[-0. -0. 0. 0. 0. 0. 0.]] 18.1

from scipy.sparse.linalg import eigsh  from ..utils import check_random_state<ins>from ..utils.extmath import svd_flip</ins> from ..utils.validation import check_is_fitted, check_array from ..exceptions import NotFittedError from ..base import BaseEstimator, TransformerMixin, _UnstableOn32BitMixin


Question: Differences among the results of KernelPCA with rbf kernel Hi there, I met with a problem: When I run KernelPCA for dimension reduction for the same datasets, the results are different in signs. Just to reduce the dimension to 7 with rbf kernel: pca = KernelPCA(ncomponents=7, kernel='rbf', copyX=False, njobs=-1) pca.fittransform(X) The same result. The results are the same except for their signs:( [[-0. -0. -0. 0. -0. -0. 0.]] [[ 0. 0. 0. -0. -0. -0. -0.]] [[-0. -0. 0. 0. 0. 0. 0.]] 18.1

maxiter=self.max_iter,                                                 v0=v0) <ins>        # flip eigenvectors' sign to enforce deterministic output        self.alphas_, _ = svd_flip(self.alphas_,                                   np.empty_like(self.alphas_).T)</ins>         # sort eigenvectors in descending order         indices = self.lambdas_.argsort()[::-1]         self.lambdas_ = self.lambdas_[indices]


Question: ENH: support DataFrames in OneHot/OrdinalEncoder without converting to array Left-over to do from Idea is to support DataFrames without converting to a contiguous array. This conversion is not needed, as the transformer encodes the input column by column anyway, so it would be rather easy to preserve the datatypes per column. This would avoid converting a potentially mixed-dtype DataFrame (eg ints and object strings) to a full object array. This can introduces a slight change in behaviour (it can change the of the in certain edge cases, eg when you had a mixture of float and int columns). (Note that is not yet necessarily means to have special handling for certain pandas dtypes such as categorical dtype, see , in an initial step, we could still do a on each column / coerce each column to a numpy array).

- convert list of strings to object dtype         - check for missing values for object dtype data (check_array does           not do that)        - return list of features (arrays): this list of features is          constructed feature by feature to preserve the data types          of pandas DataFrame columns, as otherwise information is lost          and cannot be used, eg for the `categories_` attribute.          """        X_temp = check_array(X, dtype=None)        if not hasattr(X, 'dtype') and np.issubdtype(X_temp.dtype, np.str_):            X = check_array(X, dtype=np.object)        if not (hasattr(X, 'iloc') and getattr(X, 'ndim', 0) == 2):            # if not a dataframe, do normal check_array validation            X_temp = check_array(X, dtype=None)            if (not hasattr(X, 'dtype')                    and np.issubdtype(X_temp.dtype, np.str_)):                X = check_array(X, dtype=np.object)            else:                X = X_temp            needs_validation = False         else:            X = X_temp            # pandas dataframe, do validation later column by column, in order            # to keep the dtype information to be used in the encoder.            needs_validation = True         return X        n_samples, n_features = X.shape        X_columns = []     def _fit(self, X, handle_unknown='error'):        X = self._check_X(X)        for i in range(n_features):            Xi = self._get_feature(X, feature_idx=i)            Xi = check_array(Xi, ensure_2d=False, dtype=None,                             force_all_finite=needs_validation)            X_columns.append(Xi)         n_samples, n_features = X.shape        return X_columns, n_samples, n_features    def _get_feature(self, X, feature_idx):        if hasattr(X, 'iloc'):            # pandas dataframes            return X.iloc[:, feature_idx]        # numpy arrays, sparse arrays        return X[:, feature_idx]    def _fit(self, X, handle_unknown='error'):        X_list, n_samples, n_features = self._check_X(X)          if self._categories != 'auto':            if X.dtype != object:                for cats in self._categories:                    if not np.all(np.sort(cats) == np.array(cats)):                        raise ValueError("Unsorted categories are not "                                         "supported for numerical categories")             if len(self._categories) != n_features:                 raise ValueError("Shape mismatch: if n_values is an array,"                                  " it has to be of shape (n_features,).")


Question: ENH: support DataFrames in OneHot/OrdinalEncoder without converting to array Left-over to do from Idea is to support DataFrames without converting to a contiguous array. This conversion is not needed, as the transformer encodes the input column by column anyway, so it would be rather easy to preserve the datatypes per column. This would avoid converting a potentially mixed-dtype DataFrame (eg ints and object strings) to a full object array. This can introduces a slight change in behaviour (it can change the of the in certain edge cases, eg when you had a mixture of float and int columns). (Note that is not yet necessarily means to have special handling for certain pandas dtypes such as categorical dtype, see , in an initial step, we could still do a on each column / coerce each column to a numpy array).

self.categories_ = []          for i in range(n_features):            Xi = X[:, i]            Xi = X_list[i]             if self._categories == 'auto':                 cats = _encode(Xi)             else:                cats = np.array(self._categories[i], dtype=X.dtype)                cats = np.array(self._categories[i], dtype=Xi.dtype)                if Xi.dtype != object:                    if not np.all(np.sort(cats) == cats):                        raise ValueError("Unsorted categories are not "                                         "supported for numerical categories")                 if handle_unknown == 'error':                     diff = _encode_check_unknown(Xi, cats)                     if diff:


Question: ENH: support DataFrames in OneHot/OrdinalEncoder without converting to array Left-over to do from Idea is to support DataFrames without converting to a contiguous array. This conversion is not needed, as the transformer encodes the input column by column anyway, so it would be rather easy to preserve the datatypes per column. This would avoid converting a potentially mixed-dtype DataFrame (eg ints and object strings) to a full object array. This can introduces a slight change in behaviour (it can change the of the in certain edge cases, eg when you had a mixture of float and int columns). (Note that is not yet necessarily means to have special handling for certain pandas dtypes such as categorical dtype, see , in an initial step, we could still do a on each column / coerce each column to a numpy array).

self.categories_.append(cats)      def _transform(self, X, handle_unknown='error'):        X = self._check_X(X)        X_list, n_samples, n_features = self._check_X(X)         _, n_features = X.shape        X_int = np.zeros_like(X, dtype=np.int)        X_mask = np.ones_like(X, dtype=np.bool)        X_int = np.zeros((n_samples, n_features), dtype=np.int)        X_mask = np.ones((n_samples, n_features), dtype=np.bool)          for i in range(n_features):            Xi = X[:, i]            Xi = X_list[i]             diff, valid_mask = _encode_check_unknown(Xi, self.categories_[i],                                                      return_mask=True)


Question: partialfit does not account for unobserved target values when fitting priors to data My understanding is that priors should be fitted to the data using observed target frequencies and a variant of to avoid assigning 0 probability to targets not yet observed. It seems the implementation of does not account for unobserved targets at the time of the first training batch when computing priors. RuntimeWarning: divide by zero encountered in log self.classlogprior = ((self.classcount) - scikit-learn version: 0.20.2 This behavior is not very intuitive to me. It seems requires for the right reason, but doesn't actually offset target frequencies to account for unobserved targets.

" classes.")             self.class_log_prior_ = np.log(class_prior)         elif self.fit_prior:            with warnings.catch_warnings():                # silence the warning when count is 0 because class was not yet                # observed                warnings.simplefilter("ignore", RuntimeWarning)                log_class_count = np.log(self.class_count_)             # empirical prior, with sample_weight taken into account            self.class_log_prior_ = (np.log(self.class_count_) -            self.class_log_prior_ = (log_class_count -                                      np.log(self.class_count_.sum()))         else:             self.class_log_prior_ = np.full(n_classes, -np.log(n_classes))


Question: Isolation forest - decisionfunction & averagepathlength method are memory inefficient Isolation forest consumes too much memory due to memory ineffecient implementation of anomoly score calculation. Due to this the parallelization with njobs is also impacted as anomoly score cannot be calculated in parallel for each tree. Run a simple Isolation forest with nestimators as 10 and as 50 respectively. On memory profiling, it can be seen that each building of tree is not taking much memory but in the end a lot of memory is consumed as a for loop is iteration over all trees and calculating the anomoly score of all trees together and then averaging it. - line 267-281 ` Due to this, in case of more no. of estimators(1000), the memory consumed is quite high. Possible Solution: The above for loop should only do the averaging of anomoly score from each estimator instead of calculation. The logic of isoforest anomoly score calculation can be moved to base estimator class so it is done for each tree( i guess file-similar to other method available after fitting) The memory consumption is profound as we increase no. of estimators. The fit method calls decision function & average anomoly score which are taking quite a lot memory. the memory spike is too high in the very end, that is in finall call to method. <!-- Thanks for contributing! --

from warnings import warn  from ..tree import ExtraTreeRegressorfrom ..utils import check_random_state, check_arrayfrom ..utils import (    check_random_state,    check_array,    gen_batches,    get_chunk_n_rows,) from ..utils.fixes import _joblib_parallel_argsfrom ..utils.validation import check_is_fittedfrom ..utils.validation import check_is_fitted, _num_samples from ..base import OutlierMixin  from .bagging import BaseBagging


Question: Isolation forest - decisionfunction & averagepathlength method are memory inefficient Isolation forest consumes too much memory due to memory ineffecient implementation of anomoly score calculation. Due to this the parallelization with njobs is also impacted as anomoly score cannot be calculated in parallel for each tree. Run a simple Isolation forest with nestimators as 10 and as 50 respectively. On memory profiling, it can be seen that each building of tree is not taking much memory but in the end a lot of memory is consumed as a for loop is iteration over all trees and calculating the anomoly score of all trees together and then averaging it. - line 267-281 ` Due to this, in case of more no. of estimators(1000), the memory consumed is quite high. Possible Solution: The above for loop should only do the averaging of anomoly score from each estimator instead of calculation. The logic of isoforest anomoly score calculation can be moved to base estimator class so it is done for each tree( i guess file-similar to other method available after fitting) The memory consumption is profound as we increase no. of estimators. The fit method calls decision function & average anomoly score which are taking quite a lot memory. the memory spike is too high in the very end, that is in finall call to method. <!-- Thanks for contributing! --

"match the input. Model n_features is {0} and "                              "input n_features is {1}."                              "".format(self.n_features_, X.shape[1]))        n_samples = X.shape[0]         n_samples_leaf = np.zeros(n_samples, order="f")        depths = np.zeros(n_samples, order="f")        # Take the opposite of the scores as bigger is better (here less        # abnormal)        return -self._compute_chunked_score_samples(X)    @property    def threshold_(self):        if self.behaviour != 'old':            raise AttributeError("threshold_ attribute does not exist when "                                 "behaviour != 'old'")        warn("threshold_ attribute is deprecated in 0.20 and will"             " be removed in 0.22.", DeprecationWarning)        return self._threshold_    def _compute_chunked_score_samples(self, X):        n_samples = _num_samples(X)          if self._max_features == X.shape[1]:             subsample_features = False         else:             subsample_features = True         # We get as many rows as possible within our working_memory budget        # (defined by sklearn.get_config()['working_memory']) to store        # self._max_features in each row during computation.        #        # Note:        #  - this will get at least 1 row, even if 1 row of score will        #    exceed working_memory.        #  - this does only account for temporary memory usage while loading        #    the data needed to compute the scores -- the returned scores        #    themselves are 1D.        chunk_n_rows = get_chunk_n_rows(row_bytes=16 * self._max_features,                                        max_n_rows=n_samples)        slices = gen_batches(n_samples, chunk_n_rows)        scores = np.zeros(n_samples, order="f")        for sl in slices:            # compute score on the slices of test samples:            scores[sl] = self._compute_score_samples(X[sl], subsample_features)        return scores    def _compute_score_samples(self, X, subsample_features):        """Compute the score of each samples in X going through the extra trees.        Parameters        ----------        X : array-like or sparse matrix        subsample_features : bool,            whether features should be subsampled        """        n_samples = X.shape[0]        depths = np.zeros(n_samples, order="f")         for tree, features in zip(self.estimators_, self.estimators_features_):            if subsample_features:                X_subset = X[:, features]            else:                X_subset = X            X_subset = X[:, features] if subsample_features else X             leaves_index = tree.apply(X_subset)             node_indicator = tree.decision_path(X_subset)             n_samples_leaf = tree.tree_.n_node_samples[leaves_index]


Question: Isolation forest - decisionfunction & averagepathlength method are memory inefficient Isolation forest consumes too much memory due to memory ineffecient implementation of anomoly score calculation. Due to this the parallelization with njobs is also impacted as anomoly score cannot be calculated in parallel for each tree. Run a simple Isolation forest with nestimators as 10 and as 50 respectively. On memory profiling, it can be seen that each building of tree is not taking much memory but in the end a lot of memory is consumed as a for loop is iteration over all trees and calculating the anomoly score of all trees together and then averaging it. - line 267-281 ` Due to this, in case of more no. of estimators(1000), the memory consumed is quite high. Possible Solution: The above for loop should only do the averaging of anomoly score from each estimator instead of calculation. The logic of isoforest anomoly score calculation can be moved to base estimator class so it is done for each tree( i guess file-similar to other method available after fitting) The memory consumption is profound as we increase no. of estimators. The fit method calls decision function & average anomoly score which are taking quite a lot memory. the memory spike is too high in the very end, that is in finall call to method. <!-- Thanks for contributing! --

/ (len(self.estimators_)                * _average_path_length([self.max_samples_]))         )        # Take the opposite of the scores as bigger is better (here less        # abnormal)        return -scores    @property    def threshold_(self):        if self.behaviour != 'old':            raise AttributeError("threshold_ attribute does not exist when "                                 "behaviour != 'old'")        warn("threshold_ attribute is deprecated in 0.20 and will"             " be removed in 0.22.", DeprecationWarning)        return self._threshold_        return scores   def _average_path_length(n_samples_leaf):


Question: checkclassweightbalancedclassifiers is never run?! checkclassweightbalancedclassifiers(name, Classifier, Xtrain, ytrain, Same for `

classifier.set_params(class_weight=class_weight)     coef_manual = classifier.fit(X, y).coef_.copy()     assert_allclose(coef_balanced, coef_manual)    assert_allclose(coef_balanced, coef_manual,                    err_msg="Classifier %s is not computing"                    " class_weight=balanced properly."                    % name)   @ignore_warnings(category=(DeprecationWarning, FutureWarning))


Question: TypeError when supplying a boolean X to HuberRegressor fit when fitting with boolean predictors. No error is thrown when of is (second line of code in the snipped above, ) Boolean array is expected to be converted to by as it is done by, say . is thrown: Latest versions of everything as far as I am aware: <!-- Thanks for contributing! --<!-- NP! --

self : object         """         X, y = check_X_y(            X, y, copy=False, accept_sparse=['csr'], y_numeric=True)            X, y, copy=False, accept_sparse=['csr'], y_numeric=True,            dtype=[np.float64, np.float32])         if sample_weight is not None:             sample_weight = np.array(sample_weight)             check_consistent_length(y, sample_weight)


Question: DOC Improve doc of nquantiles in QuantileTransformer The uses numpy.percentile(Xtrain, .) as the estimator of the quantile function of the training data. To know this function perfectly we just need to take . Then it is just a linear interpolation (which is done in the code afterwards). Therefore I don't think we should be able to choose and we should prevent users from thinking that the higher the better the transformation. As mentioned by IRL it is however true that it can be relevant to choose when is very large. I suggest to add more information on the impact of in the doc which currently reads: For example using 100 times more landmarks result in the same transformation Interestingly if you do not choose correctly, the linear interpolation done afterwards does not correspond to the numpy.percentile(Xtrain, .) estimator. This is not "wrong" as these are only estimators of the true quantile function/cdf but I think it is confusing and would be better to stick with the original estimator. For instance, the following raises an AssertionError.

X_scaled = X_std * (max - min) + min      where min, max = feature_range.<del> </del><ins></ins>     The transformation is calculated as (when ``axis=0``)::         X_scaled = scale * X + min - X.min(axis=0) * scale


Question: DOC Improve doc of nquantiles in QuantileTransformer The uses numpy.percentile(Xtrain, .) as the estimator of the quantile function of the training data. To know this function perfectly we just need to take . Then it is just a linear interpolation (which is done in the code afterwards). Therefore I don't think we should be able to choose and we should prevent users from thinking that the higher the better the transformation. As mentioned by IRL it is however true that it can be relevant to choose when is very large. I suggest to add more information on the impact of in the doc which currently reads: For example using 100 times more landmarks result in the same transformation Interestingly if you do not choose correctly, the linear interpolation done afterwards does not correspond to the numpy.percentile(Xtrain, .) estimator. This is not "wrong" as these are only estimators of the true quantile function/cdf but I think it is confusing and would be better to stick with the original estimator. For instance, the following raises an AssertionError.

-----     NaNs are treated as missing values: disregarded in fit, and maintained in     transform.<del>    </del><ins></ins>     We use a biased estimator for the standard deviation, equivalent to     `numpy.std(x, ddof=0)`. Note that the choice of `ddof` is unlikely to     affect model performance.


Question: DOC Improve doc of nquantiles in QuantileTransformer The uses numpy.percentile(Xtrain, .) as the estimator of the quantile function of the training data. To know this function perfectly we just need to take . Then it is just a linear interpolation (which is done in the code afterwards). Therefore I don't think we should be able to choose and we should prevent users from thinking that the higher the better the transformation. As mentioned by IRL it is however true that it can be relevant to choose when is very large. I suggest to add more information on the impact of in the doc which currently reads: For example using 100 times more landmarks result in the same transformation Interestingly if you do not choose correctly, the linear interpolation done afterwards does not correspond to the numpy.percentile(Xtrain, .) estimator. This is not "wrong" as these are only estimators of the true quantile function/cdf but I think it is confusing and would be better to stick with the original estimator. For instance, the following raises an AssertionError.

Parameters     ----------    n_quantiles : int, optional (default=1000)    n_quantiles : int, optional (default=1000 or n_samples)         Number of quantiles to be computed. It corresponds to the number         of landmarks used to discretize the cumulative distribution function.        If n_quantiles is larger than the number of samples, n_quantiles is set        to the number of samples as a larger number of quantiles does not give        a better approximation of the cumulative distribution function        estimator.      output_distribution : str, optional (default='uniform')         Marginal distribution for the transformed data. The choices are


Question: DOC Improve doc of nquantiles in QuantileTransformer The uses numpy.percentile(Xtrain, .) as the estimator of the quantile function of the training data. To know this function perfectly we just need to take . Then it is just a linear interpolation (which is done in the code afterwards). Therefore I don't think we should be able to choose and we should prevent users from thinking that the higher the better the transformation. As mentioned by IRL it is however true that it can be relevant to choose when is very large. I suggest to add more information on the impact of in the doc which currently reads: For example using 100 times more landmarks result in the same transformation Interestingly if you do not choose correctly, the linear interpolation done afterwards does not correspond to the numpy.percentile(Xtrain, .) estimator. This is not "wrong" as these are only estimators of the true quantile function/cdf but I think it is confusing and would be better to stick with the original estimator. For instance, the following raises an AssertionError.

Attributes     ----------<ins>    n_quantiles_ : integer        The actual number of quantiles used to discretize the cumulative        distribution function.</ins>     quantiles_ : ndarray, shape (n_quantiles, n_features)         The values corresponding the quantiles of reference.


Question: DOC Improve doc of nquantiles in QuantileTransformer The uses numpy.percentile(Xtrain, .) as the estimator of the quantile function of the training data. To know this function perfectly we just need to take . Then it is just a linear interpolation (which is done in the code afterwards). Therefore I don't think we should be able to choose and we should prevent users from thinking that the higher the better the transformation. As mentioned by IRL it is however true that it can be relevant to choose when is very large. I suggest to add more information on the impact of in the doc which currently reads: For example using 100 times more landmarks result in the same transformation Interestingly if you do not choose correctly, the linear interpolation done afterwards does not correspond to the numpy.percentile(Xtrain, .) estimator. This is not "wrong" as these are only estimators of the true quantile function/cdf but I think it is confusing and would be better to stick with the original estimator. For instance, the following raises an AssertionError.

self.subsample))          X = self._check_inputs(X)        n_samples = X.shape[0]        if self.n_quantiles > n_samples:            warnings.warn("n_quantiles (%s) is greater than the total number "                          "of samples (%s). n_quantiles is set to "                          "n_samples."                          % (self.n_quantiles, n_samples))        self.n_quantiles_ = max(1, min(self.n_quantiles, n_samples))         rng = check_random_state(self.random_state)          # Create the quantiles of reference        self.references_ = np.linspace(0, 1, self.n_quantiles,        self.references_ = np.linspace(0, 1, self.n_quantiles_,                                        endpoint=True)         if sparse.issparse(X):             self._sparse_fit(X, rng)


Question: DOC Improve doc of nquantiles in QuantileTransformer The uses numpy.percentile(Xtrain, .) as the estimator of the quantile function of the training data. To know this function perfectly we just need to take . Then it is just a linear interpolation (which is done in the code afterwards). Therefore I don't think we should be able to choose and we should prevent users from thinking that the higher the better the transformation. As mentioned by IRL it is however true that it can be relevant to choose when is very large. I suggest to add more information on the impact of in the doc which currently reads: For example using 100 times more landmarks result in the same transformation Interestingly if you do not choose correctly, the linear interpolation done afterwards does not correspond to the numpy.percentile(Xtrain, .) estimator. This is not "wrong" as these are only estimators of the true quantile function/cdf but I think it is confusing and would be better to stick with the original estimator. For instance, the following raises an AssertionError.

Axis used to compute the means and standard deviations along. If 0,         transform each feature, otherwise (if 1) transform each sample.     n_quantiles : int, optional (default=1000)    n_quantiles : int, optional (default=1000 or n_samples)         Number of quantiles to be computed. It corresponds to the number         of landmarks used to discretize the cumulative distribution function.        If n_quantiles is larger than the number of samples, n_quantiles is set        to the number of samples as a larger number of quantiles does not give        a better approximation of the cumulative distribution function        estimator.      output_distribution : str, optional (default='uniform')         Marginal distribution for the transformed data. The choices are


Question: returnintercept==True in ridgeregression raises an exception <!-- If your issue is a usage question, submit it here instead: StackOverflow with the scikit-learn tag: Mailing List: For more information, see User Questions: http://scikit- --<!-- Instructions For Filing a Bug: --<!-- Example: Joblib Error thrown when calling fit on LatentDirichletAllocation with evaluateevery 0--<!-- Example: No error is thrown. Please paste or describe the expected results.--(the values can differ, but at least no exception should be raised) <!-- Please paste or specifically describe the actual output or traceback. --<!-- Please run the following snippet and paste the output below. For scikit-learn = 0.20: import sklearn; sklearn.showversions() For scikit-learn < 0.20: import platform; print(platform.platform()) import sys; print("Python", sys.version) import numpy; print("NumPy", numpy.version) import scipy; print("SciPy", scipy.version) import sklearn; print("Scikit-Learn", sklearn.version) --<!-- Thanks for contributing! --

return_n_iter=False, return_intercept=False,                       X_scale=None, X_offset=None):     if return_intercept and sparse.issparse(X) and solver != 'sag':        if solver != 'auto':            warnings.warn("In Ridge, only 'sag' solver can currently fit the "                          "intercept when X is sparse. Solver has been "                          "automatically changed into 'sag'.")        solver = 'sag'    has_sw = sample_weight is not None    if solver == 'auto':        if return_intercept:            # only sag supports fitting intercept directly            solver = "sag"        elif not sparse.issparse(X):            solver = "cholesky"        else:            solver = "sparse_cg"    if solver not in ('sparse_cg', 'cholesky', 'svd', 'lsqr', 'sag', 'saga'):        raise ValueError("Known solvers are 'sparse_cg', 'cholesky', 'svd'"                         " 'lsqr', 'sag' or 'saga'. Got %s." % solver)    if return_intercept and solver != 'sag':        raise ValueError("In Ridge, only 'sag' solver can directly fit the "                         "intercept. Please change solver to 'sag' or set "                         "return_intercept=False.")      _dtype = [np.float64, np.float32]


Question: returnintercept==True in ridgeregression raises an exception <!-- If your issue is a usage question, submit it here instead: StackOverflow with the scikit-learn tag: Mailing List: For more information, see User Questions: http://scikit- --<!-- Instructions For Filing a Bug: --<!-- Example: Joblib Error thrown when calling fit on LatentDirichletAllocation with evaluateevery 0--<!-- Example: No error is thrown. Please paste or describe the expected results.--(the values can differ, but at least no exception should be raised) <!-- Please paste or specifically describe the actual output or traceback. --<!-- Please run the following snippet and paste the output below. For scikit-learn = 0.20: import sklearn; sklearn.showversions() For scikit-learn < 0.20: import platform; print(platform.platform()) import sys; print("Python", sys.version) import numpy; print("NumPy", numpy.version) import scipy; print("SciPy", scipy.version) import sklearn; print("Scikit-Learn", sklearn.version) --<!-- Thanks for contributing! --

raise ValueError("Number of samples in X and y does not correspond:"                          " %d != %d" % (n_samples, n_samples_))     has_sw = sample_weight is not None     if solver == 'auto':        # cholesky if it's a dense array and cg in any other case        if not sparse.issparse(X) or has_sw:            solver = 'cholesky'        else:            solver = 'sparse_cg'      if has_sw:         if np.atleast_1d(sample_weight).ndim > 1:


Question: returnintercept==True in ridgeregression raises an exception <!-- If your issue is a usage question, submit it here instead: StackOverflow with the scikit-learn tag: Mailing List: For more information, see User Questions: http://scikit- --<!-- Instructions For Filing a Bug: --<!-- Example: Joblib Error thrown when calling fit on LatentDirichletAllocation with evaluateevery 0--<!-- Example: No error is thrown. Please paste or describe the expected results.--(the values can differ, but at least no exception should be raised) <!-- Please paste or specifically describe the actual output or traceback. --<!-- Please run the following snippet and paste the output below. For scikit-learn = 0.20: import sklearn; sklearn.showversions() For scikit-learn < 0.20: import platform; print(platform.platform()) import sys; print("Python", sys.version) import numpy; print("NumPy", numpy.version) import scipy; print("SciPy", scipy.version) import sklearn; print("Scikit-Learn", sklearn.version) --<!-- Thanks for contributing! --

if alpha.size == 1 and n_targets > 1:         alpha = np.repeat(alpha, n_targets) <del>    if solver not in ('sparse_cg', 'cholesky', 'svd', 'lsqr', 'sag', 'saga'):        raise ValueError('Solver %s not understood' % solver)</del>      n_iter = None     if solver == 'sparse_cg':