Question: 'shape (%d, %d) or (%d, %d)' % (                         coef.shape[0], coef.shape[1], classes.size,                         n_features, classes.size, n_features + 1))            w0[:, :coef.shape[1]] = coef            if n_classes == 1:                w0[0, :coef.shape[1]] = -coef                w0[1, :coef.shape[1]] = coef            else:                w0[:, :coef.shape[1]] = coef      if multi_class == 'multinomial':         # fmin_l_bfgs_b and newton-cg accepts only ravelled parameters.

# Set dependant variable values Y = ([1, 0, 0, 1, 0, 0, 0, 0, 0, 0, 1, 1, 1, 0, 0, 1, 0, 0, 1, 1], dtype=np.int32) # First fit model normally (Z, Y) p = lr.predictproba(Z) print(sklearn.metrics.logloss(Y, p)) # ... print(lr.intercept) print() # Now fit model after a warm start (Z, Y) p = lr.predictproba(Z) print(sklearn.metrics.logloss(Y, p)) # ... print(lr.intercept) print() The predictions should be the same as the model converged the first time it was run. The predictions are different. In fact the more times you re-run the fit the worse it gets. This is actually the only reason I was able to catch the bug. It is caused by the line here w0[:, [1]] = coef As is , but is , this causes the value to be broadcast into the . This some sort of singularity issue when training resulting in worse performance. Note that had it not done exactly this i.e. was simply initialised by some random values, this bug would be very hard to catch because of course each time the model would converge just not as fast as one would hope when warm starting. The fix I believe is very easy, just need to swap the previous line to if nclasses == 1: w0[0, [1]] = -coef # Be careful to get these the right way around w0[1, [1]] = coef else: w0[:, [1]] = coef <!-- Please run the following snippet and paste the output below.


Question: 'shape (%d, %d) or (%d, %d)' % (                         coef.shape[0], coef.shape[1], classes.size,                         n_features, classes.size, n_features + 1))            w0[:, :coef.shape[1]] = coef            if n_classes == 1:                w0[0, :coef.shape[1]] = -coef                w0[1, :coef.shape[1]] = coef            else:                w0[:, :coef.shape[1]] = coef      if multi_class == 'multinomial':         # fmin_l_bfgs_b and newton-cg accepts only ravelled parameters.

import platform; print(platform.platform()) import sys; print("Python", sys.version) import numpy; print("NumPy", numpy.version) import scipy; print("SciPy", scipy.version) import sklearn; print("Scikit-Learn", sklearn.version) --Linux-4.13.0-37-generic-x86_64-with-Ubuntu-16.04-xenial Python 3.5.2 (default, Nov 23 2017, 16:37:01) NumPy 1.14.2 SciPy 1.0.0 Scikit-Learn 0.20.dev0 (built from latest master) <!-- Thanks for contributing! --


Question: "Expected n_neighbors > 0. Got %d" %                     self.n_neighbors                 )<ins>            else:                if not np.issubdtype(type(self.n_neighbors), np.integer):                    raise TypeError(                        "n_neighbors does not take %s value, "                        "enter integer value" %                        type(self.n_neighbors))</ins>          return self

Missing parameter validation in Neighbors estimator for float n_neighbors This should be caught earlier and a more helpful error message should be raised (or we could be lenient and cast to integer, but I think a better error might be better). We need to make sure that also works.


Question: if n_neighbors is None:             n_neighbors = self.n_neighbors<ins>        elif n_neighbors <= 0:            raise ValueError(                "Expected n_neighbors > 0. Got %d" %                n_neighbors            )        else:            if not np.issubdtype(type(n_neighbors), np.integer):                raise TypeError(                    "n_neighbors does not take %s value, "                    "enter integer value" %                    type(n_neighbors))</ins>          if X is not None:             query_is_train = False


Question: return np.hstack((np.ones((n_samples, 1)) * value, X))  def _transform_selected(X, transform, selected="all", copy=True):def _transform_selected(X, transform, dtype, selected="all", copy=True):     """Apply a transform function to portion of selected features      Parameters

OneHotEncoder does not output scipy sparse matrix of given dtype OneHotEncoder ignores the specified dtype in the construction of the sparse array when mixed input data are passed, i.e with both categorical and real data type Platform: Linux-4.13.0-38-generic-x86_64-with-debian-stretch-sid Python: 3.6.3 (default, Oct 13 2017, 12:02:49) [GCC 7.2.0] NumPy: NumPy SciPy: SciPy 1.0.1 Scikit-Learn: Scikit-Learn 0.19.1


Question: transform : callable         A callable transform(X) -> X_transformed <ins>    dtype : number type        Desired dtype of output.</ins>     copy : boolean, optional         Copy X even if it could be avoided.


Question: return transform(X)     else:         X_sel = transform(X[:, ind[sel]])        X_not_sel = X[:, ind[not_sel]]        # The columns of X which are not transformed need        # to be casted to the desire dtype before concatenation.        # Otherwise, the stacking will cast to the higher-precision dtype.        X_not_sel = X[:, ind[not_sel]].astype(dtype)          if sparse.issparse(X_sel) or sparse.issparse(X_not_sel):             return sparse.hstack((X_sel, X_not_sel))


Question: X : array-like, shape [n_samples, n_feature]             Input array of type int.         """        return _transform_selected(X, self._fit_transform,        return _transform_selected(X, self._fit_transform, self.dtype,                                    self.categorical_features, copy=True)      def _transform(self, X):


Question: X_out : sparse matrix if sparse=True else a 2-d array, dtype=int             Transformed input.         """        return _transform_selected(X, self._transform,        return _transform_selected(X, self._transform, self.dtype,                                    self.categorical_features, copy=True)


Question: validate : bool, optional default=True         Indicate that the input X array should be checked before calling        func. If validate is false, there will be no input validation.        If it is true, then X will be converted to a 2-dimensional NumPy        array or sparse matrix. If this conversion is not possible or X        contains NaN or infinity, an exception is raised.        ``func``. The possibilities are:        - If False, there is no input validation.        - If True, then X will be converted to a 2-dimensional NumPy array or          sparse matrix. If the conversion is not possible an exception is          raised.        .. deprecated:: 0.20           ``validate=True`` as default will be replaced by           ``validate=False`` in 0.22.      accept_sparse : boolean, optional         Indicate that func accepts a sparse matrix as input. If validate is

FunctionTransformer should not convert DataFrames to arrays by default I would expect a common use of FunctionTransformer is to apply some function to a Pandas DataFrame, ideally using its own methods or accessors. As noted in , it can be easy for users to miss that they need to set validate=False to pass through a DataFrame without converting it to a NumPy array. I think it would be more user-friendly to have by default, which would pass through DataFrames to the function, but otherwise convert its input to a 2d array. For strict backwards compatibility, the default should be changed through a deprecation cycle, warning whenever using the default validation means a DataFrame is currently converted to an array. Do others agree?


Question: Dictionary of additional keyword arguments to pass to inverse_func.      """    def __init__(self, func=None, inverse_func=None, validate=True,    def __init__(self, func=None, inverse_func=None, validate=None,                  accept_sparse=False, pass_y='deprecated', check_inverse=True,                  kw_args=None, inv_kw_args=None):         self.func = func


Question: self.kw_args = kw_args         self.inv_kw_args = inv_kw_args <ins>    def _check_input(self, X):        # FIXME: Future warning to be removed in 0.22        if self.validate is None:            self._validate = True            warnings.warn("The default validate=True will be replaced by "                          "validate=False in 0.22.", FutureWarning)        else:            self._validate = self.validate        if self._validate:            return check_array(X, accept_sparse=self.accept_sparse)        return X</ins>     def _check_inverse_transform(self, X):         """Check that func and inverse_func are the inverse."""         idx_selected = slice(None, None, max(1, X.shape[0] // 100))


Question: -------         self         """        if self.validate:            X = check_array(X, self.accept_sparse)        X = self._check_input(X)         if (self.check_inverse and not (self.func is None or                                         self.inverse_func is None)):             self._check_inverse_transform(X)


Question: kw_args=self.inv_kw_args)      def _transform(self, X, y=None, func=None, kw_args=None):        if self.validate:            X = check_array(X, self.accept_sparse)        X = self._check_input(X)          if func is None:             func = _identity


Question: for check in _yield_all_checks(name, estimator):         try:             check(name, estimator)        except SkipTest as message:        except SkipTest as exception:             # the only SkipTest thrown currently results from not             # being able to import pandas.            warnings.warn(message, SkipTestWarning)            warnings.warn(str(exception), SkipTestWarning)   def _boston_subset(n_samples=200):

catch more warnings in common tests Right now the output of travis is too large to render, partially because a lot of deprecation warnings and partially because of warnings from the common tests. The common tests should catch all the deprecation warnings and probably also convergence warnings or numerical warnings (we decrease the number of iterations for faster testing).


Question: and not isinstance(estimator, BaseSGD)):         estimator.set_params(n_iter=5)     if "max_iter" in params:        warnings.simplefilter("ignore", ConvergenceWarning)         if estimator.max_iter is not None:             estimator.set_params(max_iter=min(5, estimator.max_iter))         # LinearSVR, LinearSVCdiff --git a/sklearn/utils/testing.py b/sklearn/utils/testing.py-- a/sklearn/utils/testing.py++ b/sklearn/utils/testing.py


Question: result : the return value of `func`      """<del>    # very important to avoid uncontrolled state propagation</del>     clean_warning_registry()     with warnings.catch_warnings(record=True) as w:         # Cause all warnings to always be triggered.


Question: """Decorator to catch and hide warnings without visual nesting."""         @wraps(fn)         def wrapper(*args, **kwargs):<del>            # very important to avoid uncontrolled state propagation</del>             clean_warning_registry()             with warnings.catch_warnings():                 warnings.simplefilter("ignore", self.category)


Question: return "%s(%s)" % (name, ", ".join(args))      def __enter__(self):        clean_warning_registry()  # be safe and not propagate state + chaos        warnings.simplefilter("ignore", self.category)         if self._entered:             raise RuntimeError("Cannot enter %r twice" % self)         self._entered = True         self._filters = self._module.filters         self._module.filters = self._filters[:]         self._showwarning = self._module.showwarning        clean_warning_registry()        warnings.simplefilter("ignore", self.category)      def __exit__(self, *exc_info):         if not self._entered:


Question: self._module.filters = self._filters         self._module.showwarning = self._showwarning         self.log[:] = []        clean_warning_registry()  # be safe and not propagate state + chaos        clean_warning_registry()   assert_less = _dummy.assertLess


Question: def clean_warning_registry():    """Safe way to reset warnings."""    warnings.resetwarnings()    """Clean Python warning registry for easier testing of warning messages.    We may not need to do this any more when getting rid of Python 2, not    entirely sure. See https://bugs.python.org/issue4180 and    https://bugs.python.org/issue21724 for more details.    """     reg = "__warningregistry__"     for mod_name, mod in list(sys.modules.items()):         if 'six.moves' in mod_name:


Question: def classification_report(y_true, y_pred, labels=None, target_names=None,                          sample_weight=None, digits=2):                          sample_weight=None, digits=2, output_dict=False):     """Build a text report showing the main classification metrics      Read more in the :ref:`User Guide <classification_report>`.

output options? Is it possible to add output options to http://scikit- It would be really useful to have a output or output. Right now it outputs as a string that must be printed but it's difficult to use the results. I can make a quick helper script if that could be useful? [MRG] Classification report Dict-of-Dicts output <!-- Thanks for contributing a pull request! Please ensure you have taken a look at the contribution guidelines: --<!-- Example: -- This PR adds an option of returning the classification report in the form of a Dictionary of Dictionaries. Will add tests for the code, if the code is approved. <!-- Please be aware that we are a loose team of volunteers so patience is necessary; assistance handling other issues is very welcome. We value all user contributions, no matter how minor they are. If we are slow to review, either the pull request needs some benchmarking, tinkering, convincing, etc. or more likely the reviewers are simply busy. In either case, we ask for your understanding during the review process. For more information, see our FAQ on this topic: http://scikit- Thanks for contributing! --


Question: digits : int         Number of digits for formatting output floating point values     output_dict: bool (default = False)        If True, return output as dict     Returns     -------    report : string    report : string / dict         Text summary of the precision, recall, F1 score for each class.        Dictionary returned if output_dict is True. Dictionary has the        following structure:            {'label 1': {'precision':0.5,                         'recall':1.0,                         'f1-score':0.67,                         'support':1},             'label 2': { ... },              ...            }          The reported averages are a prevalence-weighted macro-average across         classes (equivalent to :func:`precision_recall_fscore_support` with


Question: row_fmt = u'{:>{width}s} ' + u' {:>9.{digits}f}' * 3 + u' {:>9}\n'     rows = zip(target_names, p, r, f1, s)    avg_total = [np.average(p, weights=s),                 np.average(r, weights=s),                 np.average(f1, weights=s),                 np.sum(s)]    if output_dict:        report_dict = {label[0]: label[1:] for label in rows}        for label, scores in report_dict.items():            report_dict[label] = dict(zip(headers, scores))        report_dict['avg / total'] = dict(zip(headers, avg_total))        return report_dict     for row in rows:         report += row_fmt.format(*row, width=width, digits=digits)      report += u'\n'     # compute averages    # append averages     report += row_fmt.format(last_line_heading,                             np.average(p, weights=s),                             np.average(r, weights=s),                             np.average(f1, weights=s),                             np.sum(s),                             *avg_total,                              width=width, digits=digits)      return report


Question: else:                 self.mean_ = None                 self.var_ = None<ins>                if not hasattr(self, 'n_samples_seen_'):                    self.n_samples_seen_ = X.shape[0]                else:                    self.n_samples_seen_ += X.shape[0]</ins>         else:             # First pass             if not hasattr(self, 'n_samples_seen_'):

Consistency issue in StandardScaler There is an issue of consistency with with and between the sparse and dense case. it make sense to support this case. It will return the identity matrix which is not the use case for the . If we wish a transformer to do so, one should use the I assume. we consider this behaviour normal, we need to: In the dense case, force to be after each iteration of . In the sparse case, compute the non-NaNs values and update which is not computed. It leads currently to an error if calling twice (i.e. will fail). IMO, we should make a checking at raise an error. WDYT?


Question: else:                     self.var_ = None             self.mean_, self.var_, self.n_samples_seen_ = \                _incremental_mean_and_var(X, self.mean_, self.var_,                                          self.n_samples_seen_)            if not self.with_mean and not self.with_std:                self.mean_ = None                self.var_ = None                self.n_samples_seen_ += X.shape[0]            else:                self.mean_, self.var_, self.n_samples_seen_ = \                    _incremental_mean_and_var(X, self.mean_, self.var_,                                              self.n_samples_seen_)          if self.with_std:             self.scale_ = _handle_zeros_in_scale(np.sqrt(self.var_))


Question: # Unlike the scaler object, this function allows 1d input.     # If copy is required, it will be done inside the scaler object.     X = check_array(X, copy=False, ensure_2d=False, warn_on_dtype=True,                    dtype=FLOAT_DTYPES)                    dtype=FLOAT_DTYPES, force_all_finite='allow-nan')     original_ndim = X.ndim      if original_ndim == 1:

minmaxscale does not ignore NaNs The class ignore NaNs. Its counterpart function does not. The needs to add the option . implement the tests and this fix. However, it should be done in another proper PR.


Question: import warnings  import numpy as npfrom scipy import linalgfrom scipy.sparse import issparse, csr_matrixfrom scipy import linalg, sparse  from . import check_random_state, deprecated from .fixes import np_version

randomizedsvd is slow for dokmatrix and lilmatrix (and its object-oriented interface, ) is extremely slow for certain types of sparse matrix. Either all sparse matrices should be processed in roughly the same amount of time, or a warning should be printed. silently takes up to 50x longer than necessary. Windows-10-10.0.-SP0 Python 3.6.0 (default, Dec 23 2016, 11:57:41) [MSC v.1900 64 bit (AMD64)] NumPy 1.14.4 SciPy 1.1.0 Scikit-Learn 0.19.1 Also tested on: Linux-4.16.11-1-ARCH-x8664-with-arch-Arch-Linux Python 3.6.5 (default, May 11 2018, 04:00:52) [GCC 8.1.0] NumPy 1.14.5 SciPy 1.1.0 Scikit-Learn 0.19.1


Question: Performs no input validation.     """    if issparse(X):        if not isinstance(X, csr_matrix):            X = csr_matrix(X)    if sparse.issparse(X):        if not isinstance(X, sparse.csr_matrix):            X = sparse.csr_matrix(X)         norms = csr_row_norms(X)     else:         norms = np.einsum('ij,ij->i', X, X)


Question: dot_product : array or sparse matrix         sparse if ``a`` or ``b`` is sparse and ``dense_output=False``.     """    if issparse(a) or issparse(b):    if sparse.issparse(a) or sparse.issparse(b):         ret = a * b         if dense_output and hasattr(ret, "toarray"):             ret = ret.toarray()


Question: analysis       A. Szlam et al. 2014     """<ins>    if isinstance(M, (sparse.lil_matrix, sparse.dok_matrix)):        warnings.warn("Calculating SVD of a {} is expensive. "                      "csr_matrix is more efficient.".format(                          type(M).__name__),                      sparse.SparseEfficiencyWarning)</ins>     random_state = check_random_state(random_state)     n_random = n_components + n_oversamples     n_samples, n_features = M.shape


Question: Adapated from http://stackoverflow.com/q/13426580      """    if issparse(X):    if sparse.issparse(X):         if len(X.data) == 0:             return 0         m = X.data.min()


Question: """Ensure `X.min()` >= `min_value`."""     min_ = safe_min(X)     if min_ < min_value:        if issparse(X):        if sparse.issparse(X):             raise ValueError("Cannot make the data matrix"                              " nonnegative because it is sparse."                              " Adding a value to every entry would"


Question: def fit(self, X, y=None):         """Estimate model parameters with the EM algorithm.         The method fit the model `n_init` times and set the parameters with        The method fits the model `n_init` times and set the parameters with         which the model has the largest likelihood or lower bound. Within each         trial, the method iterates between E-step and M-step for `max_iter`         times until the change of likelihood or lower bound is less than

Should mixture models have a clusterer-compatible interface Mixture models are currently a bit different. They are basically clusterers, except they are probabilistic, and are applied to inductive problems unlike many clusterers. But they are unlike clusterers in API: they have an parameter, with identical purpose to they do not store the of the training data they do not have a method And they are almost entirely documented separately. Should we make the MMs more like clusterers?


Question: -------         self         """<ins>        self.fit_predict(X, y)        return self    def fit_predict(self, X, y=None):        """Estimate model parameters using X and predict the labels for X.        The method fits the model n_init times and sets the parameters with        which the model has the largest likelihood or lower bound. Within each        trial, the method iterates between E-step and M-step for `max_iter`        times until the change of likelihood or lower bound is less than        `tol`, otherwise, a `ConvergenceWarning` is raised. After fitting, it        predicts the most probable label for the input data points.        .. versionadded:: 0.20        Parameters        ----------        X : array-like, shape (n_samples, n_features)            List of n_features-dimensional data points. Each row            corresponds to a single data point.        Returns        -------        labels : array, shape (n_samples,)            Component labels.        """</ins>         X = _check_X(X, self.n_components, ensure_min_samples=2)         self._check_initial_parameters(X)


Question: self._set_parameters(best_params)         self.n_iter_ = best_n_iter         return self        return log_resp.argmax(axis=1)      def _e_step(self, X):         """E step.


Question: from functools import partial, reduce from itertools import product import operator<ins>import time</ins> import warnings  import numpy as np

Retrieving time to refit the estimator in BaseSearchCV Basically, I'm trying to figure out how much time it takes to refit the best model on the full data after doing grid/random search. What I can so far do is retrieve the time it takes to fit and score each model: In case I run this on a single core, I could time the whole search procedure and subtract the time it took to fit the single folds during hyperparameter optimization. Nevertheless, this isn't possible any more when setting . Thus, it would be great to have an attribute which is simply the time it took to refit the best model. Usecase: for we want to support uploading the results of hyperparameter optimization, including the time it takes to do the hyperparameter optimization.


Question: if self.refit:             self.best_estimator_ = clone(base_estimator).set_params(                 **self.best_params_)            refit_start_time = time.time()             if y is not None:                 self.best_estimator_.fit(X, y, **fit_params)             else:                 self.best_estimator_.fit(X, **fit_params)            refit_end_time = time.time()            self.refit_time_ = refit_end_time - refit_start_time          # Store the only scorer not as a dict for single metric evaluation         self.scorer_ = scorers if self.multimetric_ else scorers['score']


Question: n_splits_ : int         The number of cross-validation splits (folds/iterations). <ins>    refit_time_ : float        Seconds used for refitting the best model on the whole dataset.        This is present only if ``refit`` is not False.</ins>     Notes     ------     The parameters selected are those that maximize the score of the left out


Question: n_splits_ : int         The number of cross-validation splits (folds/iterations). <ins>    refit_time_ : float        Seconds used for refitting the best model on the whole dataset.        This is present only if ``refit`` is not False.</ins>     Notes     -----     The parameters selected are those that maximize the score of the held-out


Question: if attr in params:             setattr(self, attr, params.pop(attr))         # 2. Step replacement        names, _ = zip(*getattr(self, attr))        items = getattr(self, attr)        names = []        if items:            names, _ = zip(*items)         for name in list(six.iterkeys(params)):             if '__' not in name and name in names:                 self._replace_estimator(attr, name, params.pop(name))

BaseCompostion.setparams broken where there are no estimators raises an error when the composition has no estimators. This is a marginal case, but it might be interesting to support alongside .


Question: """     Gram = Gram.copy('F') if copy_Gram else np.asfortranarray(Gram)     if copy_Xy:    if copy_Xy or not Xy.flags.writeable:         Xy = Xy.copy()      min_float = np.finfo(Gram.dtype).eps

ValueError: assignment destination is read-only, when paralleling with njobs 1 When I run with njobs 1, there is a chance to raise exception . The code is shown as follow: The bigger is, the higher chance get. When is small (lower than 2000, I verified), everything works fine. Once is bigger than 2000, there is a chance to get the exception. When is bigger than 5000, it is 100% raised. My version infor: OS: OS X 10.11.1 python: Python 2.7.10 |Anaconda 2.2.0 numpy: 1.10.1 sklearn: 0.17 The full error information is shown as follow


Question: Xy = Xy[:, np.newaxis]         if tol is not None:             norms_squared = [norms_squared]<ins>    if copy_Xy or not Xy.flags.writeable:        # Make the copy once instead of many times in _gram_omp itself.        Xy = Xy.copy()</ins>      if n_nonzero_coefs is None and tol is None:         n_nonzero_coefs = int(0.1 * len(Gram))


Question: out = _gram_omp(             Gram, Xy[:, k], n_nonzero_coefs,             norms_squared[k] if tol is not None else None, tol,            copy_Gram=copy_Gram, copy_Xy=copy_Xy,            copy_Gram=copy_Gram, copy_Xy=False,             return_path=return_path)         if return_path:             _, idx, coefs, n_iter = out


Question: ]  <ins>def _check_inputs_dtype(X, missing_values):    if (X.dtype.kind in ("f", "i", "u") and            not isinstance(missing_values, numbers.Real)):        raise ValueError("'X' and 'missing_values' types are expected to be"                         " both numerical. Got X.dtype={} and "                         " type(missing_values)={}."                         .format(X.dtype, type(missing_values)))</ins> def _get_mask(X, value_to_mask):     """Compute the boolean mask X == missing_values."""     if value_to_mask is np.nan:

Cryptic error in imputers due to missing checking in getmask By working on the it seems that there is a missing checking between and dtype: In short, is raised by equal in case of numeric and string mixed dtype. We should put a check in which is shared across classes.


Question: else:             # np.isnan does not work on object dtypes.             return _object_dtype_isnan(X)<del></del>     else:         # X == value_to_mask with object dytpes does not always perform         # element-wise for old versions of numpy


Question: else:                 raise ve <ins>        _check_inputs_dtype(X, self.missing_values)</ins>         if X.dtype.kind not in ("i", "u", "f", "O"):             raise ValueError("SimpleImputer does not support data with dtype "                              "{0}. Please provide either a numeric array (with"


Question: X = check_array(X, dtype=FLOAT_DTYPES, order="F",                         force_all_finite=force_all_finite)<ins>        _check_inputs_dtype(X, self.missing_values)</ins>          mask_missing_values = _get_mask(X, self.missing_values)         if self.initial_imputer_ is None:


Question: a new copy will always be made, even if `copy=False`:          - If X is not an array of floating values;<del>        - If X is sparse and `missing_values=0`;</del>         - If X is encoded as a CSR matrix.      Attributes

BUG: SimpleImputer gives wrong result on sparse matrix with explicit zeros The current implementation of the can't deal with zeros stored explicitly in sparse matrix. Even when stored explicitly, we'd expect that all zeros are treating equally, right ? See for example the code below: Whereas the expected result would be


Question: "data".format(fill_value))          if sparse.issparse(X):            self.statistics_ = self._sparse_fit(X,                                                self.strategy,                                                self.missing_values,                                                fill_value)            # missing_values = 0 not allowed with sparse data as it would            # force densification            if self.missing_values == 0:                raise ValueError("Imputation not possible when missing_values "                                 "== 0 and input is sparse. Provide a dense "                                 "array instead.")            else:                self.statistics_ = self._sparse_fit(X,                                                    self.strategy,                                                    self.missing_values,                                                    fill_value)         else:             self.statistics_ = self._dense_fit(X,                                                self.strategy,


Question: def _sparse_fit(self, X, strategy, missing_values, fill_value):         """Fit the transformer on sparse data."""        # Count the zeros        if missing_values == 0:            n_zeros_axis = np.zeros(X.shape[1], dtype=int)        else:            n_zeros_axis = X.shape[0] - np.diff(X.indptr)        # Mean        if strategy == "mean":            if missing_values != 0:                n_non_missing = n_zeros_axis                # Mask the missing elements                mask_missing_values = _get_mask(X.data, missing_values)                mask_valids = np.logical_not(mask_missing_values)                # Sum only the valid elements                new_data = X.data.copy()                new_data[mask_missing_values] = 0                X = sparse.csc_matrix((new_data, X.indices, X.indptr),                                      copy=False)                sums = X.sum(axis=0)                # Count the elements != 0                mask_non_zeros = sparse.csc_matrix(                    (mask_valids.astype(np.float64),                     X.indices,                     X.indptr), copy=False)                s = mask_non_zeros.sum(axis=0)                n_non_missing = np.add(n_non_missing, s)        mask_data = _get_mask(X.data, missing_values)        n_implicit_zeros = X.shape[0] - np.diff(X.indptr)             else:                sums = X.sum(axis=0)                n_non_missing = np.diff(X.indptr)        statistics = np.empty(X.shape[1])             # Ignore the error, columns with a np.nan statistics_            # are not an error at this point. These columns will            # be removed in transform            with np.errstate(all="ignore"):                return np.ravel(sums) / np.ravel(n_non_missing)        if strategy == "constant":            # for constant strategy, self.statistcs_ is used to store            # fill_value in each column            statistics.fill(fill_value)         # Median + Most frequent + Constant         else:            # Remove the missing values, for each column            columns_all = np.hsplit(X.data, X.indptr[1:-1])            mask_missing_values = _get_mask(X.data, missing_values)            mask_valids = np.hsplit(np.logical_not(mask_missing_values),                                    X.indptr[1:-1])            # astype necessary for bug in numpy.hsplit before v1.9            columns = [col[mask.astype(bool, copy=False)]                       for col, mask in zip(columns_all, mask_valids)]            # Median            if strategy == "median":                median = np.empty(len(columns))                for i, column in enumerate(columns):                    median[i] = _get_median(column, n_zeros_axis[i])                return median            # Most frequent            elif strategy == "most_frequent":                most_frequent = np.empty(len(columns))                for i, column in enumerate(columns):                    most_frequent[i] = _most_frequent(column,                                                      0,                                                      n_zeros_axis[i])                return most_frequent            # Constant            elif strategy == "constant":                return np.full(X.shape[1], fill_value)            for i in range(X.shape[1]):                column = X.data[X.indptr[i]:X.indptr[i+1]]                mask_column = mask_data[X.indptr[i]:X.indptr[i+1]]                column = column[~mask_column]                # combine explicit and implicit zeros                mask_zeros = _get_mask(column, 0)                column = column[~mask_zeros]                n_explicit_zeros = mask_zeros.sum()                n_zeros = n_implicit_zeros[i] + n_explicit_zeros                if strategy == "mean":                    s = column.size + n_zeros                    statistics[i] = np.nan if s == 0 else column.sum() / s                elif strategy == "median":                    statistics[i] = _get_median(column,                                                n_zeros)                elif strategy == "most_frequent":                    statistics[i] = _most_frequent(column,                                                   0,                                                   n_zeros)        return statistics      def _dense_fit(self, X, strategy, missing_values, fill_value):         """Fit the transformer on dense data."""


Question: # Constant         elif strategy == "constant":<ins>            # for constant strategy, self.statistcs_ is used to store            # fill_value in each column</ins>             return np.full(X.shape[1], fill_value, dtype=X.dtype)      def transform(self, X):


Question: X = X[:, valid_statistics_indexes]          # Do actual imputation        if sparse.issparse(X) and self.missing_values != 0:            mask = _get_mask(X.data, self.missing_values)            indexes = np.repeat(np.arange(len(X.indptr) - 1, dtype=np.int),                                np.diff(X.indptr))[mask]        if sparse.issparse(X):            if self.missing_values == 0:                raise ValueError("Imputation not possible when missing_values "                                 "== 0 and input is sparse. Provide a dense "                                 "array instead.")            else:                mask = _get_mask(X.data, self.missing_values)                indexes = np.repeat(np.arange(len(X.indptr) - 1, dtype=np.int),                                    np.diff(X.indptr))[mask]             X.data[mask] = valid_statistics[indexes].astype(X.dtype,                                                            copy=False)                X.data[mask] = valid_statistics[indexes].astype(X.dtype,                                                                copy=False)         else:            if sparse.issparse(X):                X = X.toarray()             mask = _get_mask(X, self.missing_values)             n_missing = np.sum(mask, axis=0)             values = np.repeat(valid_statistics, n_missing)


Question: on the decision function. If 'auto', the decision function threshold is         determined as in the original paper. <ins>        .. versionchanged:: 0.20           The default value of ``contamination`` will change from 0.1 in 0.20           to ``'auto'`` in 0.22.</ins>     max_features : int or float, optional (default=1.0)         The number of features to draw from X to train each base estimator.

IsolationForest contamination deprecation in init not in fit need to move the deprecation and fix the tests.


Question: n_jobs=n_jobs,             random_state=random_state,             verbose=verbose)<del>        if contamination == "legacy":            warnings.warn('default contamination parameter 0.1 will change '                          'in version 0.22 to "auto". This will change the '                          'predict method behavior.',                          DeprecationWarning)</del>         self.contamination = contamination      def _set_oob_score(self, X, y):


Question: -------         self : object         """<ins>        if self.contamination == "legacy":            warnings.warn('default contamination parameter 0.1 will change '                          'in version 0.22 to "auto". This will change the '                          'predict method behavior.',                          FutureWarning)            self._contamination = 0.1        else:            self._contamination = self.contamination</ins>         X = check_array(X, accept_sparse=['csc'])         if issparse(X):             # Pre-sort indices to avoid that each individual tree of the


Question: max_depth=max_depth,                                           sample_weight=sample_weight)         if self.contamination == "auto":        if self._contamination == "auto":             # 0.5 plays a special role as described in the original paper.             # we take the opposite as we consider the opposite of their score.             self.offset_ = -0.5             # need to save (depreciated) threshold_ in this case:             self._threshold_ = sp.stats.scoreatpercentile(                 self.score_samples(X), 100. * 0.1)        elif self.contamination == "legacy":  # to be rm in 0.22            self.offset_ = sp.stats.scoreatpercentile(                self.score_samples(X), 100. * 0.1)         else:             self.offset_ = sp.stats.scoreatpercentile(                self.score_samples(X), 100. * self.contamination)                self.score_samples(X), 100. * self._contamination)          return self


Question: check_input=False, max_squared_sum=max_squared_sum,         sample_weight=sample_weight)     log_reg = LogisticRegression(fit_intercept=fit_intercept)    log_reg = LogisticRegression(multi_class=multi_class)      # The score method of Logistic Regression has a classes_ attribute.     if multi_class == 'ovr':

For probabilistic scorers, LogisticRegressionCV(multiclass='multinomial') uses OvR to calculate scores Description: For scorers such as that use to get probability estimates out of a classifier, the predictions used to generate the scores for do not seem to be the same predictions as those generated by the method of . The former uses a single logistic function and normalises (one-v-rest approach), whereas the latter uses the softmax function (multinomial approach). This appears to be because the instance supplied to the scoring function at line 955 of within the helper function , () , is initialised, () , without a multiclass argument, and so takes the default, which is . It seems like altering L922 to read so that the instance supplied to the scoring function at line 955 inherits the option specified in would be a fix, but I am not a coder and would appreciate some expert insight! Likewise, I do not know whether this issue exists for other classifiers/regressors, as I have only worked with Logistic Regression. Minimal example: Versions: Linux-4.4.0-72-generic-x8664-with-Ubuntu-14.04-trusty Python 2.7.6 NumPy 1.12.0 SciPy 0.18.1 Scikit-learn 0.18.1 [WIP] fixed bug in logregscoringpath <!-- Thanks for contributing a pull request! Please ensure you have taken a look at the contribution guidelines: --<!-- Example: -- In logregscoringpath method, constructor of LogisticRegression accepted only fitintercept as argument, which caused the bug explained in the issue above. As suggested, adding multiclass as argument when creating logistic regression object, solves the problem for multiclass case. After that, it seems like other similar parameters must be passed as arguments to logistic regression constructor. Also, changed interceptscaling default value to float Tested on the code provided in the issue by with various arguments on linearmodel.logistic.logregscoringpath and linearmodel.LogisticRegression, seems ok. Probably needs more testing. <!-- Please be aware that we are a loose team of volunteers so patience is necessary; assistance handling other issues is very welcome. We value all user contributions, no matter how minor they are.


Question: check_input=False, max_squared_sum=max_squared_sum,         sample_weight=sample_weight)     log_reg = LogisticRegression(fit_intercept=fit_intercept)    log_reg = LogisticRegression(multi_class=multi_class)      # The score method of Logistic Regression has a classes_ attribute.     if multi_class == 'ovr':

If we are slow to review, either the pull request needs some benchmarking, tinkering, convincing, etc. or more likely the reviewers are simply busy. In either case, we ask for your understanding during the review process. For more information, see our FAQ on this topic: http://scikit- Thanks for contributing! --


Question: else:     from . import __check_build     from .base import clone<ins>    from .utils._show_versions import show_versions</ins>     __check_build  # avoid flakes unused variable error      __all__ = ['calibration', 'cluster', 'covariance', 'cross_decomposition',

Add sklearn.showversions() similar to pandas.showversions (with numpy blas binding info) Some numeric issues are related to the specific blas that numpy is using. I'm wondering if it makes sense to add the relevant ` invocations to the template to make it easier for people to report.


Question: 'preprocessing', 'random_projection', 'semi_supervised',                'svm', 'tree', 'discriminant_analysis', 'impute', 'compose',                # Non-modules:               'clone', 'get_config', 'set_config', 'config_context']               'clone', 'get_config', 'set_config', 'config_context',               'show_versions']   def setup_module(module):diff --git a/sklearn/utils/_show_versions.py b/sklearn/utils/_show_versions.pynew file mode 100644-- /dev/null++ b/sklearn/utils/_show_versions.py


Question: <ins>"""Utility methods to print system info for debuggingadapted from :func:`pandas.show_versions`"""# License: BSD 3 clauseimport platformimport sysimport importlibdef _get_sys_info():    """System information    Return    ------    sys_info : dict        system and Python version information    """    python = sys.version.replace('\n', ' ')    blob = [        ("python", python),        ('executable', sys.executable),        ("machine", platform.platform()),    ]    return dict(blob)def _get_deps_info():    """Overview of the installed version of main dependencies    Returns    -------    deps_info: dict        version information on relevant Python libraries    """    deps = [        "pip",        "setuptools",        "sklearn",        "numpy",        "scipy",        "Cython",        "pandas",    ]    def get_version(module):        return module.__version__    deps_info = {}    for modname in deps:        try:            if modname in sys.modules:                mod = sys.modules[modname]            else:                mod = importlib.import_module(modname)            ver = get_version(mod)            deps_info[modname] = ver        except ImportError:            deps_info[modname] = None    return deps_infodef _get_blas_info():    """Information on system BLAS    Uses the `scikit-learn` builtin method    :func:`sklearn._build_utils.get_blas_info` which may fail from time to time    Returns    -------    blas_info: dict        system BLAS information    """    from .._build_utils import get_blas_info    cblas_libs, blas_dict = get_blas_info()    macros = ['{key}={val}'.format(key=a, val=b)              for (a, b) in blas_dict.get('define_macros', [])]    blas_blob = [        ('macros', ', '.join(macros)),        ('lib_dirs', ':'.join(blas_dict.get('library_dirs', ''))),        ('cblas_libs', ', '.join(cblas_libs)),    ]    return dict(blas_blob)def show_versions():    "Print useful debugging information"    sys_info = _get_sys_info()    deps_info = _get_deps_info()    blas_info = _get_blas_info()    print('\nSystem')    print('------')    for k, stat in sys_info.items():        print("{k:>10}: {stat}".format(k=k, stat=stat))    print('\nBLAS')    print('----')    for k, stat in blas_info.items():        print("{k:>10}: {stat}".format(k=k, stat=stat))    print('\nPython deps')    print('-----------')    for k, stat in deps_info.items():        print("{k:>10}: {stat}".format(k=k, stat=stat))</ins>


Question: The metric to use when calculating distance between instances in a         feature array. If metric is a string, it must be one of the options         allowed by :func:`sklearn.metrics.pairwise.pairwise_distances`. If X is        the distance array itself, use "precomputed" as the metric.        the distance array itself, use "precomputed" as the metric. Precomputed        distance matrices must have 0 along the diagonal.      `**kwds` : optional keyword parameters         Any further parameters are passed directly to the distance function.

silhouettesamples gives incorrect result from precomputed distance matrix with diagonal entries silhouettesamples gives incorrect result from precomputed distance matrix with diagonal entries. When using silhouettesamples and metric='precomputed', if the input distance matrix has non-zero values along the diagonal then the silhouette scores are incorrect. Suggested Solution Before calculating the scores the diagonal entries of a precomputed distance matrix should be set to zero. Example: [0., 0.952, 0., 0., 0., 0.] [0., 0.552, 0., 0., 0., 0.] Darwin-17.7.0-x8664-i386-64bit Python 3.6.4 (default, Jan 16 2018, 12:04:33) [GCC 4.2.1 Compatible Clang 4.0.1 (tags/RELEASE_401/final)] NumPy 1.15.1 SciPy 1.1.0 Scikit-Learn 0.20.0


Question: """     X, labels = check_X_y(X, labels, accept_sparse=['csc', 'csr'])<ins>    # Check for diagonal entries in precomputed distance matrix    if metric == 'precomputed':        if np.any(np.diagonal(X)):            raise ValueError(                'The precomputed distance matrix contains non-zero '                'elements on the diagonal. Use np.fill_diagonal(X, 0).'            )</ins>     le = LabelEncoder()     labels = le.fit_transform(labels)     n_samples = len(labels)


Question: This implementation deviates from the original OPTICS by first performing     k-nearest-neighborhood searches on all points to identify core sizes, then     computing only the distances to unprocessed points when constructing the    cluster order. It also does not employ a heap to manage the expansion    candiates, but rather uses numpy masked arrays. This can be potentially    slower with some parameters (at the benefit from using fast numpy code).    cluster order. Note that we do not employ a heap to manage the expansion    candidates, so the time complexity will be O(n^2).      Read more in the :ref:`User Guide <optics>`.

OPTICS: self.coredistances inconsistent with documentation&R implementation In the doc, we state that `, but it's not the case. Result from scikit-learn: Result from R:


Question: This implementation deviates from the original OPTICS by first performing     k-nearest-neighborhood searches on all points to identify core sizes, then     computing only the distances to unprocessed points when constructing the    cluster order.    cluster order. Note that we do not employ a heap to manage the expansion    candidates, so the time complexity will be O(n^2).      Read more in the :ref:`User Guide <optics>`.


Question: n_jobs=self.n_jobs)          nbrs.fit(X)        # Here we first do a kNN query for each point, this differs from        # the original OPTICS that only used epsilon range queries.         self.core_distances_ = self._compute_core_distances_(X, nbrs)        # OPTICS puts an upper limit on these, use inf for undefined.        self.core_distances_[self.core_distances_ > self.max_eps] = np.inf         self.ordering_ = self._calculate_optics_order(X, nbrs)          indices_, self.labels_ = _extract_optics(self.ordering_,


Question: return self      # OPTICS helper functions<del></del>     def _compute_core_distances_(self, X, neighbors, working_memory=None):         """Compute the k-th nearest neighbor of each sample


Question: def _calculate_optics_order(self, X, nbrs):         # Main OPTICS loop. Not parallelizable. The order that entries are         # written to the 'ordering_' list is important!        # Note that this implementation is O(n^2) theoretically, but        # supposedly with very low constant factors.         processed = np.zeros(X.shape[0], dtype=bool)         ordering = np.zeros(X.shape[0], dtype=int)        ordering_idx = 0        for point in range(X.shape[0]):            if processed[point]:                continue            if self.core_distances_[point] <= self.max_eps:                while not processed[point]:                    processed[point] = True                    ordering[ordering_idx] = point                    ordering_idx += 1                    point = self._set_reach_dist(point, processed, X, nbrs)            else:  # For very noisy points                ordering[ordering_idx] = point                ordering_idx += 1                processed[point] = True        for ordering_idx in range(X.shape[0]):            # Choose next based on smallest reachability distance            # (And prefer smaller ids on ties, possibly np.inf!)            index = np.where(processed == 0)[0]            point = index[np.argmin(self.reachability_[index])]            processed[point] = True            ordering[ordering_idx] = point            if self.core_distances_[point] != np.inf:                self._set_reach_dist(point, processed, X, nbrs)         return ordering      def _set_reach_dist(self, point_index, processed, X, nbrs):         P = X[point_index:point_index + 1]        # Assume that radius_neighbors is faster without distances        # and we don't need all distances, nevertheless, this means        # we may be doing some work twice.         indices = nbrs.radius_neighbors(P, radius=self.max_eps,                                         return_distance=False)[0]          # Getting indices of neighbors that have not been processed         unproc = np.compress((~np.take(processed, indices)).ravel(),                              indices, axis=0)        # Keep n_jobs = 1 in the following lines...please        # Neighbors of current point are already processed.         if not unproc.size:            # Everything is already processed. Return to main loop            return point_index            return         # Only compute distances to unprocessed neighbors:         if self.metric == 'precomputed':             dists = X[point_index, unproc]         else:


Question: self.reachability_[unproc[improved]] = rdists[improved]         self.predecessor_[unproc[improved]] = point_index <del>        # Choose next based on smallest reachability distance        # (And prefer smaller ids on ties).        # All unprocessed points qualify, not just new neighbors ("unproc")        return (np.ma.array(self.reachability_, mask=processed)                .argmin(fill_value=np.inf))</del>     def extract_dbscan(self, eps):         """Performs DBSCAN extraction for an arbitrary epsilon.


Question: from .base import _transform_selected from .label import _encode, _encode_check_unknown  range = six.moves.range  __all__ = [     'OneHotEncoder',     'OrdinalEncoder'

OneHotEncoder throws unhelpful error messages when tranform called prior to fit <!-- If your issue is a usage question, submit it here instead: StackOverflow with the scikit-learn tag: Mailing List: For more information, see User Questions: http://scikit- --<!-- Instructions For Filing a Bug: --<!-- Example: Joblib Error thrown when calling fit on LatentDirichletAllocation with evaluateevery 0--OneHotEncoder throws an instead of a when tranform is called prior to fit if is called prior to being fit an is thrown if includes arrays of of unicode type <!-- Example: If the code is too long, feel free to put it in a public gist and link it in the issue: --<!-- Example: No error is thrown. Please paste or describe the expected results.--<!-- Please paste or specifically describe the actual output or traceback. --<!-- Please run the following snippet and paste the output below. For scikit-learn = 0.20: import sklearn; sklearn.showversions() For scikit-learn < 0.20: import platform; print(platform.platform()) import sys; print("Python", sys.version) import numpy; print("NumPy", numpy.version) import scipy; print("SciPy", scipy.version) import sklearn; print("Scikit-Learn", sklearn.version) --<!-- Thanks for contributing! --


Question: "The 'categorical_features' keyword is deprecated in "                     "version 0.20 and will be removed in 0.22. You can "                     "use the ColumnTransformer instead.", DeprecationWarning)<ins>                # Set categories_ to empty list if no categorical columns exist                n_features = X.shape[1]                sel = np.zeros(n_features, dtype=bool)                sel[np.asarray(self.categorical_features)] = True                if sum(sel) == 0:                    self.categories_ = []</ins>                 self._legacy_mode = True             self._categorical_features = self.categorical_features         else:


Question: X_out : sparse matrix if sparse=True else a 2-d array             Transformed input.         """<ins>        check_is_fitted(self, 'categories_')</ins>         if self._legacy_mode:             return _transform_selected(X, self._legacy_transform, self.dtype,                                        self._categorical_features,


Question: cats = self.categories_         if input_features is None:             input_features = ['x%d' % i for i in range(len(cats))]        elif(len(input_features) != len(self.categories_)):        elif len(input_features) != len(self.categories_):             raise ValueError(                 "input_features should have length equal to number of "                 "features ({}), got {}".format(len(self.categories_),


Question: if len(x.shape) == 0:             raise TypeError("Singleton array %r cannot be considered"                             " a valid collection." % x)        return x.shape[0]        # Check that shape is returning an integer or default to len        # Dask dataframes may not return numeric shape[0] value        if isinstance(x.shape[0], numbers.Integral):            return x.shape[0]        else:            return len(x)     else:         return len(x)

SkLearn method generating error with Dask DataFrames When using Dask Dataframes with SkLearn, I used to be able to just ask SkLearn for the score of any given algorithm. It would spit out a nice answer and I'd move on. After updating to the newest versions, all metrics that compute based on (ytrue, ypredicted) are failing. I've tested , , , and Work-around shown below, but it's not ideal because it requires me to cast from Dask Arrays to numpy arrays which won't work if the data is huge. I've asked Dask about it here: and they've said it's an issue with the SkLearn check, and that they won't be addressing it. It seems like it should be not super complicated to add a that says "if shape doesn't return a tuple revert to pretending shape didn't exist". If others think that sounds correct, I can attempt a pull-request, but I don't want to attempt to solve it on my own only to find out others don't deem that an acceptable solutions. Trace, MWE, versions, and workaround all in-line. MWE: Output of error: Problem occurs after upgrading as follows: Before bug: Update from conda, then bug starts: Work around:


Question: # continue `The rows are marked `X_mask` and will be                     # removed later.                     X_mask[:, i] = valid_mask                    Xi = Xi.copy()                    # cast Xi into the largest string type necessary                    # to handle different lengths of numpy strings                    if (self.categories_[i].dtype.kind in ('U', 'S')                            and self.categories_[i].itemsize > Xi.itemsize):                        Xi = Xi.astype(self.categories_[i].dtype)                    else:                        Xi = Xi.copy()                     Xi[~valid_mask] = self.categories_[i][0]             _, encoded = _encode(Xi, self.categories_[i], encode=True)             X_int[:, i] = encoded

OneHotEncoder ignore unknown error when categories are strings This bug is very specific, but it happens when you set OneHotEncoder to ignore unknown entries. and your labels are strings. The memory of the arrays is not handled safely and it can lead to a ValueError Basically, when you call the transform method it will sets all the unknown strings on your array to OneHotEncoder.categories[i][0] which is the first category alphabetically sorted given for fit If this OneHotEncoder.categories[i][0] is a long string, and the array that you want to transform has small strings, then it is impossible to fit the whole OneHotEncoder.categories[i][0] into the entries of the array we want to transform. So OneHotEncoder.categories[i][0] is truncated and this raise the ValueError. Here we should get an sparse matrix 2x4 false everywhere except at (1,1) the '22' that is known System: python: 2.7.12 (default, Dec 4 2017, 14:50:18) [GCC 5.4.0 ] machine: Linux-4.4.0-138-generic-x8664-with-Ubuntu-16.04-xenial executable: /usr/bin/python BLAS: macros: HAVECBLAS=None cblaslibs: openblas, openblas libdirs: /usr/lib Python deps: Cython: 0.25.2 scipy: 0.18.1 setuptools: 36.7.0 pip: 9.0.1 numpy: 1.15.2 pandas: 0.19.1 sklearn: 0.21.dev0 I already implemented a fix for this issue, where I check the size of the elements in the array before, and I cast them into objects if necessary.


Question: y_type = type_of_target(y)         y_pred = clf.predict_proba(X)         if y_type == "binary":            y_pred = y_pred[:, 1]            if y_pred.shape[1] == 2:                y_pred = y_pred[:, 1]            else:                raise ValueError('got predict_proba of shape {},'                                 ' but need classifier with two'                                 ' classes for {} scoring'.format(                                     y_pred.shape, self._score_func.__name__))         if sample_weight is not None:             return self._sign * self._score_func(y, y_pred,                                                  sample_weight=sample_weight,

ck estimator is classifier & numclasses=2 in <!-- Thanks for contributing a pull request! Please ensure you have taken a look at the contribution guidelines: --<!-- Example: See also . Please use keywords (e.g., Fixes) to create link to the issues or pull requests you resolved, so that they will automatically be closed when your pull request is merged. See --We are fixing this issue: We a test in the file that raises a ValueError if the user is either trying to use a non classifier model for a classification problem, or is using a dataset with only one class. <!-- Please be aware that we are a loose team of volunteers so patience is necessary; assistance handling other issues is very welcome. We value all user contributions, no matter how minor they are. If we are slow to review, either the pull request needs some benchmarking, tinkering, convincing, etc. or more likely the reviewers are simply busy. In either case, we ask for your understanding during the review process. For more information, see our FAQ on this topic: http://scikit- Thanks for contributing! -- BUG: Using GridSearchCV with scoring='rocauc' and GMM as classifier gives IndexError When performing grid search using GridSearchCV using ootb scoring method 'rocauc' and ootb GMM classifier from sklearn.mixture.GMM I get an index error. Code to reproduce: Sorry if the format is incorrect. First time I am posting. ERROR: File "*/python2.7/site-", line 175, in call ypred = y_pred[:, 1] IndexError: index 1 is out of bounds for axis 1 with size 1


Question: y_pred = clf.predict_proba(X)                  if y_type == "binary":                    y_pred = y_pred[:, 1]                    if y_pred.shape[1] == 2:                        y_pred = y_pred[:, 1]                    else:                        raise ValueError('got predict_proba of shape {},'                                         ' but need classifier with two'                                         ' classes for {} scoring'.format(                                             y_pred.shape,                                             self._score_func.__name__))                 elif isinstance(y_pred, list):                     y_pred = np.vstack([p[:, -1] for p in y_pred]).T


Question: a new copy will always be made, even if `copy=False`:          - If X is not an array of floating values;        - If X is encoded as a CSR matrix.        - If X is encoded as a CSR matrix;        - If add_indicator=True.    add_indicator : boolean, optional (default=False)        If True, a `MissingIndicator` transform will stack onto output        of the imputer's transform. This allows a predictive estimator        to account for missingness despite imputation. If a feature has no        missing values at fit/train time, the feature won't appear on        the missing indicator even if there are missing values at        transform/test time.      Attributes     ----------     statistics_ : array of shape (n_features,)         The imputation fill value for each feature.     indicator_ : :class:`sklearn.impute.MissingIndicator`        Indicator used to add binary indicators for missing values.        ``None`` if add_indicator is False.     See also     --------     IterativeImputer : Multivariate imputation of missing values.

addindicator switch in imputers For whatever imputers we have, but especially , we should have an parameter, which simply stacks a transform onto the output of the imputer's .


Question: >>> imp_mean = SimpleImputer(missing_values=np.nan, strategy='mean')     >>> imp_mean.fit([[7, 2, 3], [4, np.nan, 6], [10, 5, 9]])     ... # doctest: +NORMALIZE_WHITESPACE    SimpleImputer(copy=True, fill_value=None, missing_values=nan,           strategy='mean', verbose=0)    SimpleImputer(add_indicator=False, copy=True, fill_value=None,            missing_values=nan, strategy='mean', verbose=0)     >>> X = [[np.nan, 2, 3], [4, np.nan, 6], [10, np.nan, 9]]     >>> print(imp_mean.transform(X))     ... # doctest: +NORMALIZE_WHITESPACE


Question: """     def __init__(self, missing_values=np.nan, strategy="mean",                 fill_value=None, verbose=0, copy=True):                 fill_value=None, verbose=0, copy=True, add_indicator=False):         self.missing_values = missing_values         self.strategy = strategy         self.fill_value = fill_value         self.verbose = verbose         self.copy = copy        self.add_indicator = add_indicator      def _validate_input(self, X):         allowed_strategies = ["mean", "median", "most_frequent", "constant"]


Question: self.missing_values,                                                fill_value) <ins>        if self.add_indicator:            self.indicator_ = MissingIndicator(                missing_values=self.missing_values)            self.indicator_.fit(X)        else:            self.indicator_ = None</ins>         return self      def _sparse_fit(self, X, strategy, missing_values, fill_value):


Question: # for constant strategy, self.statistcs_ is used to store             # fill_value in each column             statistics.fill(fill_value)<del></del>         else:             for i in range(X.shape[1]):                 column = X.data[X.indptr[i]:X.indptr[i + 1]]


Question: raise ValueError("X has %d features per sample, expected %d"                              % (X.shape[1], self.statistics_.shape[0])) <ins>        if self.add_indicator:            X_trans_indicator = self.indicator_.transform(X)</ins>         # Delete the invalid columns if strategy is not constant         if self.strategy == "constant":             valid_statistics = statistics


Question: X[coordinates] = values <ins>        if self.add_indicator:            hstack = sparse.hstack if sparse.issparse(X) else np.hstack            X = hstack((X, X_trans_indicator))</ins>         return X      def _more_tags(self):


Question: # XXX: not handling dictionaries     if estimator_type in (list, tuple, set, frozenset):         return estimator_type([clone(e, safe=safe) for e in estimator])    elif not hasattr(estimator, 'get_params'):    elif not hasattr(estimator, 'get_params') or isinstance(estimator, type):         if not safe:             return copy.deepcopy(estimator)         else:

clone fails for parameters that are estimator types fails when one or more instance parameters are estimator types (i.e. not instances, but classes). I know this is a somewhat unusual use case, but I'm working on a project that provides wrappers for sklearn estimators () and I'd like to store the wrapped estimators as their classes - not their instances - as a parameter inside of a wrapper that behaves like an estimator itself. from sklearn.preprocessing import StandardScaler from import clone clone(StandardScaler(withmean=StandardScaler)) No error. Change , line 51 to: elif not hasattr(estimator, 'getparams') or isinstance(estimator, type): I'm not sure whether this might break stuff in other places, however. I'd happily submit a PR if this change is desired. sklearn: 0.20.0


Question: # check if the object contains several dtypes (typically a pandas     # DataFrame), and store them. If not, store None.     dtypes_orig = None    if hasattr(array, "dtypes") and hasattr(array, "__array__"):    if hasattr(array, "dtypes") and len(array.dtypes):         dtypes_orig = np.array(array.dtypes)      if dtype_numeric:

TypeError: "iteration over a 0-d array" when trying to a pandas.Series <!-- If your issue is a usage question, submit it here instead: StackOverflow with the scikit-learn tag: Mailing List: For more information, see User Questions: http://scikit- --<!-- Instructions For Filing a Bug: --<!-- Example: Joblib Error thrown when calling fit on LatentDirichletAllocation with evaluateevery 0--When trying to call on a instance, an error is thrown with scikit-learn version 0.20.0. Version 0.19.1. works just fine. The states that the input to can be "array-like", and  should fulfill this requirement since it is a "one-dimensional ndarray". <!-- Example: If the code is too long, feel free to put it in a public gist and link it in the issue: --<!-- Example: No error is thrown. Please paste or describe the expected results.--This should be the output (as it is in version 0.19.1): A workaround is replacing with , which also yields this output. <!-- Please paste or specifically describe the actual output or traceback. --<!-- Please run the following snippet and paste the output below. For scikit-learn = 0.20: import sklearn; sklearn.showversions() For scikit-learn < 0.20: import platform; print(platform.platform()) import sys; print("Python", sys.version) import numpy; print("NumPy", numpy.version) import scipy; print("SciPy", scipy.version) import sklearn; print("Scikit-Learn", sklearn.version) --<!-- Thanks for contributing! --


Question: from itertools import chain  import numpy as np<ins>import warnings</ins> from scipy import sparse  from ..base import clone, TransformerMixin

makecolumntransformer has different order of arguments than ColumnTransformer I'm not sure if we discussed this or did this on purpose, but I find this very confusing. `. I guess it's too late to change this? Though we did say it's experimental and subject to change.


Question: return False  def _validate_transformers(transformers):    """Checks if given transformers are valid.    This is a helper function to support the deprecated tuple order.    XXX Remove in v0.22    """    if not transformers:        return True    for t in transformers:        if t in ('drop', 'passthrough'):            continue        if (not (hasattr(t, "fit") or hasattr(t, "fit_transform")) or not                hasattr(t, "transform")):            return False    return Truedef _is_deprecated_tuple_order(tuples):    """Checks if the input follows the deprecated tuple order.    Returns    -------    Returns true if (transformer, columns) is not a valid assumption for the    input, but (columns, transformer) is valid. The latter is deprecated and    its support will stop in v0.22.    XXX Remove in v0.22    """    transformers, columns = zip(*tuples)    if (not _validate_transformers(transformers)            and _validate_transformers(columns)):        return True    return False def _get_transformer_list(estimators):     """     Construct (name, trans, column) tuples from list      """    transformers = [trans[1] for trans in estimators]    columns = [trans[0] for trans in estimators]    names = [trans[0] for trans in _name_estimators(transformers)]    message = ('`make_column_transformer` now expects (transformer, columns) '               'as input tuples instead of (columns, transformer). This '               'has been introduced in v0.20.1. `make_column_transformer` '               'will stop accepting the deprecated (columns, transformer) '               'order in v0.22.')    transformers, columns = zip(*estimators)    # XXX Remove in v0.22    if _is_deprecated_tuple_order(estimators):        transformers, columns = columns, transformers        warnings.warn(message, DeprecationWarning)    names, _ = zip(*_name_estimators(transformers))      transformer_list = list(zip(names, transformers, columns))     return transformer_list


Question: Parameters     ----------    *transformers : tuples of column selections and transformers    *transformers : tuples of transformers and column selections      remainder : {'drop', 'passthrough'} or estimator, default 'drop'         By default, only the specified columns in `transformers` are


Question: >>> from sklearn.preprocessing import StandardScaler, OneHotEncoder     >>> from sklearn.compose import make_column_transformer     >>> make_column_transformer(    ...     (['numerical_column'], StandardScaler()),    ...     (['categorical_column'], OneHotEncoder()))    ...     (StandardScaler(), ['numerical_column']),    ...     (OneHotEncoder(), ['categorical_column']))     ...     # doctest: +NORMALIZE_WHITESPACE +ELLIPSIS     ColumnTransformer(n_jobs=None, remainder='drop', sparse_threshold=0.3,              transformer_weights=None,


Question: y_pred : 1d array-like, or label indicator array / sparse matrix         Predicted labels, as returned by a classifier.     labels : array, shape = [n_labels], optional (default=None)    labels : array, shape = [n_labels], optional (default='deprecated')         Integer array of labels. If not provided, labels will be inferred         from y_true and y_pred.          .. versionadded:: 0.18        .. deprecated:: 0.21           This parameter ``labels`` is deprecated in version 0.21 and will           be removed in version 0.23. Hamming loss uses ``y_true.shape[1]``           for the number of labels when y_true is binary label indicators,           so it is unnecessary for the user to specify.      sample_weight : array-like of shape = [n_samples], optional         Sample weights.

Fix Issue <!-- Thanks for contributing a pull request! Please ensure you have taken a look at the contribution guidelines: --<!-- Example: See also . Please use keywords (e.g., Fixes) to create link to the issues or pull requests you resolved, so that they will automatically be closed when your pull request is merged. See -- Removed the labels parameter in hammingloss function. Removed all instances of labels within the method. Since hammingloss is used only with mulitlabel input, changed len(labels) to [1]. First time contributing to open source! Please let me know what I can do to improve! <!-- Please be aware that we are a loose team of volunteers so patience is necessary; assistance handling other issues is very welcome. We value all user contributions, no matter how minor they are. If we are slow to review, either the pull request needs some benchmarking, tinkering, convincing, etc. or more likely the reviewers are simply busy. In either case, we ask for your understanding during the review process. For more information, see our FAQ on this topic: http://scikit- Thanks for contributing! --[MRG] Handling parameter labels removal from hammingloss <!-- Thanks for contributing a pull request! Please ensure you have taken a look at the contribution guidelines: --<!-- Fix Issue Please use keywords (e.g., Fixes) to create link to the issues or pull requests you resolved, so that they will automatically be closed when your pull request is merged. See --<!-- Please be aware that we are a loose team of volunteers so patience is necessary; assistance handling other issues is very welcome. We value all user contributions, no matter how minor they are. If we are slow to review, either the pull request needs some benchmarking, tinkering, convincing, etc. or more likely the reviewers are simply busy. In either case, we ask for your understanding during the review process. For more information, see our FAQ on this topic: http://scikit- Thanks for contributing! --


Question: y_type, y_true, y_pred = _check_targets(y_true, y_pred)     check_consistent_length(y_true, y_pred, sample_weight)     if labels is None:        labels = unique_labels(y_true, y_pred)    else:        labels = np.asarray(labels)    if labels is not None:        warnings.warn("The labels parameter is unused. It was"                      " deprecated in version 0.21 and"                      " will be removed in version 0.23",                      DeprecationWarning)      if sample_weight is None:         weight_average = 1.


Question: n_differences = count_nonzero(y_true - y_pred,                                       sample_weight=sample_weight)         return (n_differences /                (y_true.shape[0] * len(labels) * weight_average))                (y_true.shape[0] * y_true.shape[1] * weight_average))      elif y_type in ["binary", "multiclass"]:         return _weighted_sum(y_true != y_pred, sample_weight, normalize=True)


Question: return True      for t in transformers:        if t in ('drop', 'passthrough'):        if isinstance(t, six.string_types) and t in ('drop', 'passthrough'):             continue         if (not (hasattr(t, "fit") or hasattr(t, "fit_transform")) or not                 hasattr(t, "transform")):

regression in ColumnTransformer in in 0.20.1 with columns=pd.Index This came up in one of my teaching notebooks actually (and might be in my book as well). This is very natural because columns are of type pd.Index, and so if you take some subset of columns from ` you'll now run into this error. So... 0.20.2?


Question: if test_size is not None:         if np.asarray(test_size).dtype.kind == 'f':            if test_size >= 1.:            if test_size >= 1. or test_size <= 0:                 raise ValueError(                    'test_size=%f should be smaller '                    'than 1.0 or be an integer' % test_size)                    'test_size=%f should be in the (0, 1) range '                    'or be an integer' % test_size)         elif np.asarray(test_size).dtype.kind != 'i':             # int values are checked during split based on the input             raise ValueError("Invalid value for test_size: %r" % test_size)      if train_size is not None:         if np.asarray(train_size).dtype.kind == 'f':            if train_size >= 1.:                raise ValueError("train_size=%f should be smaller "                                 "than 1.0 or be an integer" % train_size)            if train_size >= 1. or train_size <= 0:                raise ValueError('train_size=%f should be in the (0, 1) range '                                 'or be an integer' % train_size)             elif (np.asarray(test_size).dtype.kind == 'f' and                    (train_size + test_size) > 1.):                    (                        (train_size + test_size) > 1. or                        (train_size + test_size) < 0)):                 raise ValueError('The sum of test_size and train_size = %f, '                                 'should be smaller than 1.0. Reduce '                                 'should be in the (0, 1) range. Reduce '                                  'test_size and/or train_size.' %                                  (train_size + test_size))         elif np.asarray(train_size).dtype.kind != 'i':

traintestsplit excepting negative integers and floats The following minimal example doesn't fail: Is it a bug or indented feature? According to the docs neither of the above should make sense. See for reference for example.


Question: size of the data (n_samples)     """     if (test_size is not None and            np.asarray(test_size).dtype.kind == 'i' and            test_size >= n_samples):        raise ValueError('test_size=%d should be smaller than the number of '                         'samples %d' % (test_size, n_samples))            (np.asarray(test_size).dtype.kind == 'i' and                (test_size >= n_samples or test_size <= 0)) or            (np.asarray(test_size).dtype.kind == 'f' and                (test_size <= 0 or test_size >= 1))):        raise ValueError('test_size=%d should be either positive and smaller '                         'than the number of samples %d or a float in the '                         '(0,1) range' % (test_size, n_samples))      if (train_size is not None and            np.asarray(train_size).dtype.kind == 'i' and            train_size >= n_samples):        raise ValueError("train_size=%d should be smaller than the number of"                         " samples %d" % (train_size, n_samples))            (np.asarray(train_size).dtype.kind == 'i' and                (train_size >= n_samples or train_size <= 0)) or            (np.asarray(train_size).dtype.kind == 'f' and                (train_size <= 0 or train_size >= 1))):        raise ValueError('train_size=%d should be either positive and smaller '                         'than the number of samples %d or a float in the '                         '(0,1) range' % (train_size, n_samples))      if test_size == "default":         test_size = 0.1


Question: ...             }         The reported averages include micro average (averaging the        total true positives, false negatives and false positives), macro        average (averaging the unweighted mean per label), weighted average        (averaging the support-weighted mean per label) and sample average        (only for multilabel classification). See also        :func:`precision_recall_fscore_support` for more details on averages.        The reported averages include macro average (averaging the unweighted        mean per label), weighted average (averaging the support-weighted mean        per label), sample average (only for multilabel classification) and        micro average (averaging the total true positives, false negatives and        false positives) it is only shown for multi-label or multi-class        with a subset of classes because it is accuracy otherwise.        See also:func:`precision_recall_fscore_support` for more details        on averages.          Note that in binary classification, recall of the positive class         is also known as "sensitivity"; recall of the negative class is

Showing micro-average in classification report is confusing This is a follow up on . I don't think it makes sense to include the micro-average for multi-class classification. The three columns will always show the same value, all of which being the same as accuracy. I find that confusing. If you want to show this (I don't see why you'd want to show the same number three times), I would at least make it clear in the report that it's accuracy. IncrementalPCA fails if data size % batch size < ncomponents throws The error occurs because the last batch generated by may be smaller than . I reduced the iris data to 101 instances, so the last batch has only a single data instance, which is less than the number of components. As far as I see, none of the current unit tests run into this. ( could, if the code that raises the exception would compare with - which it should, but doesn't). Skipping the last batch if it is to small, that is, changing to fixes the problem. please confirm that this solution seems OK before I go preparing the PR and tests. No error is thrown.


Question: class 1       0.00      0.00      0.00         1          class 2       1.00      0.67      0.80         3     <BLANKLINE>       micro avg       0.60      0.60      0.60         5        accuracy                           0.60         5        macro avg       0.50      0.56      0.49         5     weighted avg       0.70      0.60      0.61         5     <BLANKLINE>    >>> y_pred = [1, 1, 0]    >>> y_true = [1, 1, 1]    >>> print(classification_report(y_true, y_pred, labels=[1, 2, 3]))                  precision    recall  f1-score   support    <BLANKLINE>               1       1.00      0.67      0.80         3               2       0.00      0.00      0.00         0               3       0.00      0.00      0.00         0    <BLANKLINE>       micro avg       1.00      0.67      0.80         3       macro avg       0.33      0.22      0.27         3    weighted avg       1.00      0.67      0.80         3    <BLANKLINE>     """      y_type, y_true, y_pred = _check_targets(y_true, y_pred)


Question: else:         labels = np.asarray(labels) <ins>    # labelled micro average    micro_is_accuracy = ((y_type == 'multiclass' or y_type == 'binary') and                         (not labels_given or                          (set(labels) == set(unique_labels(y_true, y_pred)))))</ins>     if target_names is not None and len(labels) != len(target_names):         if labels_given:             warnings.warn(


Question: # compute all applicable averages     for average in average_options:        line_heading = average + ' avg'        if average.startswith('micro') and micro_is_accuracy:            line_heading = 'accuracy'        else:            line_heading = average + ' avg'         # compute averages with specified averaging method         avg_p, avg_r, avg_f1, _ = precision_recall_fscore_support(             y_true, y_pred, labels=labels,


Question: report_dict[line_heading] = dict(                 zip(headers, [i.item() for i in avg]))         else:            report += row_fmt.format(line_heading, *avg,                                     width=width, digits=digits)            if line_heading == 'accuracy':                row_fmt_accuracy = u'{:>{width}s} ' + \                        u' {:>9.{digits}}' * 2 + u' {:>9.{digits}f}' + \                        u' {:>9}\n'                report += row_fmt_accuracy.format(line_heading, '', '',                                                  *avg[2:], width=width,                                                  digits=digits)            else:                report += row_fmt.format(line_heading, *avg,                                         width=width, digits=digits)      if output_dict:        if 'accuracy' in report_dict.keys():            report_dict['accuracy'] = report_dict['accuracy']['precision']         return report_dict     else:         return report


Question: def davies_bouldin_score(X, labels):     """Computes the Davies-Bouldin score.     The score is defined as the ratio of within-cluster distances to    between-cluster distances.    The score is defined as the average similarity measure of each cluster with    its most similar cluster, where similarity is the ratio of within-cluster    distances to between-cluster distances. Thus, clusters which are farther    apart and less dispersed will result in a better score.    The minimum score is zero, with lower values indicating better clustering.      Read more in the :ref:`User Guide <davies-bouldin_index>`.

Davies Bouldin measure: division by zero I'm facing a problem with the davies bouldin measure. This is the warning that I get: .local/lib/python3.7/site- RuntimeWarning: divide by zero encountered in truedivide score = (intradists[:, None] + intradists) / centroiddistances This is the implementation in sklearn: I found on stack overflow: With this implementation I don't get any warnings, but the results differ: Davies Bouldin measure: division by zero I'm facing a problem with the davies bouldin measure. This is the warning that I get: .local/lib/python3.7/site- RuntimeWarning: divide by zero encountered in truedivide score = (intradists[:, None] + intradists) / centroiddistances This is the implementation in sklearn: I found on stack overflow: With this implementation I don't get any warnings, but the results differ:


Question: if np.allclose(intra_dists, 0) or np.allclose(centroid_distances, 0):         return 0.0     score = (intra_dists[:, None] + intra_dists) / centroid_distances    score[score == np.inf] = np.nan    return np.mean(np.nanmax(score, axis=1))    centroid_distances[centroid_distances == 0] = np.inf    combined_intra_dists = intra_dists[:, None] + intra_dists    scores = np.max(combined_intra_dists / centroid_distances, axis=1)    return np.mean(scores)


Question: to spread out the most frequent values. It also reduces the impact of     (marginal) outliers: this is therefore a robust preprocessing scheme.     The transformation is applied on each feature independently.    The cumulative distribution function of a feature is used to project the    original values. Features values of new/unseen data that fall below    or above the fitted range will be mapped to the bounds of the output    The transformation is applied on each feature independently. First an    estimate of the cumulative distribution function of a feature is    used to map the original values to a uniform distribution. The obtained    values are then mapped to the desired output distribution using the    associated quantile function. Features values of new/unseen data that fall    below or above the fitted range will be mapped to the bounds of the output     distribution. Note that this transform is non-linear. It may distort linear     correlations between variables measured at the same scale but renders     variables measured at different scales more directly comparable.

DOC: add details to QuantileTransformer documentation I think the documentation of should say how it is implemented. There is even a about it and we could take some elements of the answer. Besides I was thinking that to map to a uniform distribution, the implementation was just computing the empirical cdf of the columns but it does not seem to be the case.


Question: def _transform_col(self, X_col, quantiles, inverse):         """Private function to transform a single feature"""         if self.output_distribution == 'normal':            output_distribution = 'norm'        else:            output_distribution = self.output_distribution        output_distribution = getattr(stats, output_distribution)        output_distribution = self.output_distribution          if not inverse:             lower_bound_x = quantiles[0]


Question: upper_bound_x = 1             lower_bound_y = quantiles[0]             upper_bound_y = quantiles[-1]            #  for inverse transform, match a uniform PDF            #  for inverse transform, match a uniform distribution             with np.errstate(invalid='ignore'):  # hide NaN comparison warnings                X_col = output_distribution.cdf(X_col)                if output_distribution == 'normal':                    X_col = stats.norm.cdf(X_col)                # else output distribution is already a uniform distribution         # find index for lower and higher bounds         with np.errstate(invalid='ignore'):  # hide NaN comparison warnings            lower_bounds_idx = (X_col - BOUNDS_THRESHOLD <                                lower_bound_x)            upper_bounds_idx = (X_col + BOUNDS_THRESHOLD >                                upper_bound_x)            if output_distribution == 'normal':                lower_bounds_idx = (X_col - BOUNDS_THRESHOLD <                                    lower_bound_x)                upper_bounds_idx = (X_col + BOUNDS_THRESHOLD >                                    upper_bound_x)            if output_distribution == 'uniform':                lower_bounds_idx = (X_col == lower_bound_x)                upper_bounds_idx = (X_col == upper_bound_x)          isfinite_mask = ~np.isnan(X_col)         X_col_finite = X_col[isfinite_mask]


Question: X_col[upper_bounds_idx] = upper_bound_y         X_col[lower_bounds_idx] = lower_bound_y        # for forward transform, match the output PDF        # for forward transform, match the output distribution         if not inverse:             with np.errstate(invalid='ignore'):  # hide NaN comparison warnings                X_col = output_distribution.ppf(X_col)            # find the value to clip the data to avoid mapping to            # infinity. Clip such that the inverse transform will be            # consistent            clip_min = output_distribution.ppf(BOUNDS_THRESHOLD -                                               np.spacing(1))            clip_max = output_distribution.ppf(1 - (BOUNDS_THRESHOLD -                                                    np.spacing(1)))            X_col = np.clip(X_col, clip_min, clip_max)                if output_distribution == 'normal':                    X_col = stats.norm.ppf(X_col)                    # find the value to clip the data to avoid mapping to                    # infinity. Clip such that the inverse transform will be                    # consistent                    clip_min = stats.norm.ppf(BOUNDS_THRESHOLD - np.spacing(1))                    clip_max = stats.norm.ppf(1 - (BOUNDS_THRESHOLD -                                                   np.spacing(1)))                    X_col = np.clip(X_col, clip_min, clip_max)                # else output distribution is uniform and the ppf is the                # identity function so we let X_col unchanged          return X_col


Question: raise ValueError('QuantileTransformer only accepts'                                  ' non-negative sparse matrices.')         # check the output PDF        # check the output distribution         if self.output_distribution not in ('normal', 'uniform'):             raise ValueError("'output_distribution' has to be either 'normal'"                              " or 'uniform'. Got '{}' instead.".format(


Question: to spread out the most frequent values. It also reduces the impact of     (marginal) outliers: this is therefore a robust preprocessing scheme.     The transformation is applied on each feature independently.    The cumulative distribution function of a feature is used to project the    original values. Features values of new/unseen data that fall below    or above the fitted range will be mapped to the bounds of the output    The transformation is applied on each feature independently. First an    estimate of the cumulative distribution function of a feature is    used to map the original values to a uniform distribution. The obtained    values are then mapped to the desired output distribution using the    associated quantile function. Features values of new/unseen data that fall    below or above the fitted range will be mapped to the bounds of the output     distribution. Note that this transform is non-linear. It may distort linear     correlations between variables measured at the same scale but renders     variables measured at different scales more directly comparable.


Question: else:             n_samples = proba[0].shape[0]            predictions = np.zeros((n_samples, self.n_outputs_))            # all dtypes should be the same, so just take the first            class_type = self.classes_[0].dtype            predictions = np.empty((n_samples, self.n_outputs_),                                   dtype=class_type)              for k in range(self.n_outputs_):                 predictions[:, k] = self.classes_[k].take(np.argmax(proba[k],

fails for multioutput ensemble models with non-numeric DVs <!-- Example: Joblib Error thrown when calling fit on LatentDirichletAllocation with evaluateevery 0--Multioutput forest models assume that the dependent variables are numeric. Passing string DVs returns the following error: I'm going to take a stab at submitting a fix today, but I wanted to file an issue to document the problem in case I'm not able to finish a fix. I wrote a test based on which currently fails: No error is thrown, can run for all ensemble multioutput models <!-- Example: No error is thrown. Please paste or describe the expected results.--<!-- Please paste or specifically describe the actual output or traceback. --I replicated this error using the current master branch of sklearn (0.21.dev0). <!-- Please run the following snippet and paste the output below. For scikit-learn = 0.20: import sklearn; sklearn.showversions() For scikit-learn < 0.20: import platform; print(platform.platform()) import sys; print("Python", sys.version) import numpy; print("NumPy", numpy.version) import scipy; print("SciPy", scipy.version) import sklearn; print("Scikit-Learn", sklearn.version) --<!-- Thanks for contributing! --


Question: raise ValueError("Logistic Regression supports only solvers in %s, got"                          " %s." % (all_solvers, solver))     all_penalties = ['l1', 'l2', 'elasticnet']    all_penalties = ['l1', 'l2', 'elasticnet', 'none']     if penalty not in all_penalties:         raise ValueError("Logistic Regression supports only penalties in %s,"                          " got %s." % (all_penalties, penalty))     if solver not in ['liblinear', 'saga'] and penalty != 'l2':        raise ValueError("Solver %s supports only l2 penalties, "    if solver not in ['liblinear', 'saga'] and penalty not in ('l2', 'none'):        raise ValueError("Solver %s supports only 'l2' or 'none' penalties, "                          "got %s penalty." % (solver, penalty))     if solver != 'liblinear' and dual:         raise ValueError("Solver %s supports only "

Suggestion: Add support for unpenalized logistic regression provides unpenalized OLS, and , which supports , also supports . But if you want plain old unpenalized logistic regression, you have to fake it by setting in to a large number, or use from instead.


Question: if penalty == 'elasticnet' and solver != 'saga':         raise ValueError("Only 'saga' solver supports elasticnet penalty,"                          " got solver={}.".format(solver))<ins>    if solver == 'liblinear' and penalty == 'none':        raise ValueError(            "penalty='none' is not supported for the liblinear solver"        )</ins>     return solver


Question: 'sag', 'saga' and 'newton-cg' solvers.)      This class implements regularized logistic regression using the    'liblinear' library, 'newton-cg', 'sag', 'saga' and 'lbfgs' solvers. It can    handle both dense and sparse input. Use C-ordered arrays or CSR matrices    containing 64-bit floats for optimal performance; any other input format    will be converted (and copied).    'liblinear' library, 'newton-cg', 'sag', 'saga' and 'lbfgs' solvers. **Note    that regularization is applied by default**. It can handle both dense    and sparse input. Use C-ordered arrays or CSR matrices containing 64-bit    floats for optimal performance; any other input format will be converted    (and copied).      The 'newton-cg', 'sag', and 'lbfgs' solvers support only L2 regularization    with primal formulation. The 'liblinear' solver supports both L1 and L2    regularization, with a dual formulation only for the L2 penalty. The    Elastic-Net regularization is only supported by the 'saga' solver.    with primal formulation, or no regularization. The 'liblinear' solver    supports both L1 and L2 regularization, with a dual formulation only for    the L2 penalty. The Elastic-Net regularization is only supported by the    'saga' solver.      Read more in the :ref:`User Guide <logistic_regression>`.      Parameters     ----------    penalty : str, 'l1', 'l2', or 'elasticnet', optional (default='l2')    penalty : str, 'l1', 'l2', 'elasticnet' or 'none', optional (default='l2')         Used to specify the norm used in the penalization. The 'newton-cg',         'sag' and 'lbfgs' solvers support only l2 penalties. 'elasticnet' is        only supported by the 'saga' solver.        only supported by the 'saga' solver. If 'none' (not supported by the        liblinear solver), no regularization is applied.          .. versionadded:: 0.19            l1 penalty with SAGA solver (allowing 'multinomial' + L1)


Question: - For multiclass problems, only 'newton-cg', 'sag', 'saga' and 'lbfgs'           handle multinomial loss; 'liblinear' is limited to one-versus-rest           schemes.        - 'newton-cg', 'lbfgs' and 'sag' only handle L2 penalty, whereas          'liblinear' and 'saga' handle L1 penalty.        - 'newton-cg', 'lbfgs', 'sag' and 'saga' handle L2 or no penalty        - 'liblinear' and 'saga' also handle L1 penalty        - 'saga' also supports 'elasticnet' penalty        - 'liblinear' does not handle no penalty          Note that 'sag' and 'saga' fast convergence is only guaranteed on         features with approximately the same scale. You can


Question: warnings.warn("l1_ratio parameter is only used when penalty is "                           "'elasticnet'. Got "                           "(penalty={})".format(self.penalty))<ins>        if self.penalty == 'none':            if self.C != 1.0:  # default values                warnings.warn(                    "Setting penalty='none' will ignore the C and l1_ratio "                    "parameters"                )                # Note that check for l1_ratio is done right above            C_ = np.inf            penalty = 'l2'        else:            C_ = self.C            penalty = self.penalty</ins>         if not isinstance(self.max_iter, numbers.Number) or self.max_iter < 0:             raise ValueError("Maximum number of iteration must be positive;"                              " got (max_iter=%r)" % self.max_iter)