Question: sklearn.cluster.AffinityPropagation doesn't support sparse affinity matrix <!-- If your issue is a usage question, submit it here instead: StackOverflow with the scikit-learn tag: Mailing List: For more information, see User Questions: http://scikit- --<!-- Instructions For Filing a Bug: --sklearn.cluster.AffinityPropagation doesn't support sparse affinity matrix. <!-- Example: Joblib Error thrown when calling fit on LatentDirichletAllocation with evaluateevery 0--A similar question is at . It focuses on default affinity. no error raised since it works for dense matrix. Traceback (most recent call last): File "<stdin", line 1, in <moduleFile "D:\Miniconda\lib\site-packages\sklearn\cluster\", line 381, in fit , verbose=self.verbose, returnniter=True) File "D:\Miniconda\lib\site-packages\sklearn\cluster\", line 115, in affinitypropagation preference = (S) File "D:\Miniconda\lib\site-packages\numpy\lib\", line 3336, in median overwriteinput=overwriteinput) File "D:\Miniconda\lib\site-packages\numpy\lib\", line 3250, in ureduce r = func(a, kwargs) File "D:\Miniconda\lib\site-packages\numpy\lib\", line 3395, in median return mean(part[indexer], axis=axis, out=out) File "D:\Miniconda\lib\site-packages\numpy\core\", line 2920, in mean out=out, kwargs) File "D:\Miniconda\lib\site-packages\numpy\core\", line 85, in mean ret = (ret / rcount) ValueError: setting an array element with a sequence. System: python: 3.6.7 (default, Oct 28 2018, 19:44:12) [MSC v.1915 64 bit (AMD64)] executable: D:\Miniconda\ machine: Windows-7-6.1.7601-SP1 BLAS: macros: SCIPYMKLH=None, HAVECBLAS=None libdirs: D:/Miniconda\Library\lib cblaslibs: mkl_rt Python deps: pip: 18.1 setuptools: 40.6.2 sklearn: 0.20.1 numpy: 1.15.4 scipy: 1.1.0 Cython: None pandas: 0.23.4 <!-- Thanks for contributing! --

y : Ignored          """        X = check_array(X, accept_sparse='csr')        if self.affinity == "precomputed":            accept_sparse = False        else:            accept_sparse = 'csr'        X = check_array(X, accept_sparse=accept_sparse)         if self.affinity == "precomputed":             self.affinity_matrix_ = X         elif self.affinity == "euclidean":


Question: AttributeError thrown when calling metrics.pairwisedistances with binary metrics and Y is None thrown when calling with binary metrics if is . No error. Should return a of shape containing the pairwise distances. This worked correctly in sklearn version 0.20.3. I think the problem was introduced in there is now a code path where has its checked without any prior check as to whether is .

maxmem = max(         ((x_density * n_samples_X + y_density * n_samples_Y) * n_features          + (x_density * n_samples_X * y_density * n_samples_Y)) / 10,        10 * 2**17)        10 * 2 ** 17)      # The increase amount of memory in 8-byte blocks is:     # - x_density * batch_size * n_features (copy of chunk of X)


Question: AttributeError thrown when calling metrics.pairwisedistances with binary metrics and Y is None thrown when calling with binary metrics if is . No error. Should return a of shape containing the pairwise distances. This worked correctly in sklearn version 0.20.3. I think the problem was introduced in there is now a code path where has its checked without any prior check as to whether is .

# Hence x² + (xd+yd)kx = M, where x=batch_size, k=n_features, M=maxmem     #                                 xd=x_density and yd=y_density     tmp = (x_density + y_density) * n_features    batch_size = (-tmp + np.sqrt(tmp**2 + 4 * maxmem)) / 2    batch_size = (-tmp + np.sqrt(tmp ** 2 + 4 * maxmem)) / 2     batch_size = max(int(batch_size), 1)      x_batches = gen_batches(X.shape[0], batch_size)


Question: AttributeError thrown when calling metrics.pairwisedistances with binary metrics and Y is None thrown when calling with binary metrics if is . No error. Should return a of shape containing the pairwise distances. This worked correctly in sklearn version 0.20.3. I think the problem was introduced in there is now a code path where has its checked without any prior check as to whether is .

K = safe_sparse_dot(X, Y.T, dense_output=True)     K *= gamma     K += coef0    np.tanh(K, K)   # compute tanh in-place    np.tanh(K, K)  # compute tanh in-place     return K


Question: AttributeError thrown when calling metrics.pairwisedistances with binary metrics and Y is None thrown when calling with binary metrics if is . No error. Should return a of shape containing the pairwise distances. This worked correctly in sklearn version 0.20.3. I think the problem was introduced in there is now a code path where has its checked without any prior check as to whether is .

K = euclidean_distances(X, Y, squared=True)     K *= -gamma    np.exp(K, K)    # exponentiate K in-place    np.exp(K, K)  # exponentiate K in-place     return K


Question: AttributeError thrown when calling metrics.pairwisedistances with binary metrics and Y is None thrown when calling with binary metrics if is . No error. Should return a of shape containing the pairwise distances. This worked correctly in sklearn version 0.20.3. I think the problem was introduced in there is now a code path where has its checked without any prior check as to whether is .

gamma = 1.0 / X.shape[1]      K = -gamma * manhattan_distances(X, Y)    np.exp(K, K)    # exponentiate K in-place    np.exp(K, K)  # exponentiate K in-place     return K


Question: AttributeError thrown when calling metrics.pairwisedistances with binary metrics and Y is None thrown when calling with binary metrics if is . No error. Should return a of shape containing the pairwise distances. This worked correctly in sklearn version 0.20.3. I think the problem was introduced in there is now a code path where has its checked without any prior check as to whether is .

dtype = bool if metric in PAIRWISE_BOOLEAN_FUNCTIONS else None         if dtype == bool and (X.dtype != bool or Y.dtype != bool):        if (dtype == bool and                (X.dtype != bool or (Y is not None and Y.dtype != bool))):             msg = "Data was converted to boolean for metric %s" % metric             warnings.warn(msg, DataConversionWarning)


Question: AttributeError thrown when calling metrics.pairwisedistances with binary metrics and Y is None thrown when calling with binary metrics if is . No error. Should return a of shape containing the pairwise distances. This worked correctly in sklearn version 0.20.3. I think the problem was introduced in there is now a code path where has its checked without any prior check as to whether is .

'yule', ] <del></del> # Helper functions - distance PAIRWISE_KERNEL_FUNCTIONS = {     # If updating this dictionary, update the doc in both distance_metrics()


Question: pairwisedistances returns zeros for metric cosine when executed in parallel returns a list of zeros when calculating with equal to -1 or greater than 2. Using calculates the expected results. Using the metric returns non-zero results, but the values seem to be integers instead of floats. I executed with different values for and . The outputs were as follows:

if Y is None:         Y = X    X, Y, dtype = _return_float_dtype(X, Y)      if effective_n_jobs(n_jobs) == 1:         return func(X, Y, **kwds)      # enforce a threading backend to prevent data communication overhead     fd = delayed(_dist_wrapper)    ret = np.empty((X.shape[0], Y.shape[0]), dtype=X.dtype, order='F')    ret = np.empty((X.shape[0], Y.shape[0]), dtype=dtype, order='F')     Parallel(backend="threading", n_jobs=n_jobs)(         fd(func, ret, s, X, Y[s], **kwds)         for s in gen_even_slices(_num_samples(Y), effective_n_jobs(n_jobs)))     if (X is Y or Y is None) and func is euclidean_distances:        # zeroing diagonal for euclidean norm.        # TODO: do it also for other norms.        np.fill_diagonal(ret, 0)     return ret


Question: Untreated overflow (?) for float32 in euclideandistances new in sklearn 21.1 I am using euclidean distances in a project and after updating, the result is wrong for just one of several datasets. When comparing it to one can see that in version 21.1 it behaves substantially different to 20.3. The matrix is an ndarray with size (100,) with float32. The data are in this zip Can be found when using sklearn 20.3, both behave identical. When using version 21.1 has many 0 entries and some unreasonably high entries Sklearn 21 System: python: 3.6.7 (default, Oct 22 2018, 11:32:17) [GCC 8.2.0] executable: /home/lenz/PycharmProjects/pyrolmm/venvsklearn21/bin/python3 machine: Linux-4.15.0-50-generic-x8664-with-Ubuntu-18.04-bionic BLAS: macros: HAVECBLAS=None, NOATLASINFO=-1 libdirs: /usr/lib/x8664-linux-gnu cblaslibs: cblas Python deps: pip: 9.0.1 setuptools: 39.0.1 sklearn: 0.21.1 numpy: 1.16.3 scipy: 1.3.0 Cython: None pandas: None For sklearn 20.3 the versions are: System: python: 3.6.7 (default, Oct 22 2018, 11:32:17) [GCC 8.2.0] executable: /home/lenz/PycharmProjects/pyrolmm/venvsklearn20/bin/python3 machine: Linux-4.15.0-50-generic-x8664-with-Ubuntu-18.04-bionic BLAS: macros: HAVECBLAS=None, NOATLASINFO=-1 libdirs: /usr/lib/x8664-linux-gnu cblas_libs: cblas Python deps: pip: 9.0.1 setuptools: 39.0.1 sklearn: 0.20.3 numpy: 1.16.3 scipy: 1.3.0 Cython: None pandas: None <!-- Thanks for contributing! --

return distances if squared else np.sqrt(distances, out=distances)  def _euclidean_distances_upcast(X, XX=None, Y=None, YY=None):def _euclidean_distances_upcast(X, XX=None, Y=None, YY=None, batch_size=None):     """Euclidean distances between X and Y      Assumes X and Y have float32 dtype.


Question: Untreated overflow (?) for float32 in euclideandistances new in sklearn 21.1 I am using euclidean distances in a project and after updating, the result is wrong for just one of several datasets. When comparing it to one can see that in version 21.1 it behaves substantially different to 20.3. The matrix is an ndarray with size (100,) with float32. The data are in this zip Can be found when using sklearn 20.3, both behave identical. When using version 21.1 has many 0 entries and some unreasonably high entries Sklearn 21 System: python: 3.6.7 (default, Oct 22 2018, 11:32:17) [GCC 8.2.0] executable: /home/lenz/PycharmProjects/pyrolmm/venvsklearn21/bin/python3 machine: Linux-4.15.0-50-generic-x8664-with-Ubuntu-18.04-bionic BLAS: macros: HAVECBLAS=None, NOATLASINFO=-1 libdirs: /usr/lib/x8664-linux-gnu cblaslibs: cblas Python deps: pip: 9.0.1 setuptools: 39.0.1 sklearn: 0.21.1 numpy: 1.16.3 scipy: 1.3.0 Cython: None pandas: None For sklearn 20.3 the versions are: System: python: 3.6.7 (default, Oct 22 2018, 11:32:17) [GCC 8.2.0] executable: /home/lenz/PycharmProjects/pyrolmm/venvsklearn20/bin/python3 machine: Linux-4.15.0-50-generic-x8664-with-Ubuntu-18.04-bionic BLAS: macros: HAVECBLAS=None, NOATLASINFO=-1 libdirs: /usr/lib/x8664-linux-gnu cblas_libs: cblas Python deps: pip: 9.0.1 setuptools: 39.0.1 sklearn: 0.20.3 numpy: 1.16.3 scipy: 1.3.0 Cython: None pandas: None <!-- Thanks for contributing! --

distances = np.empty((n_samples_X, n_samples_Y), dtype=np.float32)     x_density = X.nnz / np.prod(X.shape) if issparse(X) else 1    y_density = Y.nnz / np.prod(Y.shape) if issparse(Y) else 1    # Allow 10% more memory than X, Y and the distance matrix take (at least    # 10MiB)    maxmem = max(        ((x_density * n_samples_X + y_density * n_samples_Y) * n_features         + (x_density * n_samples_X * y_density * n_samples_Y)) / 10,        10 * 2 ** 17)    # The increase amount of memory in 8-byte blocks is:    # - x_density * batch_size * n_features (copy of chunk of X)    # - y_density * batch_size * n_features (copy of chunk of Y)    # - batch_size * batch_size (chunk of distance matrix)    # Hence x² + (xd+yd)kx = M, where x=batch_size, k=n_features, M=maxmem    #                                 xd=x_density and yd=y_density    tmp = (x_density + y_density) * n_features    batch_size = (-tmp + np.sqrt(tmp ** 2 + 4 * maxmem)) / 2    batch_size = max(int(batch_size), 1)    x_batches = gen_batches(X.shape[0], batch_size)    y_batches = gen_batches(Y.shape[0], batch_size)    if batch_size is None:        x_density = X.nnz / np.prod(X.shape) if issparse(X) else 1        y_density = Y.nnz / np.prod(Y.shape) if issparse(Y) else 1        # Allow 10% more memory than X, Y and the distance matrix take (at        # least 10MiB)        maxmem = max(            ((x_density * n_samples_X + y_density * n_samples_Y) * n_features             + (x_density * n_samples_X * y_density * n_samples_Y)) / 10,            10 * 2 ** 17)        # The increase amount of memory in 8-byte blocks is:        # - x_density * batch_size * n_features (copy of chunk of X)        # - y_density * batch_size * n_features (copy of chunk of Y)        # - batch_size * batch_size (chunk of distance matrix)        # Hence x² + (xd+yd)kx = M, where x=batch_size, k=n_features, M=maxmem        #                                 xd=x_density and yd=y_density        tmp = (x_density + y_density) * n_features        batch_size = (-tmp + np.sqrt(tmp ** 2 + 4 * maxmem)) / 2        batch_size = max(int(batch_size), 1)    x_batches = gen_batches(n_samples_X, batch_size)      for i, x_slice in enumerate(x_batches):         X_chunk = X[x_slice].astype(np.float64)


Question: Untreated overflow (?) for float32 in euclideandistances new in sklearn 21.1 I am using euclidean distances in a project and after updating, the result is wrong for just one of several datasets. When comparing it to one can see that in version 21.1 it behaves substantially different to 20.3. The matrix is an ndarray with size (100,) with float32. The data are in this zip Can be found when using sklearn 20.3, both behave identical. When using version 21.1 has many 0 entries and some unreasonably high entries Sklearn 21 System: python: 3.6.7 (default, Oct 22 2018, 11:32:17) [GCC 8.2.0] executable: /home/lenz/PycharmProjects/pyrolmm/venvsklearn21/bin/python3 machine: Linux-4.15.0-50-generic-x8664-with-Ubuntu-18.04-bionic BLAS: macros: HAVECBLAS=None, NOATLASINFO=-1 libdirs: /usr/lib/x8664-linux-gnu cblaslibs: cblas Python deps: pip: 9.0.1 setuptools: 39.0.1 sklearn: 0.21.1 numpy: 1.16.3 scipy: 1.3.0 Cython: None pandas: None For sklearn 20.3 the versions are: System: python: 3.6.7 (default, Oct 22 2018, 11:32:17) [GCC 8.2.0] executable: /home/lenz/PycharmProjects/pyrolmm/venvsklearn20/bin/python3 machine: Linux-4.15.0-50-generic-x8664-with-Ubuntu-18.04-bionic BLAS: macros: HAVECBLAS=None, NOATLASINFO=-1 libdirs: /usr/lib/x8664-linux-gnu cblas_libs: cblas Python deps: pip: 9.0.1 setuptools: 39.0.1 sklearn: 0.20.3 numpy: 1.16.3 scipy: 1.3.0 Cython: None pandas: None <!-- Thanks for contributing! --

else:             XX_chunk = XX[x_slice] <ins>        y_batches = gen_batches(n_samples_Y, batch_size)</ins>         for j, y_slice in enumerate(y_batches):             if X is Y and j < i:                 # when X is Y the distance matrix is symmetric so we only need


Question: GBDTs should bin train and validation data separately? In the new GBDTs we bin the data before calling (for early-stopping). That means that the validation set is also used to find the bin thresholds (it is of course not used to find the split points!). I feel like the "data leak" is very minimal, but it seems more correct to bin Xtrain and Xval separately. WDYT?

Returns         -------         X_binned : array-like, shape (n_samples, n_features)            The binned data.            The binned data (fortran-aligned).         """         X = check_array(X, dtype=[X_DTYPE])         check_is_fitted(self, ['bin_thresholds_', 'actual_n_bins_'])diff --git a/sklearn/ensemble/_hist_gradient_boosting/gradient_boosting.py b/sklearn/ensemble/_hist_gradient_boosting/gradient_boosting.py-- a/sklearn/ensemble/_hist_gradient_boosting/gradient_boosting.py++ b/sklearn/ensemble/_hist_gradient_boosting/gradient_boosting.py


Question: GBDTs should bin train and validation data separately? In the new GBDTs we bin the data before calling (for early-stopping). That means that the validation set is also used to find the bin thresholds (it is of course not used to find the split points!). I feel like the "data leak" is very minimal, but it seems more correct to bin Xtrain and Xval separately. WDYT?

# data.         self._in_fit = True <del>        # bin the data        if self.verbose:            print("Binning {:.3f} GB of data: ".format(X.nbytes / 1e9), end="",                  flush=True)        tic = time()        self.bin_mapper_ = _BinMapper(max_bins=self.max_bins, random_state=rng)        X_binned = self.bin_mapper_.fit_transform(X)        toc = time()        if self.verbose:            duration = toc - tic            print("{:.3f} s".format(duration))</del>          self.loss_ = self._get_loss()


Question: GBDTs should bin train and validation data separately? In the new GBDTs we bin the data before calling (for early-stopping). That means that the validation set is also used to find the bin thresholds (it is of course not used to find the split points!). I feel like the "data leak" is very minimal, but it seems more correct to bin Xtrain and Xval separately. WDYT?

# stratify for classification             stratify = y if hasattr(self.loss_, 'predict_proba') else None             X_binned_train, X_binned_val, y_train, y_val = train_test_split(                X_binned, y, test_size=self.validation_fraction,                stratify=stratify, random_state=rng)            X_train, X_val, y_train, y_val = train_test_split(                X, y, test_size=self.validation_fraction, stratify=stratify,                random_state=rng)        else:            X_train, y_train = X, y            X_val, y_val = None, None             # Predicting is faster of C-contiguous arrays, training is faster            # on Fortran arrays.            X_binned_val = np.ascontiguousarray(X_binned_val)            X_binned_train = np.asfortranarray(X_binned_train)        # Bin the data        self.bin_mapper_ = _BinMapper(max_bins=self.max_bins, random_state=rng)        X_binned_train = self._bin_data(X_train, rng, is_training_data=True)        if X_val is not None:            X_binned_val = self._bin_data(X_val, rng, is_training_data=False)         else:            X_binned_train, y_train = X_binned, y            X_binned_val, y_val = None, None            X_binned_val = None          if self.verbose:             print("Fitting gradient boosted rounds:")


Question: GBDTs should bin train and validation data separately? In the new GBDTs we bin the data before calling (for early-stopping). That means that the validation set is also used to find the bin thresholds (it is of course not used to find the split points!). I feel like the "data leak" is very minimal, but it seems more correct to bin Xtrain and Xval separately. WDYT?

for score in recent_scores]         return not any(recent_improvements) <ins>    def _bin_data(self, X, rng, is_training_data):        """Bin data X.        If is_training_data, then set the bin_mapper_ attribute.        Else, the binned data is converted to a C-contiguous array.        """        description = 'training' if is_training_data else 'validation'        if self.verbose:            print("Binning {:.3f} GB of {} data: ".format(                X.nbytes / 1e9, description), end="", flush=True)        tic = time()        if is_training_data:            X_binned = self.bin_mapper_.fit_transform(X)  # F-aligned array        else:            X_binned = self.bin_mapper_.transform(X)  # F-aligned array            # We convert the array to C-contiguous since predicting is faster            # with this layout (training is faster on F-arrays though)            X_binned = np.ascontiguousarray(X_binned)        toc = time()        if self.verbose:            duration = toc - tic            print("{:.3f} s".format(duration))        return X_binned</ins>     def _print_iteration_stats(self, iteration_start_time):         """Print info about the current fitting iteration."""         log_msg = ''


Question: IncrementalPCA should accept sparse input <!-- If your issue is a usage question, submit it here instead: StackOverflow with the scikit-learn tag: Mailing List: For more information, see User Questions: http://scikit- --<!-- Instructions For Filing a Bug: --is by design suited to application to sparse data in a way that most PCA classes are not. However, it is not written to accept this by default. No error should be thrown. I'd be happy to submit this as a PR if it's desirable. <details</details<!-- Thanks for contributing! --

# License: BSD 3 clause  import numpy as npfrom scipy import linalgfrom scipy import linalg, sparse  from .base import _BasePCA from ..utils import check_array, gen_batches


Question: IncrementalPCA should accept sparse input <!-- If your issue is a usage question, submit it here instead: StackOverflow with the scikit-learn tag: Mailing List: For more information, see User Questions: http://scikit- --<!-- Instructions For Filing a Bug: --is by design suited to application to sparse data in a way that most PCA classes are not. However, it is not written to accept this by default. No error should be thrown. I'd be happy to submit this as a PR if it's desirable. <details</details<!-- Thanks for contributing! --

but not scaled for each feature before applying the SVD.      Depending on the size of the input data, this algorithm can be much more    memory efficient than a PCA.    memory efficient than a PCA, and allows sparse input.      This algorithm has constant memory complexity, on the order    of ``batch_size``, enabling use of np.memmap files without loading the    entire file into memory.    of ``batch_size * n_features``, enabling use of np.memmap files without    loading the entire file into memory. For sparse matrices, the input    is converted to dense in batches (in order to be able to subtract the    mean) which avoids storing the entire dense matrix at any one time.      The computational overhead of each SVD is     ``O(batch_size * n_features ** 2)``, but only 2 * batch_size samples


Question: IncrementalPCA should accept sparse input <!-- If your issue is a usage question, submit it here instead: StackOverflow with the scikit-learn tag: Mailing List: For more information, see User Questions: http://scikit- --<!-- Instructions For Filing a Bug: --is by design suited to application to sparse data in a way that most PCA classes are not. However, it is not written to accept this by default. No error should be thrown. I'd be happy to submit this as a PR if it's desirable. <details</details<!-- Thanks for contributing! --

--------     >>> from sklearn.datasets import load_digits     >>> from sklearn.decomposition import IncrementalPCA    >>> from scipy import sparse     >>> X, _ = load_digits(return_X_y=True)     >>> transformer = IncrementalPCA(n_components=7, batch_size=200)     >>> # either partially fit on smaller batches of data     >>> transformer.partial_fit(X[:100, :])     IncrementalPCA(batch_size=200, n_components=7)     >>> # or let the fit function itself divide the data into batches    >>> X_transformed = transformer.fit_transform(X)    >>> X_sparse = sparse.csr_matrix(X)    >>> X_transformed = transformer.fit_transform(X_sparse)     >>> X_transformed.shape     (1797, 7)


Question: IncrementalPCA should accept sparse input <!-- If your issue is a usage question, submit it here instead: StackOverflow with the scikit-learn tag: Mailing List: For more information, see User Questions: http://scikit- --<!-- Instructions For Filing a Bug: --is by design suited to application to sparse data in a way that most PCA classes are not. However, it is not written to accept this by default. No error should be thrown. I'd be happy to submit this as a PR if it's desirable. <details</details<!-- Thanks for contributing! --

Parameters         ----------        X : array-like, shape (n_samples, n_features)        X : array-like or sparse matrix, shape (n_samples, n_features)             Training data, where n_samples is the number of samples and             n_features is the number of features.


Question: IncrementalPCA should accept sparse input <!-- If your issue is a usage question, submit it here instead: StackOverflow with the scikit-learn tag: Mailing List: For more information, see User Questions: http://scikit- --<!-- Instructions For Filing a Bug: --is by design suited to application to sparse data in a way that most PCA classes are not. However, it is not written to accept this by default. No error should be thrown. I'd be happy to submit this as a PR if it's desirable. <details</details<!-- Thanks for contributing! --

self.singular_values_ = None         self.noise_variance_ = None         X = check_array(X, copy=self.copy, dtype=[np.float64, np.float32])        X = check_array(X, accept_sparse=['csr', 'csc', 'lil'],                        copy=self.copy, dtype=[np.float64, np.float32])         n_samples, n_features = X.shape          if self.batch_size is None:


Question: IncrementalPCA should accept sparse input <!-- If your issue is a usage question, submit it here instead: StackOverflow with the scikit-learn tag: Mailing List: For more information, see User Questions: http://scikit- --<!-- Instructions For Filing a Bug: --is by design suited to application to sparse data in a way that most PCA classes are not. However, it is not written to accept this by default. No error should be thrown. I'd be happy to submit this as a PR if it's desirable. <details</details<!-- Thanks for contributing! --

for batch in gen_batches(n_samples, self.batch_size_,                                  min_batch_size=self.n_components or 0):            self.partial_fit(X[batch], check_input=False)            X_batch = X[batch]            if sparse.issparse(X_batch):                X_batch = X_batch.toarray()            self.partial_fit(X_batch, check_input=False)          return self


Question: IncrementalPCA should accept sparse input <!-- If your issue is a usage question, submit it here instead: StackOverflow with the scikit-learn tag: Mailing List: For more information, see User Questions: http://scikit- --<!-- Instructions For Filing a Bug: --is by design suited to application to sparse data in a way that most PCA classes are not. However, it is not written to accept this by default. No error should be thrown. I'd be happy to submit this as a PR if it's desirable. <details</details<!-- Thanks for contributing! --

Returns the instance itself.         """         if check_input:<ins>            if sparse.issparse(X):                raise TypeError(                    "IncrementalPCA.partial_fit does not support "                    "sparse input. Either convert data to dense "                    "or use IncrementalPCA.fit to do so in batches.")</ins>             X = check_array(X, copy=self.copy, dtype=[np.float64, np.float32])         n_samples, n_features = X.shape         if not hasattr(self, 'components_'):


Question: IncrementalPCA should accept sparse input <!-- If your issue is a usage question, submit it here instead: StackOverflow with the scikit-learn tag: Mailing List: For more information, see User Questions: http://scikit- --<!-- Instructions For Filing a Bug: --is by design suited to application to sparse data in a way that most PCA classes are not. However, it is not written to accept this by default. No error should be thrown. I'd be happy to submit this as a PR if it's desirable. <details</details<!-- Thanks for contributing! --

np.sqrt((self.n_samples_seen_ * n_samples) /                         n_total_samples) * (self.mean_ - col_batch_mean)             X = np.vstack((self.singular_values_.reshape((-1, 1)) *                          self.components_, X, mean_correction))                           self.components_, X, mean_correction))          U, S, V = linalg.svd(X, full_matrices=False)         U, V = svd_flip(U, V, u_based_decision=False)


Question: IncrementalPCA should accept sparse input <!-- If your issue is a usage question, submit it here instead: StackOverflow with the scikit-learn tag: Mailing List: For more information, see User Questions: http://scikit- --<!-- Instructions For Filing a Bug: --is by design suited to application to sparse data in a way that most PCA classes are not. However, it is not written to accept this by default. No error should be thrown. I'd be happy to submit this as a PR if it's desirable. <details</details<!-- Thanks for contributing! --

else:             self.noise_variance_ = 0.         return self<ins>    def transform(self, X):        """Apply dimensionality reduction to X.        X is projected on the first principal components previously extracted        from a training set, using minibatches of size batch_size if X is        sparse.        Parameters        ----------        X : array-like, shape (n_samples, n_features)            New data, where n_samples is the number of samples            and n_features is the number of features.        Returns        -------        X_new : array-like, shape (n_samples, n_components)        Examples        --------        >>> import numpy as np        >>> from sklearn.decomposition import IncrementalPCA        >>> X = np.array([[-1, -1], [-2, -1], [-3, -2],        ...               [1, 1], [2, 1], [3, 2]])        >>> ipca = IncrementalPCA(n_components=2, batch_size=3)        >>> ipca.fit(X)        IncrementalPCA(batch_size=3, n_components=2)        >>> ipca.transform(X) # doctest: +SKIP        """        if sparse.issparse(X):            n_samples = X.shape[0]            output = []            for batch in gen_batches(n_samples, self.batch_size_,                                     min_batch_size=self.n_components or 0):                output.append(super().transform(X[batch].toarray()))            return np.vstack(output)        else:            return super().transform(X)</ins>


Question: Design of addindicator in SimpleImputer may fail when running cross validation <!-- If your issue is a usage question, submit it here instead: StackOverflow with the scikit-learn tag: Mailing List: For more information, see User Questions: http://scikit- --<!-- Instructions For Filing a Bug: --<!-- Example: Joblib Error thrown when calling fit on LatentDirichletAllocation with evaluateevery 0--The design of depends on missing values exist in the training data. This will break cross validation. <!-- Example: If the code is too long, feel free to put it in a public gist and link it in the issue: --<!-- Example: No error is thrown. Please paste or describe the expected results.--No error is thrown. <!-- Please paste or specifically describe the actual output or traceback. --The original design was adopted because, if the training data has no missing value, there will be a column with all zeros. This type of error will appear when we try to do grid search over the parameter. One way to work around this is to split the data in such a way that missing values are available (for each column that has a missing value) in both the training set and test set. <!-- Thanks for contributing! --

if self.add_indicator:             self.indicator_ = MissingIndicator(                missing_values=self.missing_values)                missing_values=self.missing_values, error_on_new=False)             self.indicator_.fit(X)         else:             self.indicator_ = Nonediff --git a/sklearn/impute/_iterative.py b/sklearn/impute/_iterative.py-- a/sklearn/impute/_iterative.py++ b/sklearn/impute/_iterative.py


Question: Design of addindicator in SimpleImputer may fail when running cross validation <!-- If your issue is a usage question, submit it here instead: StackOverflow with the scikit-learn tag: Mailing List: For more information, see User Questions: http://scikit- --<!-- Instructions For Filing a Bug: --<!-- Example: Joblib Error thrown when calling fit on LatentDirichletAllocation with evaluateevery 0--The design of depends on missing values exist in the training data. This will break cross validation. <!-- Example: If the code is too long, feel free to put it in a public gist and link it in the issue: --<!-- Example: No error is thrown. Please paste or describe the expected results.--No error is thrown. <!-- Please paste or specifically describe the actual output or traceback. --The original design was adopted because, if the training data has no missing value, there will be a column with all zeros. This type of error will appear when we try to do grid search over the parameter. One way to work around this is to split the data in such a way that missing values are available (for each column that has a missing value) in both the training set and test set. <!-- Thanks for contributing! --

if self.add_indicator:             self.indicator_ = MissingIndicator(                missing_values=self.missing_values)                missing_values=self.missing_values, error_on_new=False)             X_trans_indicator = self.indicator_.fit_transform(X)         else:             self.indicator_ = None


Question: Zero division error in HistGradientBoosting NaN This dataset is a bit weird in that it has 100 classes with 16 samples each. The default parameter don't work very well but we should fail more gacefully. cc

from .predictor import TreePredictor from .utils import sum_parallel from .types import PREDICTOR_RECORD_DTYPE<ins>from .types import Y_DTYPEEPS = np.finfo(Y_DTYPE).eps  # to avoid zero division errors</ins>   class TreeNode:


Question: Zero division error in HistGradientBoosting NaN This dataset is a bit weird in that it has 100 classes with 16 samples each. The default parameter don't work very well but we should fail more gacefully. cc

https://arxiv.org/abs/1603.02754         """         node.value = -self.shrinkage * node.sum_gradients / (            node.sum_hessians + self.splitter.l2_regularization)            node.sum_hessians + self.splitter.l2_regularization + EPS)         self.finalized_leaves.append(node)      def _finalize_splittable_nodes(self):diff --git a/sklearn/utils/estimator_checks.py b/sklearn/utils/estimator_checks.py-- a/sklearn/utils/estimator_checks.py++ b/sklearn/utils/estimator_checks.py


Question: Zero division error in HistGradientBoosting NaN This dataset is a bit weird in that it has 100 classes with 16 samples each. The default parameter don't work very well but we should fail more gacefully. cc

hasattr(estimator, "predict_proba")):          estimator.fit(X, y)        a = estimator.predict_proba(X_test)[:, 1]        b = estimator.decision_function(X_test)        # Since the link function from decision_function() to predict_proba()        # is sometimes not precise enough (typically expit), we round to the        # 10th decimal to avoid numerical issues.        a = estimator.predict_proba(X_test)[:, 1].round(decimals=10)        b = estimator.decision_function(X_test).round(decimals=10)         assert_array_equal(rankdata(a), rankdata(b))


Question: IndexError: list index out of range in exporttext when the tree only has one feature <!-- If your issue is a usage question, submit it here instead: StackOverflow with the scikit-learn tag: Mailing List: For more information, see User Questions: http://scikit- --<!-- Instructions For Filing a Bug: --returns when there is single feature. <!-- Thanks for contributing! --

value_fmt = "{}{} value: {}\n"      if feature_names:        feature_names_ = [feature_names[i] for i in tree_.feature]        feature_names_ = [feature_names[i] if i != _tree.TREE_UNDEFINED                          else None for i in tree_.feature]     else:         feature_names_ = ["feature_{}".format(i) for i in tree_.feature]


Question: ARD Regressor accuracy degrades when upgrading Scipy 1.2.1 -1.3.0 Hi, bit of a tricky one, I'm hoping someone will have some time and/or suggestions for further investigation! There seems to be an often-occurring worsening of performance (i.e. accuracy, although run-time increases too!) from the ARD regressor when upgrading from Scipy 1.2.1 -1.3.0. On a very simple dataset (see code snippets below) where a near-perfect fit should be achievable, typical error seems to degrade from order 1E-5 to 1E-2. Notably, convergence iterations seem to increase also from ~a few (~5) to around 50-200 iterations. Here's the headline plot, plotting absolute co-efficient error when fit across 1000 datasets generated with different random seeds: ! Note how with Scipy==1.2.1, errors are largely constrained to <0.01, while with Scipy==1.3.0 they range up to 0.05 (and in a few rare cases the algorithm produces garbage results, see later). I guess this could be (probably is?) a Scipy rather than Sklearn issue, but probably the only way to confirm / isolate that would be to start here. It's also possible that this worsening of behaviour is a weirdness of my particular toy example, but the difference in behaviour seems large and unexpected enough to warrant further investigation, I'd hope! OK, so here's a short snippet on just a single seed if you're curious to try this yourself. I'm generating three vectors of normally distributed values, 250 samples. Then the target is just a perfect copy of one of those vectors (index=1). We measure the accuracy of the fit by simply checking how close that coefficient is to 1.0 (the other coefficients always shrink to 0., as you'd hope): Scipy 1.2.1: Scipy 1.3.0 It could be that there's some oddity of the random data from a single seed, so I set up some short scripts to first generate a static collection of 1000 of the datasets as seen above, then collate the results from both versions of scipy. The snippets are as follows: Make data: Test sklearn: Plot results: A little investigating summary statistics of those datasets in notebook gives the following points of comparison:

# This should remained pinned to version 1.2 and not updated like other# externals."""Copyright (c) 2001-2002 Enthought, Inc.  2003-2019, SciPy Developers.All rights reserved.Redistribution and use in source and binary forms, with or withoutmodification, are permitted provided that the following conditionsare met:1. Redistributions of source code must retain the above copyright   notice, this list of conditions and the following disclaimer.2. Redistributions in binary form must reproduce the above   copyright notice, this list of conditions and the following   disclaimer in the documentation and/or other materials provided   with the distribution.3. Neither the name of the copyright holder nor the names of its   contributors may be used to endorse or promote products derived   from this software without specific prior written permission.THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOTLIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FORA PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHTOWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOTLIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANYTHEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USEOF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE."""import numpy as npimport scipy.linalg.decomp as decompdef pinvh(a, cond=None, rcond=None, lower=True, return_rank=False,          check_finite=True):    """    Compute the (Moore-Penrose) pseudo-inverse of a Hermitian matrix.    Copied in from scipy==1.2.2, in order to preserve the default choice of the    `cond` and `above_cutoff` values which determine which values of the matrix    inversion lie below threshold and are so set to zero. Changes in scipy 1.3    resulted in a smaller default threshold and thus slower convergence of    dependent algorithms in some cases (see Sklearn github issue #14055).    Calculate a generalized inverse of a Hermitian or real symmetric matrix    using its eigenvalue decomposition and including all eigenvalues with    'large' absolute value.    Parameters    ----------    a : (N, N) array_like        Real symmetric or complex hermetian matrix to be pseudo-inverted    cond, rcond : float or None        Cutoff for 'small' eigenvalues.        Singular values smaller than rcond * largest_eigenvalue are considered        zero.        If None or -1, suitable machine precision is used.    lower : bool, optional        Whether the pertinent array data is taken from the lower or upper        triangle of a. (Default: lower)    return_rank : bool, optional        if True, return the effective rank of the matrix    check_finite : bool, optional        Whether to check that the input matrix contains only finite numbers.        Disabling may give a performance gain, but may result in problems        (crashes, non-termination) if the inputs do contain infinities or NaNs.    Returns    -------    B : (N, N) ndarray        The pseudo-inverse of matrix `a`.    rank : int        The effective rank of the matrix.  Returned if return_rank == True    Raises    ------    LinAlgError        If eigenvalue does not converge    Examples    --------    >>> from scipy.linalg import pinvh    >>> a = np.random.randn(9, 6)    >>> a = np.dot(a, a.T)    >>> B = pinvh(a)    >>> np.allclose(a, np.dot(a, np.dot(B, a)))    True    >>> np.allclose(B, np.dot(B, np.dot(a, B)))    True    """    a = decomp._asarray_validated(a, check_finite=check_finite)    s, u = decomp.eigh(a, lower=lower, check_finite=False)    if rcond is not None:        cond = rcond    if cond in [None, -1]:        t = u.dtype.char.lower()        factor = {'f': 1E3, 'd': 1E6}        cond = factor[t] * np.finfo(t).eps    # For Hermitian matrices, singular values equal abs(eigenvalues)    above_cutoff = (abs(s) > cond * np.max(abs(s)))    psigma_diag = 1.0 / s[above_cutoff]    u = u[:, above_cutoff]    B = np.dot(u * psigma_diag, np.conjugate(u).T)    if return_rank:        return B, len(psigma_diag)    else:        return Bdiff --git a/sklearn/linear_model/bayes.py b/sklearn/linear_model/bayes.py-- a/sklearn/linear_model/bayes.py++ b/sklearn/linear_model/bayes.py


Question: ARD Regressor accuracy degrades when upgrading Scipy 1.2.1 -1.3.0 Hi, bit of a tricky one, I'm hoping someone will have some time and/or suggestions for further investigation! There seems to be an often-occurring worsening of performance (i.e. accuracy, although run-time increases too!) from the ARD regressor when upgrading from Scipy 1.2.1 -1.3.0. On a very simple dataset (see code snippets below) where a near-perfect fit should be achievable, typical error seems to degrade from order 1E-5 to 1E-2. Notably, convergence iterations seem to increase also from ~a few (~5) to around 50-200 iterations. Here's the headline plot, plotting absolute co-efficient error when fit across 1000 datasets generated with different random seeds: ! Note how with Scipy==1.2.1, errors are largely constrained to <0.01, while with Scipy==1.3.0 they range up to 0.05 (and in a few rare cases the algorithm produces garbage results, see later). I guess this could be (probably is?) a Scipy rather than Sklearn issue, but probably the only way to confirm / isolate that would be to start here. It's also possible that this worsening of behaviour is a weirdness of my particular toy example, but the difference in behaviour seems large and unexpected enough to warrant further investigation, I'd hope! OK, so here's a short snippet on just a single seed if you're curious to try this yourself. I'm generating three vectors of normally distributed values, 250 samples. Then the target is just a perfect copy of one of those vectors (index=1). We measure the accuracy of the fit by simply checking how close that coefficient is to 1.0 (the other coefficients always shrink to 0., as you'd hope): Scipy 1.2.1: Scipy 1.3.0 It could be that there's some oddity of the random data from a single seed, so I set up some short scripts to first generate a static collection of 1000 of the datasets as seen above, then collate the results from both versions of scipy. The snippets are as follows: Make data: Test sklearn: Plot results: A little investigating summary statistics of those datasets in notebook gives the following points of comparison:

from math import log import numpy as np from scipy import linalgfrom scipy.linalg import pinvh  from .base import LinearModel, _rescale_data from ..base import RegressorMixin from ..utils.extmath import fast_logdet from ..utils import check_X_yfrom ..utils.fixes import pinvh   ###############################################################################diff --git a/sklearn/utils/fixes.py b/sklearn/utils/fixes.py-- a/sklearn/utils/fixes.py++ b/sklearn/utils/fixes.py


Question: ARD Regressor accuracy degrades when upgrading Scipy 1.2.1 -1.3.0 Hi, bit of a tricky one, I'm hoping someone will have some time and/or suggestions for further investigation! There seems to be an often-occurring worsening of performance (i.e. accuracy, although run-time increases too!) from the ARD regressor when upgrading from Scipy 1.2.1 -1.3.0. On a very simple dataset (see code snippets below) where a near-perfect fit should be achievable, typical error seems to degrade from order 1E-5 to 1E-2. Notably, convergence iterations seem to increase also from ~a few (~5) to around 50-200 iterations. Here's the headline plot, plotting absolute co-efficient error when fit across 1000 datasets generated with different random seeds: ! Note how with Scipy==1.2.1, errors are largely constrained to <0.01, while with Scipy==1.3.0 they range up to 0.05 (and in a few rare cases the algorithm produces garbage results, see later). I guess this could be (probably is?) a Scipy rather than Sklearn issue, but probably the only way to confirm / isolate that would be to start here. It's also possible that this worsening of behaviour is a weirdness of my particular toy example, but the difference in behaviour seems large and unexpected enough to warrant further investigation, I'd hope! OK, so here's a short snippet on just a single seed if you're curious to try this yourself. I'm generating three vectors of normally distributed values, 250 samples. Then the target is just a perfect copy of one of those vectors (index=1). We measure the accuracy of the fit by simply checking how close that coefficient is to 1.0 (the other coefficients always shrink to 0., as you'd hope): Scipy 1.2.1: Scipy 1.3.0 It could be that there's some oddity of the random data from a single seed, so I set up some short scripts to first generate a static collection of 1000 of the datasets as seen above, then collate the results from both versions of scipy. The snippets are as follows: Make data: Test sklearn: Plot results: A little investigating summary statistics of those datasets in notebook gives the following points of comparison:

# once support for sp_version < (1, 3) is dropped     from ..externals._lobpcg import lobpcg  # noqa <ins>if sp_version >= (1, 3):    # Preserves earlier default choice of pinvh cutoff `cond` value.    # Can be removed once issue #14055 is fully addressed.    from ..externals._scipy_linalg import pinvhelse:    from scipy.linalg import pinvh # noqa</ins> if sp_version >= (0, 19):     def _argmax(arr_or_spmatrix, axis=None):         return arr_or_spmatrix.argmax(axis=axis)


Question: IndexError thrown with LogisticRegressionCV and refit=False The following error is thrown when trying to estimate a regularization parameter via cross-validation, without refitting. No error is thrown.

# Take the best scores across every fold and the average of                 # all coefficients corresponding to the best scores.                 best_indices = np.argmax(scores, axis=1)                if self.multi_class == 'ovr':                if multi_class == 'ovr':                     w = np.mean([coefs_paths[i, best_indices[i], :]                                  for i in range(len(folds))], axis=0)                 else:


Question: IndexError thrown with LogisticRegressionCV and refit=False The following error is thrown when trying to estimate a regularization parameter via cross-validation, without refitting. No error is thrown.

best_indices_C = best_indices % len(self.Cs_)                 self.C_.append(np.mean(self.Cs_[best_indices_C]))                 best_indices_l1 = best_indices // len(self.Cs_)                self.l1_ratio_.append(np.mean(l1_ratios_[best_indices_l1]))                if self.penalty == 'elasticnet':                    best_indices_l1 = best_indices // len(self.Cs_)                    self.l1_ratio_.append(np.mean(l1_ratios_[best_indices_l1]))                else:                    self.l1_ratio_.append(None)              if multi_class == 'multinomial':                 self.C_ = np.tile(self.C_, n_classes)


Question: NCA fails in GridSearch due to too strict parameter checks NCA checks its parameters to have a specific type, which can easily fail in a GridSearch due to how param grid is made. Here is an example: The issue is that for : 1 is not a float, and for : np.int64 is not int Before proposing a fix for this specific situation, I'd like to have your general opinion about parameter checking. I like this idea of common parameter checking tool introduced with the NCA PR. What do you think about extending it across the code-base (or at least for new or recent estimators) ? Currently parameter checking is not always done or often partially done, and is quite redundant. For instance, here is the input validation of lda: most params aren't checked and for those who are there's a lot of duplicated code. A propose to be upgrade the new tool to be able to check open/closed intervals (currently only closed) and list membership. The api would be something like that: where validoptions would be a dict of . e.g for the param of , it can be either a float or a string in a list, which would give Sometimes a parameter can only be positive or within a given interval, e.g. of must be between 0 and 1, which would give positivity of e.g. would be .

import numpy as np import sys import time<ins>import numbers</ins> from scipy.optimize import minimize from ..utils.extmath import softmax from ..metrics import pairwise_distances


Question: NCA fails in GridSearch due to too strict parameter checks NCA checks its parameters to have a specific type, which can easily fail in a GridSearch due to how param grid is made. Here is an example: The issue is that for : 1 is not a float, and for : np.int64 is not int Before proposing a fix for this specific situation, I'd like to have your general opinion about parameter checking. I like this idea of common parameter checking tool introduced with the NCA PR. What do you think about extending it across the code-base (or at least for new or recent estimators) ? Currently parameter checking is not always done or often partially done, and is quite redundant. For instance, here is the input validation of lda: most params aren't checked and for those who are there's a lot of duplicated code. A propose to be upgrade the new tool to be able to check open/closed intervals (currently only closed) and list membership. The api would be something like that: where validoptions would be a dict of . e.g for the param of , it can be either a float or a string in a list, which would give Sometimes a parameter can only be positive or within a given interval, e.g. of must be between 0 and 1, which would give positivity of e.g. would be .

# Check the preferred dimensionality of the projected space         if self.n_components is not None:            check_scalar(self.n_components, 'n_components', int, 1)            check_scalar(                self.n_components, 'n_components', numbers.Integral, 1)              if self.n_components > X.shape[1]:                 raise ValueError('The preferred dimensionality of the '


Question: NCA fails in GridSearch due to too strict parameter checks NCA checks its parameters to have a specific type, which can easily fail in a GridSearch due to how param grid is made. Here is an example: The issue is that for : 1 is not a float, and for : np.int64 is not int Before proposing a fix for this specific situation, I'd like to have your general opinion about parameter checking. I like this idea of common parameter checking tool introduced with the NCA PR. What do you think about extending it across the code-base (or at least for new or recent estimators) ? Currently parameter checking is not always done or often partially done, and is quite redundant. For instance, here is the input validation of lda: most params aren't checked and for those who are there's a lot of duplicated code. A propose to be upgrade the new tool to be able to check open/closed intervals (currently only closed) and list membership. The api would be something like that: where validoptions would be a dict of . e.g for the param of , it can be either a float or a string in a list, which would give Sometimes a parameter can only be positive or within a given interval, e.g. of must be between 0 and 1, which would give positivity of e.g. would be .

.format(X.shape[1],                                          self.components_.shape[1]))         check_scalar(self.max_iter, 'max_iter', int, 1)        check_scalar(self.tol, 'tol', float, 0.)        check_scalar(self.verbose, 'verbose', int, 0)        check_scalar(self.max_iter, 'max_iter', numbers.Integral, 1)        check_scalar(self.tol, 'tol', numbers.Real, 0.)        check_scalar(self.verbose, 'verbose', numbers.Integral, 0)          if self.callback is not None:             if not callable(self.callback):


Question: AdaBoost's "SAMME" algorithm uses 'predict' while fitting and 'predictproba' while predicting probas Subj. This seems to me to be a wrong approach, moreover this drives to such mistakes: <preAdaBoostClassifier(algorithm="SAMME", baseestimator=SVC()).fit(trainX, trainY).predictproba(testX) NotImplementedError Traceback (most recent call last) <ipython-input-108-1d666912dadain <module() ----1 AdaBoostClassifier(algorithm="SAMME", baseestimator=SVC()).fit(trainX, trainY).predictproba(testX) /Library/Python/2.7/site- in predictproba(self, X) 716 proba = sum(estimator.predictproba(X) w 717 for estimator, w in zip(self.estimators, --718 self.estimatorweights)) 719 720 proba /= () /Library/Python/2.7/site- in <genexpr((estimator, w)) 715 else: # self.algorithm == "SAMME" 716 proba = sum(estimator.predictproba(X) w --717 for estimator, w in zip(self.estimators, 718 self.estimatorweights)) 719 /Library/Python/2.7/site- in predictproba(self, X) 493 if not self.probability: 494 raise NotImplementedError( --495 "probability estimates must be enabled to use this method") 496 497 if not in ('csvc', 'nusvc'): NotImplementedError: probability estimates must be enabled to use this method </pre

from ..tree import DecisionTreeClassifier, DecisionTreeRegressor from ..utils import check_array, check_random_state, check_X_y, safe_indexing<ins>from ..utils.extmath import softmax</ins> from ..utils.extmath import stable_cumsum from ..metrics import accuracy_score, r2_score from ..utils.validation import check_is_fitted


Question: AdaBoost's "SAMME" algorithm uses 'predict' while fitting and 'predictproba' while predicting probas Subj. This seems to me to be a wrong approach, moreover this drives to such mistakes: <preAdaBoostClassifier(algorithm="SAMME", baseestimator=SVC()).fit(trainX, trainY).predictproba(testX) NotImplementedError Traceback (most recent call last) <ipython-input-108-1d666912dadain <module() ----1 AdaBoostClassifier(algorithm="SAMME", baseestimator=SVC()).fit(trainX, trainY).predictproba(testX) /Library/Python/2.7/site- in predictproba(self, X) 716 proba = sum(estimator.predictproba(X) w 717 for estimator, w in zip(self.estimators, --718 self.estimatorweights)) 719 720 proba /= () /Library/Python/2.7/site- in <genexpr((estimator, w)) 715 else: # self.algorithm == "SAMME" 716 proba = sum(estimator.predictproba(X) w --717 for estimator, w in zip(self.estimators, 718 self.estimatorweights)) 719 /Library/Python/2.7/site- in predictproba(self, X) 493 if not self.probability: 494 raise NotImplementedError( --495 "probability estimates must be enabled to use this method") 496 497 if not in ('csvc', 'nusvc'): NotImplementedError: probability estimates must be enabled to use this method </pre

else:                 yield pred / norm <ins>    @staticmethod    def _compute_proba_from_decision(decision, n_classes):        """Compute probabilities from the decision function.        This is based eq. (4) of [1] where:            p(y=c|X) = exp((1 / K-1) f_c(X)) / sum_k(exp((1 / K-1) f_k(X)))                     = softmax((1 / K-1) * f(X))        References        ----------        .. [1] J. Zhu, H. Zou, S. Rosset, T. Hastie, "Multi-class AdaBoost",               2009.        """        if n_classes == 2:            decision = np.vstack([-decision, decision]).T / 2        else:            decision /= (n_classes - 1)        return softmax(decision, copy=False)</ins>     def predict_proba(self, X):         """Predict class probabilities for X.


Question: AdaBoost's "SAMME" algorithm uses 'predict' while fitting and 'predictproba' while predicting probas Subj. This seems to me to be a wrong approach, moreover this drives to such mistakes: <preAdaBoostClassifier(algorithm="SAMME", baseestimator=SVC()).fit(trainX, trainY).predictproba(testX) NotImplementedError Traceback (most recent call last) <ipython-input-108-1d666912dadain <module() ----1 AdaBoostClassifier(algorithm="SAMME", baseestimator=SVC()).fit(trainX, trainY).predictproba(testX) /Library/Python/2.7/site- in predictproba(self, X) 716 proba = sum(estimator.predictproba(X) w 717 for estimator, w in zip(self.estimators, --718 self.estimatorweights)) 719 720 proba /= () /Library/Python/2.7/site- in <genexpr((estimator, w)) 715 else: # self.algorithm == "SAMME" 716 proba = sum(estimator.predictproba(X) w --717 for estimator, w in zip(self.estimators, 718 self.estimatorweights)) 719 /Library/Python/2.7/site- in predictproba(self, X) 493 if not self.probability: 494 raise NotImplementedError( --495 "probability estimates must be enabled to use this method") 496 497 if not in ('csvc', 'nusvc'): NotImplementedError: probability estimates must be enabled to use this method </pre

if n_classes == 1:             return np.ones((_num_samples(X), 1))         if self.algorithm == 'SAMME.R':            # The weights are all 1. for SAMME.R            proba = sum(_samme_proba(estimator, n_classes, X)                        for estimator in self.estimators_)        else:  # self.algorithm == "SAMME"            proba = sum(estimator.predict_proba(X) * w                        for estimator, w in zip(self.estimators_,                                                self.estimator_weights_))        proba /= self.estimator_weights_.sum()        proba = np.exp((1. / (n_classes - 1)) * proba)        normalizer = proba.sum(axis=1)[:, np.newaxis]        normalizer[normalizer == 0.0] = 1.0        proba /= normalizer        return proba        decision = self.decision_function(X)        return self._compute_proba_from_decision(decision, n_classes)      def staged_predict_proba(self, X):         """Predict class probabilities for X.


Question: AdaBoost's "SAMME" algorithm uses 'predict' while fitting and 'predictproba' while predicting probas Subj. This seems to me to be a wrong approach, moreover this drives to such mistakes: <preAdaBoostClassifier(algorithm="SAMME", baseestimator=SVC()).fit(trainX, trainY).predictproba(testX) NotImplementedError Traceback (most recent call last) <ipython-input-108-1d666912dadain <module() ----1 AdaBoostClassifier(algorithm="SAMME", baseestimator=SVC()).fit(trainX, trainY).predictproba(testX) /Library/Python/2.7/site- in predictproba(self, X) 716 proba = sum(estimator.predictproba(X) w 717 for estimator, w in zip(self.estimators, --718 self.estimatorweights)) 719 720 proba /= () /Library/Python/2.7/site- in <genexpr((estimator, w)) 715 else: # self.algorithm == "SAMME" 716 proba = sum(estimator.predictproba(X) w --717 for estimator, w in zip(self.estimators, 718 self.estimatorweights)) 719 /Library/Python/2.7/site- in predictproba(self, X) 493 if not self.probability: 494 raise NotImplementedError( --495 "probability estimates must be enabled to use this method") 496 497 if not in ('csvc', 'nusvc'): NotImplementedError: probability estimates must be enabled to use this method </pre

X = self._validate_data(X)          n_classes = self.n_classes_        proba = None        norm = 0.        for weight, estimator in zip(self.estimator_weights_,                                     self.estimators_):            norm += weight            if self.algorithm == 'SAMME.R':                # The weights are all 1. for SAMME.R                current_proba = _samme_proba(estimator, n_classes, X)            else:  # elif self.algorithm == "SAMME":                current_proba = estimator.predict_proba(X) * weight            if proba is None:                proba = current_proba            else:                proba += current_proba            real_proba = np.exp((1. / (n_classes - 1)) * (proba / norm))            normalizer = real_proba.sum(axis=1)[:, np.newaxis]            normalizer[normalizer == 0.0] = 1.0            real_proba /= normalizer             yield real_proba        for decision in self.staged_decision_function(X):            yield self._compute_proba_from_decision(decision, n_classes)      def predict_log_proba(self, X):         """Predict class log-probabilities for X.


Question: [MRG] Fix 'SparseSeries deprecated: scipy-dev failing on travis' <!-- Thanks for contributing a pull request! Please ensure you have taken a look at the contribution guidelines: -- Issue: SparseSeries deprecated: scipy-dev failing on travis <!-- Example: See also . Please use keywords (e.g., Fixes) to create link to the issues or pull requests you resolved, so that they will automatically be closed when your pull request is merged. See --Use a Series with sparse values instead instead of . <!-- Please be aware that we are a loose team of volunteers so patience is necessary; assistance handling other issues is very welcome. We value all user contributions, no matter how minor they are. If we are slow to review, either the pull request needs some benchmarking, tinkering, convincing, etc. or more likely the reviewers are simply busy. In either case, we ask for your understanding during the review process. For more information, see our FAQ on this topic: http://scikit- Thanks for contributing! --

raise ValueError('Expected array-like (array or non-string sequence), '                          'got %r' % y)     sparseseries = (y.__class__.__name__ == 'SparseSeries')    if sparseseries:        raise ValueError("y cannot be class 'SparseSeries'.")    sparse_pandas = (y.__class__.__name__ in ['SparseSeries', 'SparseArray'])    if sparse_pandas:        raise ValueError("y cannot be class 'SparseSeries' or 'SparseArray'")      if is_multilabel(y):         return 'multilabel-indicator'


Question: Add joblib in showversions joblib should be to the dependencies listed in showversions or to the issue template when sklearn version is 0.20.

"Cython",         "pandas",         "matplotlib",<ins>        "joblib",</ins>     ]      def get_version(module):


Question: Named col indexing fails with ColumnTransformer remainder on changing DataFrame column ordering I am using ColumnTransformer to prepare (impute etc.) heterogenous data. I use a DataFrame to have more control on the different (types of) columns by their name. I had some really cryptic problems when downstream transformers complained of data of the wrong type, while the ColumnTransformer should have divided them up properly. I found out that ColumnTransformer silently passes the wrong columns along as when specifying columns by name, using the option, and using DataFrames where column ordering can differ between and In this case, the wrong columns are passed on to the downstream transformers, as the example demonstrates: When using ColumnTransformer with a DataFrame and specifying columns by name, should reference the same columns when fitting and when transforming (['b'] in above example), regardless of the column positions in the data during fitting and transforming. appears to, during fitting, remember remaining named DataFrame columns by their numeric index (not by their names), which (silently) leads to the wrong columns being handled downstream if the transformed DataFrame's column ordering differs from that of the fitted DataFrame. Position in module where the index is determined: My current workaround is to not use the option but specify all columns by name explicitly. System: python: 3.7.3 (default, Mar 30 2019, 03:44:34) [Clang 9.1.0 (clang-902.0.39.2)] executable: machine: Darwin-17.7.0-x8664-i386-64bit BLAS: macros: NOATLASINFO=3, HAVECBLAS=None libdirs: cblaslibs: cblas Python deps: pip: 19.1.1 setuptools: 41.0.1 sklearn: 0.21.2 numpy: 1.16.4 scipy: 1.3.0 Cython: None pandas: 0.24.2

from ..utils import Bunch from ..utils import safe_indexing from ..utils import _get_column_indices<ins>from ..utils import _check_key_type</ins> from ..utils.metaestimators import _BaseComposition from ..utils.validation import check_array, check_is_fitted


Question: Named col indexing fails with ColumnTransformer remainder on changing DataFrame column ordering I am using ColumnTransformer to prepare (impute etc.) heterogenous data. I use a DataFrame to have more control on the different (types of) columns by their name. I had some really cryptic problems when downstream transformers complained of data of the wrong type, while the ColumnTransformer should have divided them up properly. I found out that ColumnTransformer silently passes the wrong columns along as when specifying columns by name, using the option, and using DataFrames where column ordering can differ between and In this case, the wrong columns are passed on to the downstream transformers, as the example demonstrates: When using ColumnTransformer with a DataFrame and specifying columns by name, should reference the same columns when fitting and when transforming (['b'] in above example), regardless of the column positions in the data during fitting and transforming. appears to, during fitting, remember remaining named DataFrame columns by their numeric index (not by their names), which (silently) leads to the wrong columns being handled downstream if the transformed DataFrame's column ordering differs from that of the fitted DataFrame. Position in module where the index is determined: My current workaround is to not use the option but specify all columns by name explicitly. System: python: 3.7.3 (default, Mar 30 2019, 03:44:34) [Clang 9.1.0 (clang-902.0.39.2)] executable: machine: Darwin-17.7.0-x8664-i386-64bit BLAS: macros: NOATLASINFO=3, HAVECBLAS=None libdirs: cblaslibs: cblas Python deps: pip: 19.1.1 setuptools: 41.0.1 sklearn: 0.21.2 numpy: 1.16.4 scipy: 1.3.0 Cython: None pandas: 0.24.2

By setting ``remainder`` to be an estimator, the remaining         non-specified columns will use the ``remainder`` estimator. The         estimator must support :term:`fit` and :term:`transform`.<ins>        Note that using this feature requires that the DataFrame columns        input at :term:`fit` and :term:`transform` have identical order.</ins>      sparse_threshold : float, default = 0.3         If the output of the different transformers contains sparse matrices,


Question: Named col indexing fails with ColumnTransformer remainder on changing DataFrame column ordering I am using ColumnTransformer to prepare (impute etc.) heterogenous data. I use a DataFrame to have more control on the different (types of) columns by their name. I had some really cryptic problems when downstream transformers complained of data of the wrong type, while the ColumnTransformer should have divided them up properly. I found out that ColumnTransformer silently passes the wrong columns along as when specifying columns by name, using the option, and using DataFrames where column ordering can differ between and In this case, the wrong columns are passed on to the downstream transformers, as the example demonstrates: When using ColumnTransformer with a DataFrame and specifying columns by name, should reference the same columns when fitting and when transforming (['b'] in above example), regardless of the column positions in the data during fitting and transforming. appears to, during fitting, remember remaining named DataFrame columns by their numeric index (not by their names), which (silently) leads to the wrong columns being handled downstream if the transformed DataFrame's column ordering differs from that of the fitted DataFrame. Position in module where the index is determined: My current workaround is to not use the option but specify all columns by name explicitly. System: python: 3.7.3 (default, Mar 30 2019, 03:44:34) [Clang 9.1.0 (clang-902.0.39.2)] executable: machine: Darwin-17.7.0-x8664-i386-64bit BLAS: macros: NOATLASINFO=3, HAVECBLAS=None libdirs: cblaslibs: cblas Python deps: pip: 19.1.1 setuptools: 41.0.1 sklearn: 0.21.2 numpy: 1.16.4 scipy: 1.3.0 Cython: None pandas: 0.24.2

"'passthrough', or estimator. '%s' was passed instead" %                 self.remainder)         n_columns = X.shape[1]        # Make it possible to check for reordered named columns on transform        if (hasattr(X, 'columns') and                any(_check_key_type(cols, str) for cols in self._columns)):            self._df_columns = X.columns        self._n_features = X.shape[1]         cols = []         for columns in self._columns:             cols.extend(_get_column_indices(X, columns))        remaining_idx = sorted(list(set(range(n_columns)) - set(cols))) or None        remaining_idx = list(set(range(self._n_features)) - set(cols))        remaining_idx = sorted(remaining_idx) or None          self._remainder = ('remainder', self.remainder, remaining_idx)


Question: Named col indexing fails with ColumnTransformer remainder on changing DataFrame column ordering I am using ColumnTransformer to prepare (impute etc.) heterogenous data. I use a DataFrame to have more control on the different (types of) columns by their name. I had some really cryptic problems when downstream transformers complained of data of the wrong type, while the ColumnTransformer should have divided them up properly. I found out that ColumnTransformer silently passes the wrong columns along as when specifying columns by name, using the option, and using DataFrames where column ordering can differ between and In this case, the wrong columns are passed on to the downstream transformers, as the example demonstrates: When using ColumnTransformer with a DataFrame and specifying columns by name, should reference the same columns when fitting and when transforming (['b'] in above example), regardless of the column positions in the data during fitting and transforming. appears to, during fitting, remember remaining named DataFrame columns by their numeric index (not by their names), which (silently) leads to the wrong columns being handled downstream if the transformed DataFrame's column ordering differs from that of the fitted DataFrame. Position in module where the index is determined: My current workaround is to not use the option but specify all columns by name explicitly. System: python: 3.7.3 (default, Mar 30 2019, 03:44:34) [Clang 9.1.0 (clang-902.0.39.2)] executable: machine: Darwin-17.7.0-x8664-i386-64bit BLAS: macros: NOATLASINFO=3, HAVECBLAS=None libdirs: cblaslibs: cblas Python deps: pip: 19.1.1 setuptools: 41.0.1 sklearn: 0.21.2 numpy: 1.16.4 scipy: 1.3.0 Cython: None pandas: 0.24.2

"""         check_is_fitted(self, 'transformers_')         X = _check_X(X)        if self._n_features > X.shape[1]:            raise ValueError('Number of features of the input must be equal '                             'to or greater than that of the fitted '                             'transformer. Transformer n_features is {0} '                             'and input n_features is {1}.'                             .format(self._n_features, X.shape[1]))        # No column reordering allowed for named cols combined with remainder        if (self._remainder[2] is not None and                hasattr(self, '_df_columns') and                hasattr(X, 'columns')):            n_cols_fit = len(self._df_columns)            n_cols_transform = len(X.columns)            if (n_cols_transform >= n_cols_fit and                    any(X.columns[:n_cols_fit] != self._df_columns)):                raise ValueError('Column ordering must be equal for fit '                                 'and for transform when using the '                                 'remainder keyword')         Xs = self._fit_transform(X, None, _transform_one, fitted=True)         self._validate_output(Xs)


Question: plotpartialdependence() fails when used on DecisionTreeRegressor <!-- If your issue is a usage question, submit it here instead: StackOverflow with the scikit-learn tag: Mailing List: For more information, see User Questions: http://scikit- --<!-- Instructions For Filing a Bug: --fails when using a as the estimator. The problem appears to be related to the presence of a attribute (with a value of ) on the estimator, despite it being a regressor and not a classifier. Deleting the attribute from the estimator allows to successfully run. <!-- Example: If the code is too long, feel free to put it in a public gist and link it in the issue: --No error is thrown. <!-- Example: No error is thrown. Please paste or describe the expected results.--<!-- Please paste or specifically describe the actual output or traceback. --A is thrown: <!-- Please run the following snippet and paste the output below. For scikit-learn = 0.20: import sklearn; sklearn.show_versions() For scikit-learn < 0.20: import platform; print(platform.platform()) import sys; print("Python", sys.version) import numpy; print("NumPy", numpy.version) import scipy; print("SciPy", scipy.version) import sklearn; print("Scikit-Learn", sklearn.version) --<!-- Thanks for contributing! --

raise ValueError(             "'estimator' must be a fitted regressor or classifier.")     if (hasattr(estimator, 'classes_') and            isinstance(estimator.classes_[0], np.ndarray)):        raise ValueError('Multiclass-multioutput estimators are not supported')    if is_classifier(estimator):        if not hasattr(estimator, 'classes_'):            raise ValueError(                "'estimator' parameter must be a fitted estimator"            )        if isinstance(estimator.classes_[0], np.ndarray):            raise ValueError(                'Multiclass-multioutput estimators are not supported'            )      X = check_array(X)


Question: Pickling Tokenizers fails due to use of lambdas Cannot pickle a using the builtin python module, likely due to the use of lambdas in Example: Program runs without error Traceback: Instead of the builtin , use , which can capture the expression. <!-- Please run the following snippet and paste the output below. For scikit-learn = 0.20: import sklearn; sklearn.showversions() For scikit-learn < 0.20: import platform; print(platform.platform()) import sys; print("Python", sys.version) import numpy; print("NumPy", numpy.version) import scipy; print("SciPy", scipy.version) import sklearn; print("Scikit-Learn", sklearn.version) --Version information: I think this is similar to issues: (looking at the stackoverflow thread at , it suggests using which also seems to work for the toy example) Naively, I would make one of the two changes below, but I am not familiar with the scikit-learn codebase, so they might not be appropriate: the FAQ to direct people to other serialization libraries (perhaps I missed this recommendation?), e.g. at or the use of the lambdas in the vectorizer and replace them with locally def'd functions. I suspect that this solution is flawed because it doesn't account for other uses of lambdas elsewhere in the codebase, and the only complete solution would be to stop using lambdas, but these are a useful language feature.

import array from collections import defaultdict from collections.abc import Mapping<ins>from functools import partial</ins> import numbers from operator import itemgetter import re


Question: Pickling Tokenizers fails due to use of lambdas Cannot pickle a using the builtin python module, likely due to the use of lambdas in Example: Program runs without error Traceback: Instead of the builtin , use , which can capture the expression. <!-- Please run the following snippet and paste the output below. For scikit-learn = 0.20: import sklearn; sklearn.showversions() For scikit-learn < 0.20: import platform; print(platform.platform()) import sys; print("Python", sys.version) import numpy; print("NumPy", numpy.version) import scipy; print("SciPy", scipy.version) import sklearn; print("Scikit-Learn", sklearn.version) --Version information: I think this is similar to issues: (looking at the stackoverflow thread at , it suggests using which also seems to work for the toy example) Naively, I would make one of the two changes below, but I am not familiar with the scikit-learn codebase, so they might not be appropriate: the FAQ to direct people to other serialization libraries (perhaps I missed this recommendation?), e.g. at or the use of the lambdas in the vectorizer and replace them with locally def'd functions. I suspect that this solution is flawed because it doesn't account for other uses of lambdas elsewhere in the codebase, and the only complete solution would be to stop using lambdas, but these are a useful language feature.

'strip_tags']  <ins>def _preprocess(doc, accent_function=None, lower=False):    """Chain together an optional series of text preprocessing steps to    apply to a document.    Parameters    ----------    doc: str        The string to preprocess    accent_function: callable        Function for handling accented characters. Common strategies include        normalizing and removing.    lower: bool        Whether to use str.lower to lowercase all fo the text    Returns    -------    doc: str        preprocessed string    """    if lower:        doc = doc.lower()    if accent_function is not None:        doc = accent_function(doc)    return docdef _analyze(doc, analyzer=None, tokenizer=None, ngrams=None,             preprocessor=None, decoder=None, stop_words=None):    """Chain together an optional series of text processing steps to go from    a single document to ngrams, with or without tokenizing or preprocessing.    If analyzer is used, only the decoder argument is used, as the analyzer is    intended to replace the preprocessor, tokenizer, and ngrams steps.    Parameters    ----------    analyzer: callable    tokenizer: callable    ngrams: callable    preprocessor: callable    decoder: callable    stop_words: list    Returns    -------    ngrams: list        A sequence of tokens, possibly with pairs, triples, etc.    """    if decoder is not None:        doc = decoder(doc)    if analyzer is not None:        doc = analyzer(doc)    else:        if preprocessor is not None:            doc = preprocessor(doc)        if tokenizer is not None:            doc = tokenizer(doc)        if ngrams is not None:            if stop_words is not None:                doc = ngrams(doc, stop_words)            else:                doc = ngrams(doc)    return doc</ins> def strip_accents_unicode(s):     """Transform accentuated unicode symbols into their simple counterpart


Question: Pickling Tokenizers fails due to use of lambdas Cannot pickle a using the builtin python module, likely due to the use of lambdas in Example: Program runs without error Traceback: Instead of the builtin , use , which can capture the expression. <!-- Please run the following snippet and paste the output below. For scikit-learn = 0.20: import sklearn; sklearn.showversions() For scikit-learn < 0.20: import platform; print(platform.platform()) import sys; print("Python", sys.version) import numpy; print("NumPy", numpy.version) import scipy; print("SciPy", scipy.version) import sklearn; print("Scikit-Learn", sklearn.version) --Version information: I think this is similar to issues: (looking at the stackoverflow thread at , it suggests using which also seems to work for the toy example) Naively, I would make one of the two changes below, but I am not familiar with the scikit-learn codebase, so they might not be appropriate: the FAQ to direct people to other serialization libraries (perhaps I missed this recommendation?), e.g. at or the use of the lambdas in the vectorizer and replace them with locally def'd functions. I suspect that this solution is flawed because it doesn't account for other uses of lambdas elsewhere in the codebase, and the only complete solution would be to stop using lambdas, but these are a useful language feature.

if self.preprocessor is not None:             return self.preprocessor         # unfortunately python functools package does not have an efficient        # `compose` function that would have allowed us to chain a dynamic        # number of functions. However the cost of a lambda call is a few        # hundreds of nanoseconds which is negligible when compared to the        # cost of tokenizing a string of 1000 chars for instance.        noop = lambda x: x         # accent stripping         if not self.strip_accents:            strip_accents = noop            strip_accents = None         elif callable(self.strip_accents):             strip_accents = self.strip_accents         elif self.strip_accents == 'ascii':


Question: Pickling Tokenizers fails due to use of lambdas Cannot pickle a using the builtin python module, likely due to the use of lambdas in Example: Program runs without error Traceback: Instead of the builtin , use , which can capture the expression. <!-- Please run the following snippet and paste the output below. For scikit-learn = 0.20: import sklearn; sklearn.showversions() For scikit-learn < 0.20: import platform; print(platform.platform()) import sys; print("Python", sys.version) import numpy; print("NumPy", numpy.version) import scipy; print("SciPy", scipy.version) import sklearn; print("Scikit-Learn", sklearn.version) --Version information: I think this is similar to issues: (looking at the stackoverflow thread at , it suggests using which also seems to work for the toy example) Naively, I would make one of the two changes below, but I am not familiar with the scikit-learn codebase, so they might not be appropriate: the FAQ to direct people to other serialization libraries (perhaps I missed this recommendation?), e.g. at or the use of the lambdas in the vectorizer and replace them with locally def'd functions. I suspect that this solution is flawed because it doesn't account for other uses of lambdas elsewhere in the codebase, and the only complete solution would be to stop using lambdas, but these are a useful language feature.

raise ValueError('Invalid value for "strip_accents": %s' %                              self.strip_accents)         if self.lowercase:            return lambda x: strip_accents(x.lower())        else:            return strip_accents        return partial(            _preprocess, accent_function=strip_accents, lower=self.lowercase        )      def build_tokenizer(self):         """Return a function that splits a string into a sequence of tokens"""         if self.tokenizer is not None:             return self.tokenizer         token_pattern = re.compile(self.token_pattern)        return lambda doc: token_pattern.findall(doc)        return token_pattern.findall      def get_stop_words(self):         """Build or fetch the effective stop words list"""


Question: Pickling Tokenizers fails due to use of lambdas Cannot pickle a using the builtin python module, likely due to the use of lambdas in Example: Program runs without error Traceback: Instead of the builtin , use , which can capture the expression. <!-- Please run the following snippet and paste the output below. For scikit-learn = 0.20: import sklearn; sklearn.showversions() For scikit-learn < 0.20: import platform; print(platform.platform()) import sys; print("Python", sys.version) import numpy; print("NumPy", numpy.version) import scipy; print("SciPy", scipy.version) import sklearn; print("Scikit-Learn", sklearn.version) --Version information: I think this is similar to issues: (looking at the stackoverflow thread at , it suggests using which also seems to work for the toy example) Naively, I would make one of the two changes below, but I am not familiar with the scikit-learn codebase, so they might not be appropriate: the FAQ to direct people to other serialization libraries (perhaps I missed this recommendation?), e.g. at or the use of the lambdas in the vectorizer and replace them with locally def'd functions. I suspect that this solution is flawed because it doesn't account for other uses of lambdas elsewhere in the codebase, and the only complete solution would be to stop using lambdas, but these are a useful language feature.

if callable(self.analyzer):             if self.input in ['file', 'filename']:                 self._validate_custom_analyzer()            return lambda doc: self.analyzer(self.decode(doc))            return partial(                _analyze, analyzer=self.analyzer, decoder=self.decode            )          preprocess = self.build_preprocessor()          if self.analyzer == 'char':            return lambda doc: self._char_ngrams(preprocess(self.decode(doc)))            return partial(_analyze, ngrams=self._char_ngrams,                           preprocessor=preprocess, decoder=self.decode)          elif self.analyzer == 'char_wb':            return lambda doc: self._char_wb_ngrams(                preprocess(self.decode(doc)))            return partial(_analyze, ngrams=self._char_wb_ngrams,                           preprocessor=preprocess, decoder=self.decode)          elif self.analyzer == 'word':             stop_words = self.get_stop_words()             tokenize = self.build_tokenizer()             self._check_stop_words_consistency(stop_words, preprocess,                                                tokenize)            return lambda doc: self._word_ngrams(                tokenize(preprocess(self.decode(doc))), stop_words)            return partial(_analyze, ngrams=self._word_ngrams,                           tokenizer=tokenize, preprocessor=preprocess,                           decoder=self.decode, stop_words=stop_words)          else:             raise ValueError('%s is not a valid tokenization scheme/analyzer' %


Question: PLS reports "array must not contain nan" if a feature is constant Originally reported at by Reproduce with: The obscure error message is due to the presence of a column containing only 0.

similar to the Power method for determining the eigenvectors and     eigenvalues of a X'Y.     """    y_score = Y[:, [0]]    for col in Y.T:        if np.any(np.abs(col) > np.finfo(np.double).eps):            y_score = col.reshape(len(col), 1)            break     x_weights_old = 0     ite = 1     X_pinv = Y_pinv = None


Question: Need for conversion with SAG Running the following code snippet will lead to an error: I am wondering if we are not to strict here, and we should include a since that will expect a C-contiguous numpy array.

seed=seed)         intercept_decay = SPARSE_INTERCEPT_DECAY     else:        X = np.ascontiguousarray(X)         dataset = ArrayData(X, y, sample_weight, seed=seed)         intercept_decay = 1.0 diff --git a/sklearn/linear_model/ridge.py b/sklearn/linear_model/ridge.py-- a/sklearn/linear_model/ridge.py++ b/sklearn/linear_model/ridge.py


Question: Need for conversion with SAG Running the following code snippet will lead to an error: I am wondering if we are not to strict here, and we should include a since that will expect a C-contiguous numpy array.

_accept_sparse = _get_valid_accept_sparse(sparse.issparse(X), solver)         X = check_array(X, accept_sparse=_accept_sparse, dtype=_dtype,                         order="C")        y = check_array(y, dtype=X.dtype, ensure_2d=False, order="C")        y = check_array(y, dtype=X.dtype, ensure_2d=False, order=None)     check_consistent_length(X, y)      n_samples, n_features = X.shape


Question: Cloning custom transform replaces values in init dictionary <!-- If your issue is a usage question, submit it here instead: StackOverflow with the scikit-learn tag: Mailing List: For more information, see User Questions: http://scikit- --<!-- Instructions For Filing a Bug: --<!-- Example: Joblib Error thrown when calling fit on LatentDirichletAllocation with evaluateevery 0--Let us say we have a custom transform that has some arguments. When the is instantiated, these arguments are set in the init. When we clone (as happens in , for example), the arguments get copied successfully. However, if the arguments are sent to a structure such as a dictionary, the clone replaces them with None. In cases where None does not cause errors, this creates a silent error, as the cloned version of will run, producing different results from its original version (which is how I ran into this problem in the first place). Fully replicable example follows. <!-- Thanks for contributing! --

"""         out = dict()         for key in self._get_param_names():            value = getattr(self, key, None)            try:                value = getattr(self, key)            except AttributeError:                warnings.warn('From version 0.24, get_params will raise an '                              'AttributeError if a parameter cannot be '                              'retrieved as an instance attribute. Previously '                              'it would return None.',                              FutureWarning)                value = None             if deep and hasattr(value, 'get_params'):                 deep_items = value.get_params().items()                 out.update((key + '__' + k, val) for k, val in deep_items)diff --git a/sklearn/gaussian_process/kernels.py b/sklearn/gaussian_process/kernels.py-- a/sklearn/gaussian_process/kernels.py++ b/sklearn/gaussian_process/kernels.py


Question: Cloning custom transform replaces values in init dictionary <!-- If your issue is a usage question, submit it here instead: StackOverflow with the scikit-learn tag: Mailing List: For more information, see User Questions: http://scikit- --<!-- Instructions For Filing a Bug: --<!-- Example: Joblib Error thrown when calling fit on LatentDirichletAllocation with evaluateevery 0--Let us say we have a custom transform that has some arguments. When the is instantiated, these arguments are set in the init. When we clone (as happens in , for example), the arguments get copied successfully. However, if the arguments are sent to a structure such as a dictionary, the clone replaces them with None. In cases where None does not cause errors, this creates a silent error, as the cloned version of will run, producing different results from its original version (which is how I ran into this problem in the first place). Fully replicable example follows. <!-- Thanks for contributing! --

from collections import namedtuple import math from inspect import signature<ins>import warnings</ins>  import numpy as np from scipy.special import kv, gamma


Question: Cloning custom transform replaces values in init dictionary <!-- If your issue is a usage question, submit it here instead: StackOverflow with the scikit-learn tag: Mailing List: For more information, see User Questions: http://scikit- --<!-- Instructions For Filing a Bug: --<!-- Example: Joblib Error thrown when calling fit on LatentDirichletAllocation with evaluateevery 0--Let us say we have a custom transform that has some arguments. When the is instantiated, these arguments are set in the init. When we clone (as happens in , for example), the arguments get copied successfully. However, if the arguments are sent to a structure such as a dictionary, the clone replaces them with None. In cases where None does not cause errors, this creates a silent error, as the cloned version of will run, producing different results from its original version (which is how I ran into this problem in the first place). Fully replicable example follows. <!-- Thanks for contributing! --

" %s doesn't follow this convention."                                % (cls, ))         for arg in args:            params[arg] = getattr(self, arg, None)            try:                value = getattr(self, arg)            except AttributeError:                warnings.warn('From version 0.24, get_params will raise an '                              'AttributeError if a parameter cannot be '                              'retrieved as an instance attribute. Previously '                              'it would return None.',                              FutureWarning)                value = None            params[arg] = value         return params      def set_params(self, **params):diff --git a/sklearn/pipeline.py b/sklearn/pipeline.py-- a/sklearn/pipeline.py++ b/sklearn/pipeline.py


Question: Cloning custom transform replaces values in init dictionary <!-- If your issue is a usage question, submit it here instead: StackOverflow with the scikit-learn tag: Mailing List: For more information, see User Questions: http://scikit- --<!-- Instructions For Filing a Bug: --<!-- Example: Joblib Error thrown when calling fit on LatentDirichletAllocation with evaluateevery 0--Let us say we have a custom transform that has some arguments. When the is instantiated, these arguments are set in the init. When we clone (as happens in , for example), the arguments get copied successfully. However, if the arguments are sent to a structure such as a dictionary, the clone replaces them with None. In cases where None does not cause errors, this creates a silent error, as the cloned version of will run, producing different results from its original version (which is how I ran into this problem in the first place). Fully replicable example follows. <!-- Thanks for contributing! --

def __init__(self, steps, memory=None, verbose=False):         self.steps = steps        self._validate_steps()         self.memory = memory         self.verbose = verbose        self._validate_steps()      def get_params(self, deep=True):         """Get parameters for this estimator.


Question: [BUG] Optics float minsamples NN instantiation None yet. When passing minsamples as a float to optics l439 & 440 execute to bring it into integer ranges, but don't convert to int: When instantiating the NearestNeighbours class with a float it raises due to the float (l448). Error message: Fix: the int(...) is for backwards compatibbility to Python 2 where with T Number, while Python3 <!-- Please be aware that we are a loose team of volunteers so patience is necessary; assistance handling other issues is very welcome. We value all user contributions, no matter how minor they are. If we are slow to review, either the pull request needs some benchmarking, tinkering, convincing, etc. or more likely the reviewers are simply busy. In either case, we ask for your understanding during the review process. For more information, see our FAQ on this topic: http://scikit- Thanks for contributing! --

Parameters     ----------    min_samples : int > 1 or float between 0 and 1 (default=None)    min_samples : int > 1 or float between 0 and 1 (default=5)         The number of samples in a neighborhood for a point to be considered as         a core point. Also, up and down steep regions can't have more then         ``min_samples`` consecutive non-steep points. Expressed as an absolute


Question: [BUG] Optics float minsamples NN instantiation None yet. When passing minsamples as a float to optics l439 & 440 execute to bring it into integer ranges, but don't convert to int: When instantiating the NearestNeighbours class with a float it raises due to the float (l448). Error message: Fix: the int(...) is for backwards compatibbility to Python 2 where with T Number, while Python3 <!-- Please be aware that we are a loose team of volunteers so patience is necessary; assistance handling other issues is very welcome. We value all user contributions, no matter how minor they are. If we are slow to review, either the pull request needs some benchmarking, tinkering, convincing, etc. or more likely the reviewers are simply busy. In either case, we ask for your understanding during the review process. For more information, see our FAQ on this topic: http://scikit- Thanks for contributing! --

A feature array, or array of distances between samples if         metric='precomputed'     min_samples : int (default=5)    min_samples : int > 1 or float between 0 and 1         The number of samples in a neighborhood for a point to be considered         as a core point. Expressed as an absolute number or a fraction of the         number of samples (rounded to be at least 2).


Question: [BUG] Optics float minsamples NN instantiation None yet. When passing minsamples as a float to optics l439 & 440 execute to bring it into integer ranges, but don't convert to int: When instantiating the NearestNeighbours class with a float it raises due to the float (l448). Error message: Fix: the int(...) is for backwards compatibbility to Python 2 where with T Number, while Python3 <!-- Please be aware that we are a loose team of volunteers so patience is necessary; assistance handling other issues is very welcome. We value all user contributions, no matter how minor they are. If we are slow to review, either the pull request needs some benchmarking, tinkering, convincing, etc. or more likely the reviewers are simply busy. In either case, we ask for your understanding during the review process. For more information, see our FAQ on this topic: http://scikit- Thanks for contributing! --

n_samples = X.shape[0]     _validate_size(min_samples, n_samples, 'min_samples')     if min_samples <= 1:        min_samples = max(2, min_samples * n_samples)        min_samples = max(2, int(min_samples * n_samples))      # Start all points as 'unprocessed' ##     reachability_ = np.empty(n_samples)