Question: FITS-standard unit parsing fails on some types of exponents Why don't these work: When these all do: The non-working versions give e.g.: which is not how I would interpret the . Tested on 2.0.7 and 3.0.3

_lr_method = 'LALR' _lr_signature = 'DOUBLE_STAR STAR PERIOD SOLIDUS CARET OPEN_PAREN CLOSE_PAREN FUNCNAME UNIT SIGN UINT UFLOAT\n            main : product_of_units\n                 | factor product_of_units\n                 | factor product product_of_units\n                 | division_product_of_units\n                 | factor division_product_of_units\n                 | factor product division_product_of_units\n                 | inverse_unit\n                 | factor inverse_unit\n                 | factor product inverse_unit\n                 | factor\n            \n            division_product_of_units : division_product_of_units division product_of_units\n                                      | product_of_units\n            \n            inverse_unit : division unit_expression\n            \n            factor : factor_fits\n                   | factor_float\n                   | factor_int\n            \n            factor_float : signed_float\n                         | signed_float UINT signed_int\n                         | signed_float UINT power numeric_power\n            \n            factor_int : UINT\n                       | UINT signed_int\n                       | UINT power numeric_power\n                       | UINT UINT signed_int\n                       | UINT UINT power numeric_power\n            \n            factor_fits : UINT power OPEN_PAREN signed_int CLOSE_PAREN\n                        | UINT power signed_int\n                        | UINT SIGN UINT\n                        | UINT OPEN_PAREN signed_int CLOSE_PAREN\n            \n            product_of_units : unit_expression product product_of_units\n                             | unit_expression product_of_units\n                             | unit_expression\n            \n            unit_expression : function\n                            | unit_with_power\n                            | OPEN_PAREN product_of_units CLOSE_PAREN\n            \n            unit_with_power : UNIT power numeric_power\n                            | UNIT numeric_power\n                            | UNIT\n            \n            numeric_power : sign UINT\n                          | OPEN_PAREN paren_expr CLOSE_PAREN\n            \n            paren_expr : sign UINT\n                       | signed_float\n                       | frac\n            \n            frac : sign UINT division sign UINT\n            \n            sign : SIGN\n                 |\n            \n            product : STAR\n                    | PERIOD\n            \n            division : SOLIDUS\n            \n            power : DOUBLE_STAR\n                  | CARET\n            \n            signed_int : SIGN UINT\n            \n            signed_float : sign UINT\n                         | sign UFLOAT\n            \n            function_name : FUNCNAME\n            \n            function : function_name OPEN_PAREN main CLOSE_PAREN\n            '_lr_signature = 'DOUBLE_STAR STAR PERIOD SOLIDUS CARET OPEN_PAREN CLOSE_PAREN FUNCNAME UNIT SIGN UINT UFLOAT\n            main : product_of_units\n                 | factor product_of_units\n                 | factor product product_of_units\n                 | division_product_of_units\n                 | factor division_product_of_units\n                 | factor product division_product_of_units\n                 | inverse_unit\n                 | factor inverse_unit\n                 | factor product inverse_unit\n                 | factor\n            \n            division_product_of_units : division_product_of_units division product_of_units\n                                      | product_of_units\n            \n            inverse_unit : division unit_expression\n            \n            factor : factor_fits\n                   | factor_float\n                   | factor_int\n            \n            factor_float : signed_float\n                         | signed_float UINT signed_int\n                         | signed_float UINT power numeric_power\n            \n            factor_int : UINT\n                       | UINT signed_int\n                       | UINT power numeric_power\n                       | UINT UINT signed_int\n                       | UINT UINT power numeric_power\n            \n            factor_fits : UINT power OPEN_PAREN signed_int CLOSE_PAREN\n                        | UINT power OPEN_PAREN UINT CLOSE_PAREN\n                        | UINT power signed_int\n                        | UINT power UINT\n                        | UINT SIGN UINT\n                        | UINT OPEN_PAREN signed_int CLOSE_PAREN\n            \n            product_of_units : unit_expression product product_of_units\n                             | unit_expression product_of_units\n                             | unit_expression\n            \n            unit_expression : function\n                            | unit_with_power\n                            | OPEN_PAREN product_of_units CLOSE_PAREN\n            \n            unit_with_power : UNIT power numeric_power\n                            | UNIT numeric_power\n                            | UNIT\n            \n            numeric_power : sign UINT\n                          | OPEN_PAREN paren_expr CLOSE_PAREN\n            \n            paren_expr : sign UINT\n                       | signed_float\n                       | frac\n            \n            frac : sign UINT division sign UINT\n            \n            sign : SIGN\n                 |\n            \n            product : STAR\n                    | PERIOD\n            \n            division : SOLIDUS\n            \n            power : DOUBLE_STAR\n                  | CARET\n            \n            signed_int : SIGN UINT\n            \n            signed_float : sign UINT\n                         | sign UFLOAT\n            \n            function_name : FUNCNAME\n            \n            function : function_name OPEN_PAREN main CLOSE_PAREN\n            '     _lr_action_items = {'OPEN_PAREN':([0,3,6,7,8,9,10,11,12,13,14,16,17,18,19,21,23,26,27,28,29,34,36,38,39,41,42,43,46,47,53,54,55,58,59,62,63,64,66,67,72,73,75,76,77,78,80,],[13,13,13,-14,-15,-16,13,-32,-33,13,35,-17,-48,41,45,-54,13,-46,-47,13,13,57,-21,-49,-50,13,45,-36,-52,-53,-34,-23,45,-26,-22,-27,-18,45,-35,-38,-24,-51,-28,-19,-55,-39,-25,]),'UINT':([0,14,15,16,17,19,20,34,37,38,39,41,42,44,45,46,47,55,56,57,60,64,69,81,82,],[14,33,-44,40,-48,-45,46,-45,62,-49,-50,14,-45,67,-45,-52,-53,-45,73,-45,73,-45,79,-45,83,]),'SOLIDUS':([0,2,3,4,6,7,8,9,11,12,14,16,19,22,23,24,26,27,30,36,41,43,46,47,48,49,51,52,53,54,58,59,62,63,66,67,72,73,75,76,77,78,79,80,],[17,-12,17,17,-31,-14,-15,-16,-32,-33,-20,-17,-37,-12,17,17,-46,-47,-30,-21,17,-36,-52,-53,-12,17,-11,-29,-34,-23,-26,-22,-27,-18,-35,-38,-24,-51,-28,-19,-55,-39,17,-25,]),'UNIT':([0,3,6,7,8,9,10,11,12,13,14,16,17,19,23,26,27,28,29,36,41,43,46,47,53,54,58,59,62,63,66,67,72,73,75,76,77,78,80,],[19,19,19,-14,-15,-16,19,-32,-33,19,-20,-17,-48,-37,19,-46,-47,19,19,-21,19,-36,-52,-53,-34,-23,-26,-22,-27,-18,-35,-38,-24,-51,-28,-19,-55,-39,-25,]),'FUNCNAME':([0,3,6,7,8,9,10,11,12,13,14,16,17,19,23,26,27,28,29,36,41,43,46,47,53,54,58,59,62,63,66,67,72,73,75,76,77,78,80,],[21,21,21,-14,-15,-16,21,-32,-33,21,-20,-17,-48,-37,21,-46,-47,21,21,-21,21,-36,-52,-53,-34,-23,-26,-22,-27,-18,-35,-38,-24,-51,-28,-19,-55,-39,-25,]),'SIGN':([0,14,17,19,33,34,35,38,39,40,41,42,45,55,57,64,81,],[15,37,-48,15,56,60,56,-49,-50,56,15,15,15,15,60,15,15,]),'UFLOAT':([0,15,20,41,45,57,60,69,],[-45,-44,47,-45,-45,-45,-44,47,]),'$end':([1,2,3,4,5,6,7,8,9,11,12,14,16,19,22,24,25,30,31,36,43,46,47,48,49,50,51,52,53,54,58,59,62,63,66,67,72,73,75,76,77,78,80,],[0,-1,-10,-4,-7,-31,-14,-15,-16,-32,-33,-20,-17,-37,-2,-5,-8,-30,-13,-21,-36,-52,-53,-3,-6,-9,-11,-29,-34,-23,-26,-22,-27,-18,-35,-38,-24,-51,-28,-19,-55,-39,-25,]),'CLOSE_PAREN':([2,3,4,5,6,7,8,9,11,12,14,16,19,22,24,25,30,31,32,36,43,46,47,48,49,50,51,52,53,54,58,59,61,62,63,65,66,67,68,70,71,72,73,74,75,76,77,78,79,80,83,],[-1,-10,-4,-7,-31,-14,-15,-16,-32,-33,-20,-17,-37,-2,-5,-8,-30,-13,53,-21,-36,-52,-53,-3,-6,-9,-11,-29,-34,-23,-26,-22,75,-27,-18,77,-35,-38,78,-41,-42,-24,-51,80,-28,-19,-55,-39,-40,-25,-43,]),'STAR':([3,6,7,8,9,11,12,14,16,19,36,43,46,47,53,54,58,59,62,63,66,67,72,73,75,76,77,78,80,],[26,26,-14,-15,-16,-32,-33,-20,-17,-37,-21,-36,-52,-53,-34,-23,-26,-22,-27,-18,-35,-38,-24,-51,-28,-19,-55,-39,-25,]),'PERIOD':([3,6,7,8,9,11,12,14,16,19,36,43,46,47,53,54,58,59,62,63,66,67,72,73,75,76,77,78,80,],[27,27,-14,-15,-16,-32,-33,-20,-17,-37,-21,-36,-52,-53,-34,-23,-26,-22,-27,-18,-35,-38,-24,-51,-28,-19,-55,-39,-25,]),'DOUBLE_STAR':([14,19,33,40,],[38,38,38,38,]),'CARET':([14,19,33,40,],[39,39,39,39,]),}_lr_action_items = {'OPEN_PAREN':([0,3,6,7,8,9,10,11,12,13,14,16,17,18,19,21,23,26,27,28,29,34,36,38,39,41,42,43,46,47,53,54,55,57,59,60,63,64,65,67,68,73,74,77,78,79,80,82,83,],[13,13,13,-14,-15,-16,13,-34,-35,13,35,-17,-50,41,45,-56,13,-48,-49,13,13,58,-21,-51,-52,13,45,-38,-54,-55,-36,-23,45,-28,-27,-22,-29,-18,45,-37,-40,-24,-53,-30,-19,-57,-41,-26,-25,]),'UINT':([0,14,15,16,17,19,20,34,37,38,39,41,42,44,45,46,47,55,56,58,61,65,70,84,85,],[14,33,-46,40,-50,-47,46,57,63,-51,-52,14,-47,68,-47,-54,-55,-47,74,75,74,-47,81,-47,86,]),'SOLIDUS':([0,2,3,4,6,7,8,9,11,12,14,16,19,22,23,24,26,27,30,36,41,43,46,47,48,49,51,52,53,54,57,59,60,63,64,67,68,73,74,77,78,79,80,81,82,83,],[17,-12,17,17,-33,-14,-15,-16,-34,-35,-20,-17,-39,-12,17,17,-48,-49,-32,-21,17,-38,-54,-55,-12,17,-11,-31,-36,-23,-28,-27,-22,-29,-18,-37,-40,-24,-53,-30,-19,-57,-41,17,-26,-25,]),'UNIT':([0,3,6,7,8,9,10,11,12,13,14,16,17,19,23,26,27,28,29,36,41,43,46,47,53,54,57,59,60,63,64,67,68,73,74,77,78,79,80,82,83,],[19,19,19,-14,-15,-16,19,-34,-35,19,-20,-17,-50,-39,19,-48,-49,19,19,-21,19,-38,-54,-55,-36,-23,-28,-27,-22,-29,-18,-37,-40,-24,-53,-30,-19,-57,-41,-26,-25,]),'FUNCNAME':([0,3,6,7,8,9,10,11,12,13,14,16,17,19,23,26,27,28,29,36,41,43,46,47,53,54,57,59,60,63,64,67,68,73,74,77,78,79,80,82,83,],[21,21,21,-14,-15,-16,21,-34,-35,21,-20,-17,-50,-39,21,-48,-49,21,21,-21,21,-38,-54,-55,-36,-23,-28,-27,-22,-29,-18,-37,-40,-24,-53,-30,-19,-57,-41,-26,-25,]),'SIGN':([0,14,17,19,33,34,35,38,39,40,41,42,45,55,58,65,84,],[15,37,-50,15,56,61,56,-51,-52,56,15,15,15,15,61,15,15,]),'UFLOAT':([0,15,20,41,45,58,61,70,],[-47,-46,47,-47,-47,-47,-46,47,]),'$end':([1,2,3,4,5,6,7,8,9,11,12,14,16,19,22,24,25,30,31,36,43,46,47,48,49,50,51,52,53,54,57,59,60,63,64,67,68,73,74,77,78,79,80,82,83,],[0,-1,-10,-4,-7,-33,-14,-15,-16,-34,-35,-20,-17,-39,-2,-5,-8,-32,-13,-21,-38,-54,-55,-3,-6,-9,-11,-31,-36,-23,-28,-27,-22,-29,-18,-37,-40,-24,-53,-30,-19,-57,-41,-26,-25,]),'CLOSE_PAREN':([2,3,4,5,6,7,8,9,11,12,14,16,19,22,24,25,30,31,32,36,43,46,47,48,49,50,51,52,53,54,57,59,60,62,63,64,66,67,68,69,71,72,73,74,75,76,77,78,79,80,81,82,83,86,],[-1,-10,-4,-7,-33,-14,-15,-16,-34,-35,-20,-17,-39,-2,-5,-8,-32,-13,53,-21,-38,-54,-55,-3,-6,-9,-11,-31,-36,-23,-28,-27,-22,77,-29,-18,79,-37,-40,80,-43,-44,-24,-53,82,83,-30,-19,-57,-41,-42,-26,-25,-45,]),'STAR':([3,6,7,8,9,11,12,14,16,19,36,43,46,47,53,54,57,59,60,63,64,67,68,73,74,77,78,79,80,82,83,],[26,26,-14,-15,-16,-34,-35,-20,-17,-39,-21,-38,-54,-55,-36,-23,-28,-27,-22,-29,-18,-37,-40,-24,-53,-30,-19,-57,-41,-26,-25,]),'PERIOD':([3,6,7,8,9,11,12,14,16,19,36,43,46,47,53,54,57,59,60,63,64,67,68,73,74,77,78,79,80,82,83,],[27,27,-14,-15,-16,-34,-35,-20,-17,-39,-21,-38,-54,-55,-36,-23,-28,-27,-22,-29,-18,-37,-40,-24,-53,-30,-19,-57,-41,-26,-25,]),'DOUBLE_STAR':([14,19,33,40,],[38,38,38,38,]),'CARET':([14,19,33,40,],[39,39,39,39,]),}  _lr_action = {} for _k, _v in _lr_action_items.items():


Question: FITS-standard unit parsing fails on some types of exponents Why don't these work: When these all do: The non-working versions give e.g.: which is not how I would interpret the . Tested on 2.0.7 and 3.0.3


Question: FITS-standard unit parsing fails on some types of exponents Why don't these work: When these all do: The non-working versions give e.g.: which is not how I would interpret the . Tested on 2.0.7 and 3.0.3

('factor_int -> UINT UINT signed_int','factor_int',3,'p_factor_int','generic.py',257),   ('factor_int -> UINT UINT power numeric_power','factor_int',4,'p_factor_int','generic.py',258),   ('factor_fits -> UINT power OPEN_PAREN signed_int CLOSE_PAREN','factor_fits',5,'p_factor_fits','generic.py',276),  ('factor_fits -> UINT power signed_int','factor_fits',3,'p_factor_fits','generic.py',277),  ('factor_fits -> UINT SIGN UINT','factor_fits',3,'p_factor_fits','generic.py',278),  ('factor_fits -> UINT OPEN_PAREN signed_int CLOSE_PAREN','factor_fits',4,'p_factor_fits','generic.py',279),  ('product_of_units -> unit_expression product product_of_units','product_of_units',3,'p_product_of_units','generic.py',298),  ('product_of_units -> unit_expression product_of_units','product_of_units',2,'p_product_of_units','generic.py',299),  ('product_of_units -> unit_expression','product_of_units',1,'p_product_of_units','generic.py',300),  ('unit_expression -> function','unit_expression',1,'p_unit_expression','generic.py',311),  ('unit_expression -> unit_with_power','unit_expression',1,'p_unit_expression','generic.py',312),  ('unit_expression -> OPEN_PAREN product_of_units CLOSE_PAREN','unit_expression',3,'p_unit_expression','generic.py',313),  ('unit_with_power -> UNIT power numeric_power','unit_with_power',3,'p_unit_with_power','generic.py',322),  ('unit_with_power -> UNIT numeric_power','unit_with_power',2,'p_unit_with_power','generic.py',323),  ('unit_with_power -> UNIT','unit_with_power',1,'p_unit_with_power','generic.py',324),  ('numeric_power -> sign UINT','numeric_power',2,'p_numeric_power','generic.py',335),  ('numeric_power -> OPEN_PAREN paren_expr CLOSE_PAREN','numeric_power',3,'p_numeric_power','generic.py',336),  ('paren_expr -> sign UINT','paren_expr',2,'p_paren_expr','generic.py',345),  ('paren_expr -> signed_float','paren_expr',1,'p_paren_expr','generic.py',346),  ('paren_expr -> frac','paren_expr',1,'p_paren_expr','generic.py',347),  ('frac -> sign UINT division sign UINT','frac',5,'p_frac','generic.py',356),  ('sign -> SIGN','sign',1,'p_sign','generic.py',362),  ('sign -> <empty>','sign',0,'p_sign','generic.py',363),  ('product -> STAR','product',1,'p_product','generic.py',372),  ('product -> PERIOD','product',1,'p_product','generic.py',373),  ('division -> SOLIDUS','division',1,'p_division','generic.py',379),  ('power -> DOUBLE_STAR','power',1,'p_power','generic.py',385),  ('power -> CARET','power',1,'p_power','generic.py',386),  ('signed_int -> SIGN UINT','signed_int',2,'p_signed_int','generic.py',392),  ('signed_float -> sign UINT','signed_float',2,'p_signed_float','generic.py',398),  ('signed_float -> sign UFLOAT','signed_float',2,'p_signed_float','generic.py',399),  ('function_name -> FUNCNAME','function_name',1,'p_function_name','generic.py',405),  ('function -> function_name OPEN_PAREN main CLOSE_PAREN','function',4,'p_function','generic.py',411),  ('factor_fits -> UINT power OPEN_PAREN UINT CLOSE_PAREN','factor_fits',5,'p_factor_fits','generic.py',277),  ('factor_fits -> UINT power signed_int','factor_fits',3,'p_factor_fits','generic.py',278),  ('factor_fits -> UINT power UINT','factor_fits',3,'p_factor_fits','generic.py',279),  ('factor_fits -> UINT SIGN UINT','factor_fits',3,'p_factor_fits','generic.py',280),  ('factor_fits -> UINT OPEN_PAREN signed_int CLOSE_PAREN','factor_fits',4,'p_factor_fits','generic.py',281),  ('product_of_units -> unit_expression product product_of_units','product_of_units',3,'p_product_of_units','generic.py',300),  ('product_of_units -> unit_expression product_of_units','product_of_units',2,'p_product_of_units','generic.py',301),  ('product_of_units -> unit_expression','product_of_units',1,'p_product_of_units','generic.py',302),  ('unit_expression -> function','unit_expression',1,'p_unit_expression','generic.py',313),  ('unit_expression -> unit_with_power','unit_expression',1,'p_unit_expression','generic.py',314),  ('unit_expression -> OPEN_PAREN product_of_units CLOSE_PAREN','unit_expression',3,'p_unit_expression','generic.py',315),  ('unit_with_power -> UNIT power numeric_power','unit_with_power',3,'p_unit_with_power','generic.py',324),  ('unit_with_power -> UNIT numeric_power','unit_with_power',2,'p_unit_with_power','generic.py',325),  ('unit_with_power -> UNIT','unit_with_power',1,'p_unit_with_power','generic.py',326),  ('numeric_power -> sign UINT','numeric_power',2,'p_numeric_power','generic.py',337),  ('numeric_power -> OPEN_PAREN paren_expr CLOSE_PAREN','numeric_power',3,'p_numeric_power','generic.py',338),  ('paren_expr -> sign UINT','paren_expr',2,'p_paren_expr','generic.py',347),  ('paren_expr -> signed_float','paren_expr',1,'p_paren_expr','generic.py',348),  ('paren_expr -> frac','paren_expr',1,'p_paren_expr','generic.py',349),  ('frac -> sign UINT division sign UINT','frac',5,'p_frac','generic.py',358),  ('sign -> SIGN','sign',1,'p_sign','generic.py',364),  ('sign -> <empty>','sign',0,'p_sign','generic.py',365),  ('product -> STAR','product',1,'p_product','generic.py',374),  ('product -> PERIOD','product',1,'p_product','generic.py',375),  ('division -> SOLIDUS','division',1,'p_division','generic.py',381),  ('power -> DOUBLE_STAR','power',1,'p_power','generic.py',387),  ('power -> CARET','power',1,'p_power','generic.py',388),  ('signed_int -> SIGN UINT','signed_int',2,'p_signed_int','generic.py',394),  ('signed_float -> sign UINT','signed_float',2,'p_signed_float','generic.py',400),  ('signed_float -> sign UFLOAT','signed_float',2,'p_signed_float','generic.py',401),  ('function_name -> FUNCNAME','function_name',1,'p_function_name','generic.py',407),  ('function -> function_name OPEN_PAREN main CLOSE_PAREN','function',4,'p_function','generic.py',413), ]


Question: units: 's / m' and 's / m' are not convertible To paraphrase from , exceptions like (and a few other cases with first, second and third powers of second over meter, curiously) started popping up in our travis tests as seen at https://travis- . For a brief overview, running and getting into : I've managed to figure out that this is localized to 3.1. As suggested by on slack, I ran with and I was able to identify the first bad commit as I'm currently trying to find out the exact cause of this bug in that commit, but I figured it's best to submit this issue here before I accidentally close this tab or something.

"bases must be sequence of UnitBase instances")             powers = [validate_power(p) for p in powers]         if not decompose and len(bases) == 1:            # Short-cut; with one unit there's nothing to expand and gather.        if not decompose and len(bases) == 1 and powers[0] >= 0:            # Short-cut; with one unit there's nothing to expand and gather,            # as that has happened already when creating the unit.  But do only            # positive powers, since for negative powers we need to re-sort.             unit = bases[0]             power = powers[0]             if power == 1:


Question: units: 's / m' and 's / m' are not convertible To paraphrase from , exceptions like (and a few other cases with first, second and third powers of second over meter, curiously) started popping up in our travis tests as seen at https://travis- . For a brief overview, running and getting into : I've managed to figure out that this is localized to 3.1. As suggested by on slack, I ran with and I was able to identify the first bad commit as I'm currently trying to find out the exact cause of this bug in that commit, but I figured it's best to submit this issue here before I accidentally close this tab or something.

self._bases = unit.bases                 self._powers = [operator.mul(*resolve_fractions(p, power))                                 for p in unit.powers]<ins></ins>             self._scale = sanitize_scale(scale)         else:             # Regular case: use inputs as preliminary scale, bases, and powers,


Question: Problem with the part of unit equivalencies? In the newly equivalencies: We notice that the implementation of seems to be wrong, as highlighted in the following figure: ! If , and , should it be equivalent to 140 Mpc, instead of 70Mpc? I can reproduce this so it is not a typo...

from astropy import cosmology         H0 = cosmology.default_cosmology.get().H0     h100_val_unit = Unit(H0.to((si.km/si.s)/astrophys.Mpc).value/100 * astrophys.littleh)    h100_val_unit = Unit(100/(H0.to_value((si.km/si.s)/astrophys.Mpc)) * astrophys.littleh)      return [(h100_val_unit, None)]


Question: ncpprior referenced before assignment in Bayesian Blocks There is a bug in the bayesian blocks algorithm of It's not a big deal so I show you below how to solve it directly. When I call: I get: You can fix this just by changing: adding an else sentence EDIT: Text formatting ncpprior referenced before assignment in Bayesian Blocks There is a bug in the bayesian blocks algorithm of It's not a big deal so I show you below how to solve it directly. When I call: I get: You can fix this just by changing: adding an else sentence EDIT: Text formatting

If ``ncp_prior`` is not explicitly defined, compute it from ``gamma``         or ``p0``.         """        if self.ncp_prior is not None:            return self.ncp_prior        elif self.gamma is not None:        if self.gamma is not None:             return -np.log(self.gamma)         elif self.p0 is not None:             return self.p0_prior(N)         else:            raise ValueError("``ncp_prior`` is not defined, and cannot compute "                             "it as neither ``gamma`` nor ``p0`` is defined.")            raise ValueError("``ncp_prior`` cannot be computed as neither "                             "``gamma`` nor ``p0`` is defined.")      def fit(self, t, x=None, sigma=None):         """Fit the Bayesian Blocks model given the specified fitness function.


Question: ncpprior referenced before assignment in Bayesian Blocks There is a bug in the bayesian blocks algorithm of It's not a big deal so I show you below how to solve it directly. When I call: I get: You can fix this just by changing: adding an else sentence EDIT: Text formatting ncpprior referenced before assignment in Bayesian Blocks There is a bug in the bayesian blocks algorithm of It's not a big deal so I show you below how to solve it directly. When I call: I get: You can fix this just by changing: adding an else sentence EDIT: Text formatting

# Compute ncp_prior if not defined         if self.ncp_prior is None:             ncp_prior = self.compute_ncp_prior(N)<ins>        else:            ncp_prior = self.ncp_prior</ins>         # ----------------------------------------------------------------         # Start with first data cell; add one cell at each iteration         # ----------------------------------------------------------------


Question: Adding/subtracting ABmag Quantities loses the "type" of magnitude The following code raises a , because it appears the math operation lost track of the "type" of magnitude. and are both , so I would expect their difference to also be an ABmag. prints the following, and then raises: If the is changed to , the exception is different:

import numpy as np from astropy.units import (Unit, UnitBase, UnitsError, UnitTypeError,from astropy.units import (Unit, UnitBase, UnitsError, UnitTypeError, UnitConversionError,                            dimensionless_unscaled, Quantity)  __all__ = ['FunctionUnitBase', 'FunctionQuantity']


Question: Adding/subtracting ABmag Quantities loses the "type" of magnitude The following code raises a , because it appears the math operation lost track of the "type" of magnitude. and are both , so I would expect their difference to also be an ABmag. prints the following, and then raises: If the is changed to , the exception is different:

return self.function_unit.to(other_function_unit, value)          else:            # when other is not a function unit            return self.physical_unit.to(other, self.to_physical(value),                                         equivalencies)            try:                # when other is not a function unit                return self.physical_unit.to(other, self.to_physical(value),                                             equivalencies)            except UnitConversionError as e:                if self.function_unit == Unit('mag'):                    # One can get to raw magnitudes via math that strips the dimensions off.                    # Include extra information in the exception to remind users of this.                    msg = "Did you perhaps subtract magnitudes so the unit got lost?"                    e.args += (msg,)                    raise e                else:                    raise      def is_unity(self):         return False


Question: Header.fromstring does not accept Python 3 bytes According to , the method "...creates an HDU header from a byte string containing the entire header data." By "byte string" here it really means the type which on Python 2 could be raw binary data, but on Python 3 explicitly is not. In fact it does work on Python 3's unicode s, but here it assumes that the data can be ASCII-encoded. Its counterpart, will work with files opened in text or binary mode. So probably the simplest solution for now (as opposed to adding new methods or something like that) is to change to accept unicode or bytes string types. likely needs a similar treatment.

"""          card = cls()        if isinstance(image, bytes):            # FITS supports only ASCII, but decode as latin1 and just take all            # bytes for now; if it results in mojibake due to e.g. UTF-8            # encoded data in a FITS header that's OK because it shouldn't be            # there in the first place            image = image.decode('latin1')         card._image = _pad(image)         card._verified = False         return carddiff --git a/astropy/io/fits/header.py b/astropy/io/fits/header.py-- a/astropy/io/fits/header.py++ b/astropy/io/fits/header.py


Question: Header.fromstring does not accept Python 3 bytes According to , the method "...creates an HDU header from a byte string containing the entire header data." By "byte string" here it really means the type which on Python 2 could be raw binary data, but on Python 3 explicitly is not. In fact it does work on Python 3's unicode s, but here it assumes that the data can be ASCII-encoded. Its counterpart, will work with files opened in text or binary mode. So probably the simplest solution for now (as opposed to adding new methods or something like that) is to change to accept unicode or bytes string types. likely needs a similar treatment.

END_CARD = 'END' + ' ' * 77  __doctest_skip__ = ['Header', 'Header.*']__doctest_skip__ = ['Header', 'Header.comments', 'Header.fromtextfile',                    'Header.totextfile', 'Header.set', 'Header.update']   class Header:


Question: Header.fromstring does not accept Python 3 bytes According to , the method "...creates an HDU header from a byte string containing the entire header data." By "byte string" here it really means the type which on Python 2 could be raw binary data, but on Python 3 explicitly is not. In fact it does work on Python 3's unicode s, but here it assumes that the data can be ASCII-encoded. Its counterpart, will work with files opened in text or binary mode. So probably the simplest solution for now (as opposed to adding new methods or something like that) is to change to accept unicode or bytes string types. likely needs a similar treatment.

Parameters         ----------        data : str           String containing the entire header.        data : str or bytes           String or bytes containing the entire header.  In the case of bytes           they will be decoded using latin-1 (only plain ASCII characters are           allowed in FITS headers but latin-1 allows us to retain any invalid           bytes that might appear in malformatted FITS files).          sep : str, optional             The string separating cards from each other, such as a newline.  By             default there is no card separator (as is the case in a raw FITS            file).            file).  In general this is only used in cases where a header was            printed as text (e.g. with newlines after each card) and you want            to create a new `Header` from it by copy/pasting.        Examples        --------        >>> from astropy.io.fits import Header        >>> hdr = Header({'SIMPLE': True})        >>> Header.fromstring(hdr.tostring()) == hdr        True        If you want to create a `Header` from printed text it's not necessary        to have the exact binary structure as it would appear in a FITS file,        with the full 80 byte card length.  Rather, each "card" can end in a        newline and does not have to be padded out to a full card length as        long as it "looks like" a FITS header:        >>> hdr = Header.fromstring(\"\"\"\\        ... SIMPLE  =                    T / conforms to FITS standard        ... BITPIX  =                    8 / array data type        ... NAXIS   =                    0 / number of array dimensions        ... EXTEND  =                    T        ... \"\"\", sep='\\n')        >>> hdr['SIMPLE']        True        >>> hdr['BITPIX']        8        >>> len(hdr)        4          Returns         -------


Question: Header.fromstring does not accept Python 3 bytes According to , the method "...creates an HDU header from a byte string containing the entire header data." By "byte string" here it really means the type which on Python 2 could be raw binary data, but on Python 3 explicitly is not. In fact it does work on Python 3's unicode s, but here it assumes that the data can be ASCII-encoded. Its counterpart, will work with files opened in text or binary mode. So probably the simplest solution for now (as opposed to adding new methods or something like that) is to change to accept unicode or bytes string types. likely needs a similar treatment.

# immediately at the separator         require_full_cardlength = set(sep).issubset(VALID_HEADER_CHARS) <ins>        if isinstance(data, bytes):            # FITS supports only ASCII, but decode as latin1 and just take all            # bytes for now; if it results in mojibake due to e.g. UTF-8            # encoded data in a FITS header that's OK because it shouldn't be            # there in the first place--accepting it here still gives us the            # opportunity to display warnings later during validation            CONTINUE = b'CONTINUE'            END = b'END'            end_card = END_CARD.encode('ascii')            sep = sep.encode('latin1')            empty = b''        else:            CONTINUE = 'CONTINUE'            END = 'END'            end_card = END_CARD            empty = ''</ins>         # Split the header into individual cards         idx = 0         image = []


Question: Header.fromstring does not accept Python 3 bytes According to , the method "...creates an HDU header from a byte string containing the entire header data." By "byte string" here it really means the type which on Python 2 could be raw binary data, but on Python 3 explicitly is not. In fact it does work on Python 3's unicode s, but here it assumes that the data can be ASCII-encoded. Its counterpart, will work with files opened in text or binary mode. So probably the simplest solution for now (as opposed to adding new methods or something like that) is to change to accept unicode or bytes string types. likely needs a similar treatment.

idx = end_idx + len(sep)              if image:                if next_image[:8] == 'CONTINUE':                if next_image[:8] == CONTINUE:                     image.append(next_image)                     continue                cards.append(Card.fromstring(''.join(image)))                cards.append(Card.fromstring(empty.join(image)))              if require_full_cardlength:                if next_image == END_CARD:                if next_image == end_card:                     image = []                     break             else:                if next_image.split(sep)[0].rstrip() == 'END':                if next_image.split(sep)[0].rstrip() == END:                     image = []                     break


Question: Header.fromstring does not accept Python 3 bytes According to , the method "...creates an HDU header from a byte string containing the entire header data." By "byte string" here it really means the type which on Python 2 could be raw binary data, but on Python 3 explicitly is not. In fact it does work on Python 3's unicode s, but here it assumes that the data can be ASCII-encoded. Its counterpart, will work with files opened in text or binary mode. So probably the simplest solution for now (as opposed to adding new methods or something like that) is to change to accept unicode or bytes string types. likely needs a similar treatment.

# Add the last image that was found before the end, if any         if image:            cards.append(Card.fromstring(''.join(image)))            cards.append(Card.fromstring(empty.join(image)))          return cls._fromcards(cards)


Question: Support new clip ufunc Starting with numpy 1.17, will be based on a , which means we can ensure it works properly with . (Until we do so, we might also get failures.)

def cumsum(self, axis=None, dtype=None, out=None):         return self._wrap_function(np.cumsum, axis, dtype, out=out)    def clip(self, a_min, a_max, out=None):        return self._wrap_function(np.clip, self._to_own_unit(a_min),                                   self._to_own_unit(a_max), out=out)diff --git a/astropy/units/quantity.py b/astropy/units/quantity.py-- a/astropy/units/quantity.py++ b/astropy/units/quantity.py


Question: Support new clip ufunc Starting with numpy 1.17, will be based on a , which means we can ensure it works properly with . (Until we do so, we might also get failures.)

UnitBase, UnitsError, UnitConversionError, UnitTypeError) from .utils import is_effectively_unity from .format.latex import Latexfrom astropy.utils.compat import NUMPY_LT_1_14, NUMPY_LT_1_16from astropy.utils.compat import NUMPY_LT_1_14, NUMPY_LT_1_16, NUMPY_LT_1_17 from astropy.utils.compat.misc import override__dir__ from astropy.utils.exceptions import AstropyDeprecationWarning, AstropyWarning from astropy.utils.misc import isiterable, InheritDocstrings


Question: Support new clip ufunc Starting with numpy 1.17, will be based on a , which means we can ensure it works properly with . (Until we do so, we might also get failures.)

kwargs['out'] = (out_array,) if function.nout == 1 else out_array          # Same for inputs, but here also convert if necessary.        arrays = [(converter(input_.value) if converter else                   getattr(input_, 'value', input_))                  for input_, converter in zip(inputs, converters)]        arrays = []        for input_, converter in zip(inputs, converters):            input_ = getattr(input_, 'value', input_)            arrays.append(converter(input_) if converter else input_)          # Call our superclass's __array_ufunc__         result = super().__array_ufunc__(function, method, *arrays, **kwargs)


Question: Support new clip ufunc Starting with numpy 1.17, will be based on a , which means we can ensure it works properly with . (Until we do so, we might also get failures.)

result = function(*args, **kwargs)         return self._result_as_quantity(result, unit, out)     def clip(self, a_min, a_max, out=None):        return self._wrap_function(np.clip, self._to_own_unit(a_min),                                   self._to_own_unit(a_max), out=out)    if NUMPY_LT_1_17:        def clip(self, a_min, a_max, out=None):            return self._wrap_function(np.clip, self._to_own_unit(a_min),                                       self._to_own_unit(a_max), out=out)      def trace(self, offset=0, axis1=0, axis2=1, dtype=None, out=None):         return self._wrap_function(np.trace, offset, axis1, axis2, dtype,diff --git a/astropy/units/quantity_helper/converters.py b/astropy/units/quantity_helper/converters.py-- a/astropy/units/quantity_helper/converters.py++ b/astropy/units/quantity_helper/converters.py


Question: Support new clip ufunc Starting with numpy 1.17, will be based on a , which means we can ensure it works properly with . (Until we do so, we might also get failures.)

converters, result_unit = ufunc_helper(function, *units)          if any(converter is False for converter in converters):            # for two-argument ufuncs with a quantity and a non-quantity,            # for multi-argument ufuncs with a quantity and a non-quantity,             # the quantity normally needs to be dimensionless, *except*             # if the non-quantity can have arbitrary unit, i.e., when it             # is all zero, infinity or NaN.  In that case, the non-quantity             # can just have the unit of the quantity             # (this allows, e.g., `q > 0.` independent of unit)            maybe_arbitrary_arg = args[converters.index(False)]             try:                if can_have_arbitrary_unit(maybe_arbitrary_arg):                    converters = [None, None]                else:                    raise UnitConversionError(                        "Can only apply '{0}' function to "                        "dimensionless quantities when other "                        "argument is not a quantity (unless the "                        "latter is all zero/infinity/nan)"                        .format(function.__name__))                # Don't fold this loop in the test above: this rare case                # should not make the common case slower.                for i, converter in enumerate(converters):                    if converter is not False:                        continue                    if can_have_arbitrary_unit(args[i]):                        converters[i] = None                    else:                        raise UnitConversionError(                            "Can only apply '{0}' function to "                            "dimensionless quantities when other "                            "argument is not a quantity (unless the "                            "latter is all zero/infinity/nan)"                            .format(function.__name__))             except TypeError:                 # _can_have_arbitrary_unit failed: arg could not be compared                 # with zero or checked to be finite. Then, ufunc will fail too.                 raise TypeError("Unsupported operand type(s) for ufunc {0}: "                                "'{1}' and '{2}'"                                .format(function.__name__,                                        args[0].__class__.__name__,                                        args[1].__class__.__name__))                                "'{1}'".format(function.__name__,                                               ','.join([arg.__class__.__name__                                                         for arg in args])))          # In the case of np.power and np.float_power, the unit itself needs to         # be modified by an amount that depends on one of the input values,diff --git a/astropy/units/quantity_helper/helpers.py b/astropy/units/quantity_helper/helpers.py-- a/astropy/units/quantity_helper/helpers.py++ b/astropy/units/quantity_helper/helpers.py


Question: Support new clip ufunc Starting with numpy 1.17, will be based on a , which means we can ensure it works properly with . (Until we do so, we might also get failures.)

return converters, (dimensionless_unscaled, result_unit)  <ins>def helper_clip(f, unit1, unit2, unit3):    # Treat the array being clipped as primary.    converters = [None]    if unit1 is None:        result_unit = dimensionless_unscaled        try:            converters += [(None if unit is None else                            get_converter(unit, dimensionless_unscaled))                           for unit in (unit2, unit3)]        except UnitsError:            raise UnitConversionError(                "Can only apply '{0}' function to quantities with "                "compatible dimensions".format(f.__name__))    else:        result_unit = unit1        for unit in unit2, unit3:            try:                converter = get_converter(_d(unit), result_unit)            except UnitsError:                if unit is None:                    # special case: OK if unitless number is zero, inf, nan                    converters.append(False)                else:                    raise UnitConversionError(                        "Can only apply '{0}' function to quantities with "                        "compatible dimensions".format(f.__name__))            else:                converters.append(converter)    return converters, result_unit</ins> # list of ufuncs: # http://docs.scipy.org/doc/numpy/reference/ufuncs.html#available-ufuncs


Question: Support new clip ufunc Starting with numpy 1.17, will be based on a , which means we can ensure it works properly with . (Until we do so, we might also get failures.)

UFUNC_HELPERS[np.heaviside] = helper_heaviside UFUNC_HELPERS[np.float_power] = helper_power UFUNC_HELPERS[np.divmod] = helper_divmod# Check for clip ufunc; note that np.clip is a wrapper function, not the ufunc.if isinstance(getattr(np.core.umath, 'clip', None), np.ufunc):    UFUNC_HELPERS[np.core.umath.clip] = helper_clipdiff --git a/astropy/utils/compat/numpycompat.py b/astropy/utils/compat/numpycompat.py-- a/astropy/utils/compat/numpycompat.py++ b/astropy/utils/compat/numpycompat.py


Question: Support new clip ufunc Starting with numpy 1.17, will be based on a , which means we can ensure it works properly with . (Until we do so, we might also get failures.)

__all__ = ['NUMPY_LT_1_14', 'NUMPY_LT_1_14_1', 'NUMPY_LT_1_14_2',           'NUMPY_LT_1_16']           'NUMPY_LT_1_16', 'NUMPY_LT_1_17']  # TODO: It might also be nice to have aliases to these named for specific # features/bugs we're checking for (ex:


Question: Support new clip ufunc Starting with numpy 1.17, will be based on a , which means we can ensure it works properly with . (Until we do so, we might also get failures.)

NUMPY_LT_1_14_1 = not minversion('numpy', '1.14.1') NUMPY_LT_1_14_2 = not minversion('numpy', '1.14.2') NUMPY_LT_1_16 = not minversion('numpy', '1.16')<ins>NUMPY_LT_1_17 = not minversion('numpy', '1.17')</ins>


Question: float16 quantities get upgraded to float64 automatically When trying to create a from a (not something I actually intended to do, I was experimenting while investigating other issue) it gets upgraded automatically to , which is something that does not happen with other float types: However: Somewhat related:

dtype : ~numpy.dtype, optional         The dtype of the resulting Numpy array or scalar that will         hold the value.  If not provided, it is determined from the input,        except that any input that cannot represent float (integer and bool)        is converted to float.        except that any integer and (non-Quantity) object inputs are converted        to float by default.      copy : bool, optional         If `True` (default), then the value is copied.  Otherwise, a copy will


Question: float16 quantities get upgraded to float64 automatically When trying to create a from a (not something I actually intended to do, I was experimenting while investigating other issue) it gets upgraded automatically to , which is something that does not happen with other float types: However: Somewhat related:

if not copy:                     return value                 if not (np.can_cast(np.float32, value.dtype) or                        value.dtype.fields):                if value.dtype.kind in 'iu':                     dtype = float              return np.array(value, dtype=dtype, copy=copy, order=order,


Question: float16 quantities get upgraded to float64 automatically When trying to create a from a (not something I actually intended to do, I was experimenting while investigating other issue) it gets upgraded automatically to , which is something that does not happen with other float types: However: Somewhat related:

"Numpy numeric type.")          # by default, cast any integer, boolean, etc., to float        if dtype is None and (not (np.can_cast(np.float32, value.dtype)                                   or value.dtype.fields)                              or value.dtype.kind == 'O'):        if dtype is None and value.dtype.kind in 'iuO':             value = value.astype(float)          value = value.view(cls)


Question: Misleading sqlmigrate "App 'apps.somethings' does not have migrations." error message Description This ticket is very similar to As shown above, validation should be sqlmigrate.

<ins>from django.apps import apps</ins> from django.core.management.base import BaseCommand, CommandError from django.db import DEFAULT_DB_ALIAS, connections from django.db.migrations.executor import MigrationExecutor


Question: Misleading sqlmigrate "App 'apps.somethings' does not have migrations." error message Description This ticket is very similar to As shown above, validation should be sqlmigrate.

# Resolve command-line arguments into a migration         app_label, migration_name = options['app_label'], options['migration_name']<ins>        # Validate app_label        try:            apps.get_app_config(app_label)        except LookupError as err:            raise CommandError(str(err))</ins>         if app_label not in executor.loader.migrated_apps:             raise CommandError("App '%s' does not have migrations" % app_label)         try:


Question: Make URLValidator reject invalid characters in the username and password Description (last modified by Tim Bell) Since , core.validators.URLValidator accepts URLs with usernames and passwords. RFC 1738 section 3.1 requires "Within the user and password field, any ":", " or "/" must be encoded"; however, those characters are currently accepted without being %-encoded. That allows certain invalid URLs to pass validation incorrectly. (The issue originates in Diego Perini's gist, from which the implementation in was derived.) An example URL that should be invalid is ; furthermore, many of the test cases in would be rendered valid under the current implementation by appending a query string of the form to them. I note Tim Graham's concern about adding complexity to the validation regex. However, I take the opposite position to Danilo Bargen about invalid URL edge cases: it's not fine if invalid URLs (even so-called "edge cases") are accepted when the regex could be fixed simply to reject them correctly. I also note that a URL of the form above was encountered in a production setting, so that this is a genuine use case, not merely an academic exercise. Pull request:  Make URLValidator reject invalid characters in the username and password Description (last modified by Tim Bell) Since , core.validators.URLValidator accepts URLs with usernames and passwords. RFC 1738 section 3.1 requires "Within the user and password field, any ":", " or "/" must be encoded"; however, those characters are currently accepted without being %-encoded. That allows certain invalid URLs to pass validation incorrectly. (The issue originates in Diego Perini's gist, from which the implementation in was derived.) An example URL that should be invalid is ; furthermore, many of the test cases in would be rendered valid under the current implementation by appending a query string of the form to them. I note Tim Graham's concern about adding complexity to the validation regex. However, I take the opposite position to Danilo Bargen about invalid URL edge cases: it's not fine if invalid URLs (even so-called "edge cases") are accepted when the regex could be fixed simply to reject them correctly. I also note that a URL of the form above was encountered in a production setting, so that this is a genuine use case, not merely an academic exercise. Pull request:

regex = _lazy_re_compile(         r'^(?:[a-z0-9\.\-\+]*)://'  # scheme is validated separately        r'(?:\S+(?::\S*)?@)?'  # user:pass authentication        r'(?:[^\s:@/]+(?::[^\s:@/]*)?@)?'  # user:pass authentication         r'(?:' + ipv4_re + '|' + ipv6_re + '|' + host_re + ')'         r'(?::\d{2,5})?'  # port         r'(?:[/?#][^\s]*)?'  # resource path


Question: Add an option to django-admin to always colorize output Description With Django management commands, it is currently possible disable colors with the --no-colors flag. What I'd like to have is basically the other side of the coin: a --force-colors flag that instructs Django to output ANSI color sequences in cases it would disable colors by default (typically, when the output is piped to another command, as documented). My real world use-case is the following one: I have a custom Django command to import data. I run this command myself, and I'd like to send a colored log (HTML seems perfect for this) to the data curators. I can use the  utility for this, but that doesn't work since Django disable colors when the output is piped. Other *nix commands have a special flag for this exact use-case, for example $ ls --color=always

"""     show_last = {         '--version', '--verbosity', '--traceback', '--settings', '--pythonpath',        '--no-color',        '--no-color', '--force_color',     }      def _reordered_actions(self, actions):


Question: Add an option to django-admin to always colorize output Description With Django management commands, it is currently possible disable colors with the --no-colors flag. What I'd like to have is basically the other side of the coin: a --force-colors flag that instructs Django to output ANSI color sequences in cases it would disable colors by default (typically, when the output is piped to another command, as documented). My real world use-case is the following one: I have a custom Django command to import data. I run this command myself, and I'd like to send a colored log (HTML seems perfect for this) to the data curators. I can use the  utility for this, but that doesn't work since Django disable colors when the output is piped. Other *nix commands have a special flag for this exact use-case, for example $ ls --color=always

# Command-specific options not defined by the argument parser.     stealth_options = ()     def __init__(self, stdout=None, stderr=None, no_color=False):    def __init__(self, stdout=None, stderr=None, no_color=False, force_color=False):         self.stdout = OutputWrapper(stdout or sys.stdout)         self.stderr = OutputWrapper(stderr or sys.stderr)        if no_color and force_color:            raise CommandError("'no_color' and 'force_color' can't be used together.")         if no_color:             self.style = no_style()         else:            self.style = color_style()            self.style = color_style(force_color)             self.stderr.style_func = self.style.ERROR      def get_version(self):


Question: Add an option to django-admin to always colorize output Description With Django management commands, it is currently possible disable colors with the --no-colors flag. What I'd like to have is basically the other side of the coin: a --force-colors flag that instructs Django to output ANSI color sequences in cases it would disable colors by default (typically, when the output is piped to another command, as documented). My real world use-case is the following one: I have a custom Django command to import data. I run this command myself, and I'd like to send a colored log (HTML seems perfect for this) to the data curators. I can use the  utility for this, but that doesn't work since Django disable colors when the output is piped. Other *nix commands have a special flag for this exact use-case, for example $ ls --color=always

'--no-color', action='store_true',             help="Don't colorize the command output.",         )<ins>        parser.add_argument(            '--force-color', action='store_true',            help='Force colorization of the command output.',        )</ins>         self.add_arguments(parser)         return parser


Question: Add an option to django-admin to always colorize output Description With Django management commands, it is currently possible disable colors with the --no-colors flag. What I'd like to have is basically the other side of the coin: a --force-colors flag that instructs Django to output ANSI color sequences in cases it would disable colors by default (typically, when the output is piped to another command, as documented). My real world use-case is the following one: I have a custom Django command to import data. I run this command myself, and I'd like to send a colored log (HTML seems perfect for this) to the data curators. I can use the  utility for this, but that doesn't work since Django disable colors when the output is piped. Other *nix commands have a special flag for this exact use-case, for example $ ls --color=always

controlled by the ``requires_system_checks`` attribute, except if         force-skipped).         """        if options['no_color']:        if options['force_color'] and options['no_color']:            raise CommandError("The --no-color and --force-color options can't be used together.")        if options['force_color']:            self.style = color_style(force_color=True)        elif options['no_color']:             self.style = no_style()             self.stderr.style_func = None         if options.get('stdout'):diff --git a/django/core/management/color.py b/django/core/management/color.py-- a/django/core/management/color.py++ b/django/core/management/color.py


Question: Add an option to django-admin to always colorize output Description With Django management commands, it is currently possible disable colors with the --no-colors flag. What I'd like to have is basically the other side of the coin: a --force-colors flag that instructs Django to output ANSI color sequences in cases it would disable colors by default (typically, when the output is piped to another command, as documented). My real world use-case is the following one: I have a custom Django command to import data. I run this command myself, and I'd like to send a colored log (HTML seems perfect for this) to the data curators. I can use the  utility for this, but that doesn't work since Django disable colors when the output is piped. Other *nix commands have a special flag for this exact use-case, for example $ ls --color=always

return make_style('nocolor')  def color_style():def color_style(force_color=False):     """     Return a Style object from the Django color scheme.     """    if not supports_color():    if not force_color and not supports_color():         return no_style()     return make_style(os.environ.get('DJANGO_COLORS', ''))


Question: Add validation of related fields and lookups in model Meta.ordering Description When the ordering class member in Meta of a model contains a field from a related model, and that field does not exist, django's makemigrations does not throw an error. However, if it is a direct field member of the same class, makemigrations does throw an error. Example below tested on Django 2.0.5 from import models class Agreement(models.Model): agreementid = models.AutoField(verbosename='ID', serialize=False, autocreated=True, primarykey=True) #class Meta: # generates error in makemigrations # app.Agreement: (models.E015) 'ordering' refers to the nonexistent field 'id'. # ordering = ['id'] class Order(models.Model): agreement = models.ForeignKey(Agreement, models.DONOTHING) class Meta: # does not generate error in makemigrations # but does so during runtime # e.g. [x for x in ()] ordering = ['agreement_id']

# Convert "-field" to "field".         fields = ((f[1:] if f.startswith('-') else f) for f in fields)         # Skip ordering in the format field1__field2 (FIXME: checking        # this format would be nice, but it's a little fiddly).        fields = (f for f in fields if LOOKUP_SEP not in f)        # Separate related field and non related fields.        _fields = []        related_fields = []        for f in fields:            if LOOKUP_SEP in f:                related_fields.append(f)            else:                _fields.append(f)        fields = _fields        # Check related fields.        for field in related_fields:            _cls = cls            fld = None            for part in field.split(LOOKUP_SEP):                try:                    fld = _cls._meta.get_field(part)                    if fld.is_relation:                        _cls = fld.get_path_info()[-1].to_opts.model                except (FieldDoesNotExist, AttributeError):                    if fld is None or fld.get_transform(part) is None:                        errors.append(                            checks.Error(                                "'ordering' refers to the nonexistent field, "                                "related field or lookup '%s'." % field,                                obj=cls,                                id='models.E015',                            )                        )          # Skip ordering on pk. This is always a valid order_by field         # but is an alias and therefore won't be found by opts.get_field.


Question: diffsettings raises misleading exception message if using settings.configure() Description If, rather than using the env var DJANGOSETTINGSMODULE one uses settings.configure(...), attempting to call diffsettings can fail because it calls () unconditionally, with the exception Requested settings, but settings are not configured. You must either define the environment variable DJANGOSETTINGSMODULE or call settings.configure() before accessing settings. were the call instead: if not settings.configured: () things would work correctly.

from django.conf import settings, Settings, global_settings          # Because settings are imported lazily, we need to explicitly load them.        settings._setup()        if not settings.configured:            settings._setup()          user_settings = module_to_dict(settings._wrapped)         default = options['default']


Question: Trunc() should allow passing isdst resolution to avoid NonExistentTimeError/AmbiguousTimeError Description (last modified by Alexander Holmbäck) When Trunc() truncates to a nonexisting or ambiguous datetime, the exception raised by pytz remains unhandled. The expected behavior would, IMO, be to not check the validity of truncated dates. This test for example: import datetime import pytz from import Trunc from import TestCase from import timezone from .models import Log class TestTruncateToInvalidTime(TestCase): def testtruncatetodstendsstockholm(self): tzinfo = pytz.timezone('Europe/Stockholm') timestamp = datetime.datetime(2018, 10, 28, 2, tzinfo=tzinfo) (timestamp=timestamp) logs = Log.objects.annotate(day=Trunc('timestamp', 'hour')).all() timezone.activate(tzinfo) self.assertEqual(logs[0], 28) Results in the following error: ERROR: testtruncatetodstendsstockholm () Traceback (most recent call last): File "", line 47, in testtruncatetodstendsstockholm self.assertEqual(logs[0], 28) File "", line 303, in getitem qs.fetchall() File "", line 1190, in fetchall self.resultcache = list(self.iterableclass(self)) File "", line 64, in iter for row in compiler.resultsiter(results): File "", line 1013, in applyconverters value = converter(value, expression, connection) File "", line 225, in convertvalue value = timezone.makeaware(value, ) File "", line 270, in makeaware return timezone.localize(value, isdst=is_dst) File "/home/alex/.virtualenvs/djangodev/lib/python3.6/site-", line 363, in localize raise AmbiguousTimeError(dt) pytz.exceptions.AmbiguousTimeError: 2018-10-28 02:00:00

kind = None     tzinfo = None     def __init__(self, expression, output_field=None, tzinfo=None, **extra):    def __init__(self, expression, output_field=None, tzinfo=None, is_dst=None, **extra):         self.tzinfo = tzinfo        self.is_dst = is_dst         super().__init__(expression, output_field=output_field, **extra)      def as_sql(self, compiler, connection):


Question: Trunc() should allow passing isdst resolution to avoid NonExistentTimeError/AmbiguousTimeError Description (last modified by Alexander Holmbäck) When Trunc() truncates to a nonexisting or ambiguous datetime, the exception raised by pytz remains unhandled. The expected behavior would, IMO, be to not check the validity of truncated dates. This test for example: import datetime import pytz from import Trunc from import TestCase from import timezone from .models import Log class TestTruncateToInvalidTime(TestCase): def testtruncatetodstendsstockholm(self): tzinfo = pytz.timezone('Europe/Stockholm') timestamp = datetime.datetime(2018, 10, 28, 2, tzinfo=tzinfo) (timestamp=timestamp) logs = Log.objects.annotate(day=Trunc('timestamp', 'hour')).all() timezone.activate(tzinfo) self.assertEqual(logs[0], 28) Results in the following error: ERROR: testtruncatetodstendsstockholm () Traceback (most recent call last): File "", line 47, in testtruncatetodstendsstockholm self.assertEqual(logs[0], 28) File "", line 303, in getitem qs.fetchall() File "", line 1190, in fetchall self.resultcache = list(self.iterableclass(self)) File "", line 64, in iter for row in compiler.resultsiter(results): File "", line 1013, in applyconverters value = converter(value, expression, connection) File "", line 225, in convertvalue value = timezone.makeaware(value, ) File "", line 270, in makeaware return timezone.localize(value, isdst=is_dst) File "/home/alex/.virtualenvs/djangodev/lib/python3.6/site-", line 363, in localize raise AmbiguousTimeError(dt) pytz.exceptions.AmbiguousTimeError: 2018-10-28 02:00:00

pass             elif value is not None:                 value = value.replace(tzinfo=None)                value = timezone.make_aware(value, self.tzinfo)                value = timezone.make_aware(value, self.tzinfo, is_dst=self.is_dst)             elif not connection.features.has_zoneinfo_database:                 raise ValueError(                     'Database returned an invalid datetime value. Are time '


Question: Trunc() should allow passing isdst resolution to avoid NonExistentTimeError/AmbiguousTimeError Description (last modified by Alexander Holmbäck) When Trunc() truncates to a nonexisting or ambiguous datetime, the exception raised by pytz remains unhandled. The expected behavior would, IMO, be to not check the validity of truncated dates. This test for example: import datetime import pytz from import Trunc from import TestCase from import timezone from .models import Log class TestTruncateToInvalidTime(TestCase): def testtruncatetodstendsstockholm(self): tzinfo = pytz.timezone('Europe/Stockholm') timestamp = datetime.datetime(2018, 10, 28, 2, tzinfo=tzinfo) (timestamp=timestamp) logs = Log.objects.annotate(day=Trunc('timestamp', 'hour')).all() timezone.activate(tzinfo) self.assertEqual(logs[0], 28) Results in the following error: ERROR: testtruncatetodstendsstockholm () Traceback (most recent call last): File "", line 47, in testtruncatetodstendsstockholm self.assertEqual(logs[0], 28) File "", line 303, in getitem qs.fetchall() File "", line 1190, in fetchall self.resultcache = list(self.iterableclass(self)) File "", line 64, in iter for row in compiler.resultsiter(results): File "", line 1013, in applyconverters value = converter(value, expression, connection) File "", line 225, in convertvalue value = timezone.makeaware(value, ) File "", line 270, in makeaware return timezone.localize(value, isdst=is_dst) File "/home/alex/.virtualenvs/djangodev/lib/python3.6/site-", line 363, in localize raise AmbiguousTimeError(dt) pytz.exceptions.AmbiguousTimeError: 2018-10-28 02:00:00

class Trunc(TruncBase):     def __init__(self, expression, kind, output_field=None, tzinfo=None, **extra):    def __init__(self, expression, kind, output_field=None, tzinfo=None, is_dst=None, **extra):         self.kind = kind        super().__init__(expression, output_field=output_field, tzinfo=tzinfo, **extra)        super().__init__(            expression, output_field=output_field, tzinfo=tzinfo,            is_dst=is_dst, **extra        )   class TruncYear(TruncBase):


Question: Add app_label validation to showmigrations Description The app label validation was to some commands as above tickets. But showmigrations command doesn't contain it. So I think the validation should be to showmigrations command.

from django.core.management.base import BaseCommand, CommandErrorimport sysfrom django.apps import appsfrom django.core.management.base import BaseCommand from django.db import DEFAULT_DB_ALIAS, connections from django.db.migrations.loader import MigrationLoader


Question: Add app_label validation to showmigrations Description The app label validation was to some commands as above tickets. But showmigrations command doesn't contain it. So I think the validation should be to showmigrations command.

return self.show_list(connection, options['app_label'])      def _validate_app_names(self, loader, app_names):        invalid_apps = []        has_bad_names = False         for app_name in app_names:            if app_name not in loader.migrated_apps:                invalid_apps.append(app_name)        if invalid_apps:            raise CommandError('No migrations present for: %s' % (', '.join(sorted(invalid_apps))))            try:                apps.get_app_config(app_name)            except LookupError as err:                self.stderr.write(str(err))                has_bad_names = True        if has_bad_names:            sys.exit(2)      def show_list(self, connection, app_names=None):         """


Question: Add app_label validation to showmigrations Description The app label validation was to some commands as above tickets. But showmigrations command doesn't contain it. So I think the validation should be to showmigrations command.

self.stdout.write("[X]  %s.%s%s" % (node.key[0], node.key[1], deps))             else:                 self.stdout.write("[ ]  %s.%s%s" % (node.key[0], node.key[1], deps))<ins>        if not plan:            self.stdout.write('(no migrations)', self.style.ERROR)</ins>


Question: Models history doesn't use verbose names Description The history for a model object (within the admin section) should show human-readable messages, favoring verbose names over field names. However, this is not currently the case. For example, consider a model with the following class variable: pubdate = models.DateTimeField("date published") Changing the publication date for an object of that model, will display "Changed pubdate." in its admin history, rather than "Change date published." as one would expect (as older versions of Django did).

elif 'changed' in sub_message:                     sub_message['changed']['fields'] = get_text_list(                        sub_message['changed']['fields'], gettext('and')                        [gettext(field_name) for field_name in sub_message['changed']['fields']], gettext('and')                     )                     if 'name' in sub_message['changed']:                         sub_message['changed']['name'] = gettext(sub_message['changed']['name'])diff --git a/django/contrib/admin/utils.py b/django/contrib/admin/utils.py-- a/django/contrib/admin/utils.py++ b/django/contrib/admin/utils.py


Question: Models history doesn't use verbose names Description The history for a model object (within the admin section) should show human-readable messages, favoring verbose names over field names. However, this is not currently the case. For example, consider a model with the following class variable: pubdate = models.DateTimeField("date published") Changing the publication date for an object of that model, will display "Changed pubdate." in its admin history, rather than "Change date published." as one would expect (as older versions of Django did).

Translations are deactivated so that strings are stored untranslated.     Translation happens later on LogEntry access.     """    # Evaluating `form.changed_data` prior to disabling translations is required    # to avoid fields affected by localization from being included incorrectly,    # e.g. where date formats differ such as MM/DD/YYYY vs DD/MM/YYYY.    changed_data = form.changed_data    with translation_override(None):        # Deactivate translations while fetching verbose_name for form        # field labels and using `field_name`, if verbose_name is not provided.        # Translations will happen later on LogEntry access.        changed_field_labels = _get_changed_field_labels_from_form(form, changed_data)     change_message = []     if add:         change_message.append({'added': {}})     elif form.changed_data:        change_message.append({'changed': {'fields': form.changed_data}})        change_message.append({'changed': {'fields': changed_field_labels}})     if formsets:         with translation_override(None):             for formset in formsets:


Question: Models history doesn't use verbose names Description The history for a model object (within the admin section) should show human-readable messages, favoring verbose names over field names. However, this is not currently the case. For example, consider a model with the following class variable: pubdate = models.DateTimeField("date published") Changing the publication date for an object of that model, will display "Changed pubdate." in its admin history, rather than "Change date published." as one would expect (as older versions of Django did).

'changed': {                             'name': str(changed_object._meta.verbose_name),                             'object': str(changed_object),                            'fields': changed_fields,                            'fields': _get_changed_field_labels_from_form(formset.forms[0], changed_fields),                         }                     })                 for deleted_object in formset.deleted_objects:


Question: Models history doesn't use verbose names Description The history for a model object (within the admin section) should show human-readable messages, favoring verbose names over field names. However, this is not currently the case. For example, consider a model with the following class variable: pubdate = models.DateTimeField("date published") Changing the publication date for an object of that model, will display "Changed pubdate." in its admin history, rather than "Change date published." as one would expect (as older versions of Django did).

}                     })     return change_message<ins>def _get_changed_field_labels_from_form(form, changed_data):    changed_field_labels = []    for field_name in changed_data:        try:            verbose_field_name = form.fields[field_name].label or field_name        except KeyError:            verbose_field_name = field_name        changed_field_labels.append(str(verbose_field_name))    return changed_field_labels</ins>


Question: Union queryset with ordering breaks on ordering with derived querysets Description (last modified by Sergei Maertens) May be related to Simple reproduction (the exact models are not relevant I think): Dimension.objects.valueslist('id', flat=True) <QuerySet [10, 11, 12, 13, 14, 15, 16, 17, 18]qs = ( (pkin=[10, 11]) .union((pkin=[16, 17]) .orderby('order') ) qs <QuerySet [<Dimension: boeksoort, <Dimension: grootboek, <Dimension: kenteken, <Dimension: activa]qs.orderby().valueslist('pk', flat=True) <QuerySet [16, 11, 10, 17]qs [breaks] Traceback: Traceback (most recent call last): File "<input", line 1, in <moduleqs File "/home/bbt/.virtualenvs/ispnext/lib/python3.6/site-", line 248, in repr data = list(self[:REPROUTPUTSIZE + 1]) File "/home/bbt/.virtualenvs/ispnext/lib/python3.6/site-", line 272, in iter self.fetchall() File "/home/bbt/.virtualenvs/ispnext/lib/python3.6/site-", line 1179, in fetchall self.resultcache = list(self.iterableclass(self)) File "/home/bbt/.virtualenvs/ispnext/lib/python3.6/site-", line 53, in iter results = compiler.executesql(chunkedfetch=self.chunkedfetch, chunksize=self.chunksize) File "/home/bbt/.virtualenvs/ispnext/lib/python3.6/site-", line 1068, in executesql cursor.execute(sql, params) File "/home/bbt/.virtualenvs/ispnext/lib/python3.6/site-", line 100, in execute return super().execute(sql, params) File "/home/bbt/.virtualenvs/ispnext/lib/python3.6/site-", line 68, in execute return self.executewithwrappers(sql, params, many=False, executor=self.execute) File "/home/bbt/.virtualenvs/ispnext/lib/python3.6/site-", line 77, in executewithwrappers return executor(sql, params, many, context) File "/home/bbt/.virtualenvs/ispnext/lib/python3.6/site-", line 85, in execute return (sql, params) File "/home/bbt/.virtualenvs/ispnext/lib/python3.6/site-", line 89, in exit raise djexcvalue.withtraceback(traceback) from excvalue File "/home/bbt/.virtualenvs/ispnext/lib/python3.6/site-", line 85, in execute return (sql, params) ORDER BY position 4 is not in select list LINE 1: ...dimensionsdimension"."id" IN (16, 17)) ORDER BY (4) ASC LIM... ^ Evaluating the qs instead of creating a new qs makes the code proceed as expected. [ for dim in qs]

resolved.set_source_expressions([RawSQL('%d' % (idx + 1), ())])                         break                 else:                    raise DatabaseError('ORDER BY term does not match any column in the result set.')                    if col_alias:                        raise DatabaseError('ORDER BY term does not match any column in the result set.')                    # Add column used in ORDER BY clause without an alias to                    # the selected columns.                    self.query.add_select_col(src)                    resolved.set_source_expressions([RawSQL('%d' % len(self.query.select), ())])             sql, params = self.compile(resolved)             # Don't add the same column twice, but the order direction is             # not taken into account so we strip it. When this entire methoddiff --git a/django/db/models/sql/query.py b/django/db/models/sql/query.py-- a/django/db/models/sql/query.py++ b/django/db/models/sql/query.py


Question: Union queryset with ordering breaks on ordering with derived querysets Description (last modified by Sergei Maertens) May be related to Simple reproduction (the exact models are not relevant I think): Dimension.objects.valueslist('id', flat=True) <QuerySet [10, 11, 12, 13, 14, 15, 16, 17, 18]qs = ( (pkin=[10, 11]) .union((pkin=[16, 17]) .orderby('order') ) qs <QuerySet [<Dimension: boeksoort, <Dimension: grootboek, <Dimension: kenteken, <Dimension: activa]qs.orderby().valueslist('pk', flat=True) <QuerySet [16, 11, 10, 17]qs [breaks] Traceback: Traceback (most recent call last): File "<input", line 1, in <moduleqs File "/home/bbt/.virtualenvs/ispnext/lib/python3.6/site-", line 248, in repr data = list(self[:REPROUTPUTSIZE + 1]) File "/home/bbt/.virtualenvs/ispnext/lib/python3.6/site-", line 272, in iter self.fetchall() File "/home/bbt/.virtualenvs/ispnext/lib/python3.6/site-", line 1179, in fetchall self.resultcache = list(self.iterableclass(self)) File "/home/bbt/.virtualenvs/ispnext/lib/python3.6/site-", line 53, in iter results = compiler.executesql(chunkedfetch=self.chunkedfetch, chunksize=self.chunksize) File "/home/bbt/.virtualenvs/ispnext/lib/python3.6/site-", line 1068, in executesql cursor.execute(sql, params) File "/home/bbt/.virtualenvs/ispnext/lib/python3.6/site-", line 100, in execute return super().execute(sql, params) File "/home/bbt/.virtualenvs/ispnext/lib/python3.6/site-", line 68, in execute return self.executewithwrappers(sql, params, many=False, executor=self.execute) File "/home/bbt/.virtualenvs/ispnext/lib/python3.6/site-", line 77, in executewithwrappers return executor(sql, params, many, context) File "/home/bbt/.virtualenvs/ispnext/lib/python3.6/site-", line 85, in execute return (sql, params) File "/home/bbt/.virtualenvs/ispnext/lib/python3.6/site-", line 89, in exit raise djexcvalue.withtraceback(traceback) from excvalue File "/home/bbt/.virtualenvs/ispnext/lib/python3.6/site-", line 85, in execute return (sql, params) ORDER BY position 4 is not in select list LINE 1: ...dimensionsdimension"."id" IN (16, 17)) ORDER BY (4) ASC LIM... ^ Evaluating the qs instead of creating a new qs makes the code proceed as expected. [ for dim in qs]

self.select = ()         self.values_select = () <ins>    def add_select_col(self, col):        self.select += col,        self.values_select += col.output_field.name,</ins>     def set_select(self, cols):         self.default_cols = False         self.select = tuple(cols)


Question: Using RelatedOnlyFieldListFilter with reverse ManyToMany crashes Description (last modified by Tim Graham) Using RelatedOnlyFieldListFilter with a reverse ManyToMany relation causes this exception: getchoices() got an unexpected keyword argument 'limitchoicesto' This method in ForeignObjectRel.getchoices is missing the parameter that Field.get_choices has. Pull Request:  Demo of how to trigger bug:

self.related_model._meta.model_name,         )     def get_choices(self, include_blank=True, blank_choice=BLANK_CHOICE_DASH, ordering=()):    def get_choices(        self, include_blank=True, blank_choice=BLANK_CHOICE_DASH,        limit_choices_to=None, ordering=(),    ):         """         Return choices with a default blank choices included, for use         as <select> choices for this field.


Question: Using RelatedOnlyFieldListFilter with reverse ManyToMany crashes Description (last modified by Tim Graham) Using RelatedOnlyFieldListFilter with a reverse ManyToMany relation causes this exception: getchoices() got an unexpected keyword argument 'limitchoicesto' This method in ForeignObjectRel.getchoices is missing the parameter that Field.get_choices has. Pull Request:  Demo of how to trigger bug:

Analog of django.db.models.fields.Field.get_choices(), provided         initially for utilization by RelatedFieldListFilter.         """        qs = self.related_model._default_manager.all()        limit_choices_to = limit_choices_to or self.limit_choices_to        qs = self.related_model._default_manager.complex_filter(limit_choices_to)         if ordering:             qs = qs.order_by(*ordering)         return (blank_choice if include_blank else []) + [


Question: Allow icontains lookup to accept uuids with or without dashes Description We have Django 2.1 project with model admin which includes an UUIDField in listdisplay and searchfields. The UUID is displayed with dashes on changelist (e.g. "-6852-47be-82be-") and if the user cut'n'paste it to the search field, I would expect admin to find it. This works however only on Postgres but fails on Oracle. I can understand why this happens (Oracle backend stores uuid as string) and I believe I can workaround it by customizing getsearchresults but I think should be internal thing that Django handles gracefully - search should be possible by the value as displayed in admin.

from django.core.exceptions import EmptyResultSet from django.db.models.expressions import Case, Exists, Func, Value, When from django.db.models.fields import (    BooleanField, DateTimeField, Field, IntegerField,    BooleanField, CharField, DateTimeField, Field, IntegerField, UUIDField, ) from django.db.models.query_utils import RegisterLookupMixin from django.utils.datastructures import OrderedSet


Question: Allow icontains lookup to accept uuids with or without dashes Description We have Django 2.1 project with model admin which includes an UUIDField in listdisplay and searchfields. The UUID is displayed with dashes on changelist (e.g. "-6852-47be-82be-") and if the user cut'n'paste it to the search field, I would expect admin to find it. This works however only on Postgres but fails on Oracle. I can understand why this happens (Oracle backend stores uuid as string) and I believe I can workaround it by customizing getsearchresults but I think should be internal thing that Django handles gracefully - search should be possible by the value as displayed in admin.

class YearLte(YearLookup, LessThanOrEqual):     def get_bound_params(self, start, finish):         return (finish,)<ins>class UUIDTextMixin:    """    Strip hyphens from a value when filtering a UUIDField on backends without    a native datatype for UUID.    """    def process_rhs(self, qn, connection):        if not connection.features.has_native_uuid_field:            from django.db.models.functions import Replace            if self.rhs_is_direct_value():                self.rhs = Value(self.rhs)            self.rhs = Replace(self.rhs, Value('-'), Value(''), output_field=CharField())        rhs, params = super().process_rhs(qn, connection)        return rhs, params@UUIDField.register_lookupclass UUIDIExact(UUIDTextMixin, IExact):    pass@UUIDField.register_lookupclass UUIDContains(UUIDTextMixin, Contains):    pass@UUIDField.register_lookupclass UUIDIContains(UUIDTextMixin, IContains):    pass@UUIDField.register_lookupclass UUIDStartsWith(UUIDTextMixin, StartsWith):    pass@UUIDField.register_lookupclass UUIDIStartsWith(UUIDTextMixin, IStartsWith):    pass@UUIDField.register_lookupclass UUIDEndsWith(UUIDTextMixin, EndsWith):    pass@UUIDField.register_lookupclass UUIDIEndsWith(UUIDTextMixin, IEndsWith):    pass</ins>


Question: Refactor AutoField logic into a mixin, implement checks and validators. Description Currently AutoField inherits from Field and BigAutoField from AutoField. In effect they largely redefine IntegerField and BigIntegerField respectively, but add in the auto field "behaviour". As a result they do not perform some of the system checks, e.g. maxlength warning, nor the validation checks, e.g. range checks, that the integer fields do. The proposal is to move all the auto field "behaviour" into a new AutoFieldMixin and fix AutoField and BigAutoField to inherit from this new mixin and IntegerField and BigIntegerField respectively. Many attributes and methods would be nicely inherited from the correct parent field type without requiring redefinition: description emptystringsallowed defaulterrormessages getprepvalue() topython() AutoField and BigAutoField could also inherit the following checks from IntegerField: IntegerField.checkmaxlengthwarning() AutoField and BigAutoField could also perform minimum and maximum value validation checks inherited from IntegerField. This should be backwards compatible and potentially will make it easier to define new types of auto fields based on other fields in the future.

'BigIntegerField': (-9223372036854775808, 9223372036854775807),         'PositiveSmallIntegerField': (0, 32767),         'PositiveIntegerField': (0, 2147483647),        'SmallAutoField': (-32768, 32767),        'AutoField': (-2147483648, 2147483647),        'BigAutoField': (-9223372036854775808, 9223372036854775807),     }     set_operators = {         'union': 'UNION',diff --git a/django/db/backends/oracle/operations.py b/django/db/backends/oracle/operations.py-- a/django/db/backends/oracle/operations.py++ b/django/db/backends/oracle/operations.py


Question: Refactor AutoField logic into a mixin, implement checks and validators. Description Currently AutoField inherits from Field and BigAutoField from AutoField. In effect they largely redefine IntegerField and BigIntegerField respectively, but add in the auto field "behaviour". As a result they do not perform some of the system checks, e.g. maxlength warning, nor the validation checks, e.g. range checks, that the integer fields do. The proposal is to move all the auto field "behaviour" into a new AutoFieldMixin and fix AutoField and BigAutoField to inherit from this new mixin and IntegerField and BigIntegerField respectively. Many attributes and methods would be nicely inherited from the correct parent field type without requiring redefinition: description emptystringsallowed defaulterrormessages getprepvalue() topython() AutoField and BigAutoField could also inherit the following checks from IntegerField: IntegerField.checkmaxlengthwarning() AutoField and BigAutoField could also perform minimum and maximum value validation checks inherited from IntegerField. This should be backwards compatible and potentially will make it easier to define new types of auto fields based on other fields in the future.

class DatabaseOperations(BaseDatabaseOperations):    # Oracle uses NUMBER(11) and NUMBER(19) for integer fields.    # Oracle uses NUMBER(5), NUMBER(11), and NUMBER(19) for integer fields.    # SmallIntegerField uses NUMBER(11) instead of NUMBER(5), which is used by    # SmallAutoField, to preserve backward compatibility.     integer_field_ranges = {         'SmallIntegerField': (-99999999999, 99999999999),         'IntegerField': (-99999999999, 99999999999),         'BigIntegerField': (-9999999999999999999, 9999999999999999999),         'PositiveSmallIntegerField': (0, 99999999999),         'PositiveIntegerField': (0, 99999999999),        'SmallAutoField': (-99999, 99999),        'AutoField': (-99999999999, 99999999999),        'BigAutoField': (-9999999999999999999, 9999999999999999999),     }     set_operators = {**BaseDatabaseOperations.set_operators, 'difference': 'MINUS'} diff --git a/django/db/models/fields/__init__.py b/django/db/models/fields/__init__.py-- a/django/db/models/fields/__init__.py++ b/django/db/models/fields/__init__.py


Question: Refactor AutoField logic into a mixin, implement checks and validators. Description Currently AutoField inherits from Field and BigAutoField from AutoField. In effect they largely redefine IntegerField and BigIntegerField respectively, but add in the auto field "behaviour". As a result they do not perform some of the system checks, e.g. maxlength warning, nor the validation checks, e.g. range checks, that the integer fields do. The proposal is to move all the auto field "behaviour" into a new AutoFieldMixin and fix AutoField and BigAutoField to inherit from this new mixin and IntegerField and BigIntegerField respectively. Many attributes and methods would be nicely inherited from the correct parent field type without requiring redefinition: description emptystringsallowed defaulterrormessages getprepvalue() topython() AutoField and BigAutoField could also inherit the following checks from IntegerField: IntegerField.checkmaxlengthwarning() AutoField and BigAutoField could also perform minimum and maximum value validation checks inherited from IntegerField. This should be backwards compatible and potentially will make it easier to define new types of auto fields based on other fields in the future.

return getattr(obj, self.attname)  <del>class AutoField(Field):    description = _("Integer")    empty_strings_allowed = False    default_error_messages = {        'invalid': _('“%(value)s” value must be an integer.'),    }    def __init__(self, *args, **kwargs):        kwargs['blank'] = True        super().__init__(*args, **kwargs)    def check(self, **kwargs):        return [            *super().check(**kwargs),            *self._check_primary_key(),        ]    def _check_primary_key(self):        if not self.primary_key:            return [                checks.Error(                    'AutoFields must set primary_key=True.',                    obj=self,                    id='fields.E100',                ),            ]        else:            return []    def deconstruct(self):        name, path, args, kwargs = super().deconstruct()        del kwargs['blank']        kwargs['primary_key'] = True        return name, path, args, kwargs    def get_internal_type(self):        return "AutoField"    def to_python(self, value):        if value is None:            return value        try:            return int(value)        except (TypeError, ValueError):            raise exceptions.ValidationError(                self.error_messages['invalid'],                code='invalid',                params={'value': value},            )    def rel_db_type(self, connection):        return IntegerField().db_type(connection=connection)    def validate(self, value, model_instance):        pass    def get_db_prep_value(self, value, connection, prepared=False):        if not prepared:            value = self.get_prep_value(value)            value = connection.ops.validate_autopk_value(value)        return value    def get_prep_value(self, value):        from django.db.models.expressions import OuterRef        value = super().get_prep_value(value)        if value is None or isinstance(value, OuterRef):            return value        try:            return int(value)        except (TypeError, ValueError) as e:            raise e.__class__(                "Field '%s' expected a number but got %r." % (self.name, value),            ) from e    def contribute_to_class(self, cls, name, **kwargs):        assert not cls._meta.auto_field, "Model %s can't have more than one AutoField." % cls._meta.label        super().contribute_to_class(cls, name, **kwargs)        cls._meta.auto_field = self    def formfield(self, **kwargs):        return Noneclass BigAutoField(AutoField):    description = _("Big (8 byte) integer")    def get_internal_type(self):        return "BigAutoField"    def rel_db_type(self, connection):        return BigIntegerField().db_type(connection=connection)class SmallAutoField(AutoField):    description = _('Small integer')    def get_internal_type(self):        return 'SmallAutoField'    def rel_db_type(self, connection):        return SmallIntegerField().db_type(connection=connection)</del> class BooleanField(Field):     empty_strings_allowed = False     default_error_messages = {


Question: Refactor AutoField logic into a mixin, implement checks and validators. Description Currently AutoField inherits from Field and BigAutoField from AutoField. In effect they largely redefine IntegerField and BigIntegerField respectively, but add in the auto field "behaviour". As a result they do not perform some of the system checks, e.g. maxlength warning, nor the validation checks, e.g. range checks, that the integer fields do. The proposal is to move all the auto field "behaviour" into a new AutoFieldMixin and fix AutoField and BigAutoField to inherit from this new mixin and IntegerField and BigIntegerField respectively. Many attributes and methods would be nicely inherited from the correct parent field type without requiring redefinition: description emptystringsallowed defaulterrormessages getprepvalue() topython() AutoField and BigAutoField could also inherit the following checks from IntegerField: IntegerField.checkmaxlengthwarning() AutoField and BigAutoField could also perform minimum and maximum value validation checks inherited from IntegerField. This should be backwards compatible and potentially will make it easier to define new types of auto fields based on other fields in the future.

'form_class': forms.UUIDField,             **kwargs,         })<ins>class AutoFieldMixin:    def __init__(self, *args, **kwargs):        kwargs['blank'] = True        super().__init__(*args, **kwargs)    def check(self, **kwargs):        return [            *super().check(**kwargs),            *self._check_primary_key(),        ]    def _check_primary_key(self):        if not self.primary_key:            return [                checks.Error(                    'AutoFields must set primary_key=True.',                    obj=self,                    id='fields.E100',                ),            ]        else:            return []    def deconstruct(self):        name, path, args, kwargs = super().deconstruct()        del kwargs['blank']        kwargs['primary_key'] = True        return name, path, args, kwargs    def validate(self, value, model_instance):        pass    def get_db_prep_value(self, value, connection, prepared=False):        if not prepared:            value = self.get_prep_value(value)            value = connection.ops.validate_autopk_value(value)        return value    def get_prep_value(self, value):        from django.db.models.expressions import OuterRef        return value if isinstance(value, OuterRef) else super().get_prep_value(value)    def contribute_to_class(self, cls, name, **kwargs):        assert not cls._meta.auto_field, (            "Model %s can't have more than one auto-generated field."            % cls._meta.label        )        super().contribute_to_class(cls, name, **kwargs)        cls._meta.auto_field = self    def formfield(self, **kwargs):        return Noneclass AutoFieldMeta(type):    """    Metaclass to maintain backward inheritance compatibility for AutoField.    It is intended that AutoFieldMixin become public API when it is possible to    create a non-integer automatically-generated field using column defaults    stored in the database.    In many areas Django also relies on using isinstance() to check for an    automatically-generated field as a subclass of AutoField. A new flag needs    to be implemented on Field to be used instead.    When these issues have been addressed, this metaclass could be used to    deprecate inheritance from AutoField and use of isinstance() with AutoField    for detecting automatically-generated fields.    """    @property    def _subclasses(self):        return (BigAutoField, SmallAutoField)    def __instancecheck__(self, instance):        return isinstance(instance, self._subclasses) or super().__instancecheck__(instance)    def __subclasscheck__(self, subclass):        return subclass in self._subclasses or super().__subclasscheck__(subclass)class AutoField(AutoFieldMixin, IntegerField, metaclass=AutoFieldMeta):    def get_internal_type(self):        return 'AutoField'    def rel_db_type(self, connection):        return IntegerField().db_type(connection=connection)class BigAutoField(AutoFieldMixin, BigIntegerField):    def get_internal_type(self):        return 'BigAutoField'    def rel_db_type(self, connection):        return BigIntegerField().db_type(connection=connection)class SmallAutoField(AutoFieldMixin, SmallIntegerField):    def get_internal_type(self):        return 'SmallAutoField'    def rel_db_type(self, connection):        return SmallIntegerField().db_type(connection=connection)</ins>


Question: Infinite loop in ExceptionReporter.gettracebackframes() Description The following code generates a cause/context cycle ( is excvalue): except WrapperException as exc: raise The while exc_value loop then never terminates.

while exc_value:             exceptions.append(exc_value)             exc_value = explicit_or_implicit_cause(exc_value)<ins>            if exc_value in exceptions:                # Avoid infinite loop if there's a cyclic reference (#29393).                break</ins>          frames = []         # No exceptions were supplied to ExceptionReporter


Question: Raised FieldError's should include the field that caused the error Description Currently there are many places in  that raise a FieldError however frustratingly the field that caused the error is not included. I'm proposing that, where possible, all raised FieldError's raised should include, as part of the error message, the field that caused the error.

'can only be used to update, not to insert.' % (value, field)                 )             if value.contains_aggregate:                raise FieldError("Aggregate functions are not allowed in this query")                raise FieldError(                    'Aggregate functions are not allowed in this query '                    '(%s=%r).' % (field.name, value)                )             if value.contains_over_clause:                raise FieldError('Window expressions are not allowed in this query.')                raise FieldError(                    'Window expressions are not allowed in this query (%s=%r).'                    % (field.name, value)                )         else:             value = field.get_db_prep_save(value, connection=self.connection)         return value


Question: Raised FieldError's should include the field that caused the error Description Currently there are many places in  that raise a FieldError however frustratingly the field that caused the error is not included. I'm proposing that, where possible, all raised FieldError's raised should include, as part of the error message, the field that caused the error.

if hasattr(val, 'resolve_expression'):                 val = val.resolve_expression(self.query, allow_joins=False, for_save=True)                 if val.contains_aggregate:                    raise FieldError("Aggregate functions are not allowed in this query")                    raise FieldError(                        'Aggregate functions are not allowed in this query '                        '(%s=%r).' % (field.name, val)                    )                 if val.contains_over_clause:                    raise FieldError('Window expressions are not allowed in this query.')                    raise FieldError(                        'Window expressions are not allowed in this query '                        '(%s=%r).' % (field.name, val)                    )             elif hasattr(val, 'prepare_database_save'):                 if field.remote_field:                     val = field.get_db_prep_save(


Question: SQLite (pre 3.25.0) does not support window functions, raises OperationalError Description Window functions are supported in SQLite 3.25.0 and newer, but Python 3.6.7 and 3.7.1 only ships with SQLite 3.21.0. Window function syntax is invalid for older versions. As per the title, window functions therefore aren't supported, but Django doesn't check the SQLite version or availability of window functions. Instead, when the generated SQL is executed, the sqlite3 Python library raises the SQLite syntax error as sqlite3.OperationalError, which in turn is reraised as I believe this is not intended behaviour, as it is incredibly confusing, and not documented. Typically, if a database feature is not supported, Django will explicitly raise an error when attempting it, rather than allowing the SQL execution to fail. It is also normally documented. The following code raises an exception (it should work for any model): from import F, Window from import RowNumber MyModel.objects.annotate(rn=Window(expression=RowNumber(), orderby=[F('pk')])) Basic Python code that will also raise sqlite3.OperationalError: import sqlite3 conn = sqlite3.connect(":memory:") c = () c.execute("CREATE TABLE t0(x INTEGER PRIMARY KEY, y TEXT)") c.execute("INSERT INTO t0 VALUES (1, 'aaa'), (2, 'ccc'), (3, 'bbb')") c.execute("SELECT x, y, rownumber() OVER (ORDER BY y) AS row_number FROM t0 ORDER BY x") Tested on master branch (commit ) with Python 3.6.6 64-bit (Windows 10 x64). This likely also affects 2.0 and 2.1 branches.

# Does the backend support window expressions (expression OVER (...))?     supports_over_clause = False    supports_frame_range_fixed_distance = False      # Does the backend support CAST with precision?     supports_cast_with_precision = Truediff --git a/django/db/backends/oracle/features.py b/django/db/backends/oracle/features.py-- a/django/db/backends/oracle/features.py++ b/django/db/backends/oracle/features.py