Question: Wrong behavior on queryset update when multiple inheritance Description Queryset update has a wrong behavior when queryset class inherits multiple classes. The update happens not on child class but on other parents class instances. Here an easy example to show the problem: class Base(models.Model): baseid = models.AutoField(primarykey=True) fieldbase = models.IntegerField() class OtherBase(models.Model): otherbaseid = models.AutoField(primarykey=True) fieldotherbase = models.IntegerField() class Child(Base, OtherBase): pass Then in django shell: In [1]: (fieldotherbase=100) <QuerySet [{'otherbaseid': 1, 'fieldotherbase': 100}]In [2]: (fieldotherbase=101) <QuerySet [{'otherbaseid': 2, 'fieldotherbase': 101}]In [3]: (fieldbase=0, fieldotherbase=0) <Child: Child object (1)In [4]: (fieldbase=1, fieldotherbase=1) <Child: Child object (2)In [5]: (fieldotherbase=55) SELECT "applianceschild"."baseptrid" FROM "applianceschild" Execution time: 0.000647s [Database: default] UPDATE "appliancesotherbase" SET "fieldotherbase" = 55 WHERE "appliancesotherbase"."otherbaseid" IN (1, 2) Execution time: 0.001414s [Database: default] Out[5]: 2 In [6]: ('fieldotherbase') <QuerySet [{'fieldotherbase': 0}, {'fieldotherbase': 1}]In [7]: (otherbaseidin=[1,2]).values('fieldotherbase') <QuerySet [{'fieldotherbase': 55}, {'fieldotherbase': 55}]As seen on the above code, updating Child fields from second parent has no effect. Worse is that OtherBase fields where modifed because query is using primiary keys from Base class.

# don't want them to change), or the db backend doesn't support             # selecting from the updating table (e.g. MySQL).             idents = []            related_ids = collections.defaultdict(list)             for rows in query.get_compiler(self.using).execute_sql(MULTI):                 idents.extend(r[0] for r in rows)                for parent, index in related_ids_index:                    related_ids[parent].extend(r[index] for r in rows)             self.query.add_filter("pk__in", idents)            self.query.related_ids = idents            self.query.related_ids = related_ids         else:             # The fast path. Filters and updates in one query.             self.query.add_filter("pk__in", query)diff --git a/django/db/models/sql/subqueries.py b/django/db/models/sql/subqueries.py-- a/django/db/models/sql/subqueries.py++ b/django/db/models/sql/subqueries.py


Question: Wrong behavior on queryset update when multiple inheritance Description Queryset update has a wrong behavior when queryset class inherits multiple classes. The update happens not on child class but on other parents class instances. Here an easy example to show the problem: class Base(models.Model): baseid = models.AutoField(primarykey=True) fieldbase = models.IntegerField() class OtherBase(models.Model): otherbaseid = models.AutoField(primarykey=True) fieldotherbase = models.IntegerField() class Child(Base, OtherBase): pass Then in django shell: In [1]: (fieldotherbase=100) <QuerySet [{'otherbaseid': 1, 'fieldotherbase': 100}]In [2]: (fieldotherbase=101) <QuerySet [{'otherbaseid': 2, 'fieldotherbase': 101}]In [3]: (fieldbase=0, fieldotherbase=0) <Child: Child object (1)In [4]: (fieldbase=1, fieldotherbase=1) <Child: Child object (2)In [5]: (fieldotherbase=55) SELECT "applianceschild"."baseptrid" FROM "applianceschild" Execution time: 0.000647s [Database: default] UPDATE "appliancesotherbase" SET "fieldotherbase" = 55 WHERE "appliancesotherbase"."otherbaseid" IN (1, 2) Execution time: 0.001414s [Database: default] Out[5]: 2 In [6]: ('fieldotherbase') <QuerySet [{'fieldotherbase': 0}, {'fieldotherbase': 1}]In [7]: (otherbaseidin=[1,2]).values('fieldotherbase') <QuerySet [{'fieldotherbase': 55}, {'fieldotherbase': 55}]As seen on the above code, updating Child fields from second parent has no effect. Worse is that OtherBase fields where modifed because query is using primiary keys from Base class.

query = UpdateQuery(model)             query.values = values             if self.related_ids is not None:                query.add_filter("pk__in", self.related_ids)                query.add_filter("pk__in", self.related_ids[model])             result.append(query)         return result


Question: RegisterLookupMixin.unregisterlookup() should clear the lookup cache. Description (last modified by Himanshu Balasamanta) In current source code, in the unregisterlookup method, , the cache is not cleared, which should be done, as it is done in registerlookup,  Corresponding to this change, minor changes need to be brought in the test. The PR generated is

if lookup_name is None:             lookup_name = lookup.lookup_name         del cls.class_lookups[lookup_name]<ins>        cls._clear_cached_lookups()</ins>   def select_related_descend(field, restricted, requested, load_fields, reverse=False):


Question: Django 3.2.4+ autoreload breaks on empty string in TEMPLATES DIRS. Description Django versions 3.2.3 changes the way template dirs are handled, they are now normalized using pathlib.Path. People having an invalid value in TEMPLATESDIRS? will notice that autoreload stops working. "DIRS": ("TEMPLATESDIRS", "").split(",") # wrong, should be filter(None, ("TEMPLATESDIRS", "").split(",")) or anything else that produces this: "DIRS": [''] # wrong will break autoreload. This happens because was previously comparing the empty string to a directory, and would never match. Now the normalization transforms the empty string into the root of the project. The result is that templatechanged() will now always return True, preventing the autoreload when the app code changes Change that produced the regression Commits in main and stable/3.2.x:   Previous reports

if not isinstance(backend, DjangoTemplates):             continue         items.update(cwd / to_path(dir) for dir in backend.engine.dirs)        items.update(cwd / to_path(dir) for dir in backend.engine.dirs if dir)          for loader in backend.engine.template_loaders:             if not hasattr(loader, "get_dirs"):


Question: Django 3.2.4+ autoreload breaks on empty string in TEMPLATES DIRS. Description Django versions 3.2.3 changes the way template dirs are handled, they are now normalized using pathlib.Path. People having an invalid value in TEMPLATESDIRS? will notice that autoreload stops working. "DIRS": ("TEMPLATESDIRS", "").split(",") # wrong, should be filter(None, ("TEMPLATESDIRS", "").split(",")) or anything else that produces this: "DIRS": [''] # wrong will break autoreload. This happens because was previously comparing the empty string to a directory, and would never match. Now the normalization transforms the empty string into the root of the project. The result is that templatechanged() will now always return True, preventing the autoreload when the app code changes Change that produced the regression Commits in main and stable/3.2.x:   Previous reports

items.update(                 cwd / to_path(directory)                 for directory in loader.get_dirs()                if not is_django_path(directory)                if directory and not is_django_path(directory)             )     return items


Question: clear select fields for of distinct, non-sliced querysets Description The fix () for bug introduced an inefficiency in .exists() on distinct querysets in that now all distinct querysets are selecting all their fields. I propose the fields should be left on the queryset only if (q.distinct and (q.highmark is not None or q.lowmark is not None)). As I started writing the test for this, I realized I would have to inspect the query itself, which seems... like a bad idea for a test. Would this be reasonable to submit the (one-line-change) patch without a test, or does someone have a suggestion for a way to make the test that's less brittle?

def exists(self, using, limit=True):         q = self.clone()        if not q.distinct:        if not (q.distinct and q.is_sliced):             if q.group_by is True:                 q.add_fields(                     (f.attname for f in self.model._meta.concrete_fields), False


Question: Enable cached template loader in development. Description (last modified by Carlton Gibson) Following the changes to allow auto-reloading cached templates in , there was discussion on PR to enable the cached template loader by default even in development. (Folks not wanting that would specify loaders.)

loaders = ["django.template.loaders.filesystem.Loader"]             if app_dirs:                 loaders += ["django.template.loaders.app_directories.Loader"]            if not debug:                loaders = [("django.template.loaders.cached.Loader", loaders)]            loaders = [("django.template.loaders.cached.Loader", loaders)]         else:             if app_dirs:                 raise ImproperlyConfigured(


Question: Unnecessary redirect in LogoutView when ?next=... contains "unsafe" URL Description Reproduction instructions Set LOGOUTREDIRECTURL Wire () at /logout/ in the URLconf Add this form to any template: <form action="/logout/?next=http://evil/" method="POST"<input type="submit" value="Logout"{% csrftoken %}</formLog in, then use the form to log out Expected result You are logged out; the next parameter is ignored; you are redirected to LOGOUTREDIRECTURL Actual result There is an intermediary, useless redirect; see the logs of the development server: [16/Apr/2022 19:05:38] "POST /logout/?next=http://evil/ HTTP/1.1" 302 0 [16/Apr/2022 19:05:38] "GET /logout/ HTTP/1.1" 302 0 [16/Apr/2022 19:05:38] "GET /en/ HTTP/1.1" 200 I noticed this via code inspection. The implementation of LogoutView.getnextpage seemed a bit weird to me. This stems from  which predates the introduction of LOGOUTREDIRECTURL. From the user's perspective, the behavior is correct. There's just an extra round-trip and needlessly complicated code.

# Security check -- Ensure the user-originating redirection URL is             # safe.             if not url_is_safe:                next_page = self.request.path                if settings.LOGOUT_REDIRECT_URL:                    next_page = resolve_url(settings.LOGOUT_REDIRECT_URL)                else:                    next_page = self.request.path         return next_page      def get_context_data(self, **kwargs):


Question: Inconsistent / Unexpected handling of assigning unsaved model to Generic Foreign Key Description addresses the issue of assigned an unsaved model to a ForeignKey or OneToOneField (raises error when save() called), however the same logic / pattern does not apply to GFKs. Given: class ModelA(models.Model): name = models.CharField(maxlength=20) class ModelB(models.Model): gfkctype = models.ForeignKey(ContentType, ondelete=models.PROTECT) gfkid = models.PositiveIntegerField() gfk = GenericForeignKey('gfkctype', 'gfkid') class ModelC(models.Model): fk = models.ForeignKey(ModelA, ondelete=models.CASCADE) Foreign Key Behaviour: In [2]: a = ModelA(name='Model A') In [3]: c = ModelC(fk=a) In [4]: c.fk Out[4]: <ModelA: ModelA object (None)In [5]: c.save() ... ValueError: save() prohibited to prevent data loss due to unsaved related object 'fk'. In [6]: a.save() (0.016) INSERT INTO "testappmodela" ("name") VALUES ('Model A'); args=['Model A'] In [7]: c.fk Out[7]: <ModelA: ModelA object (1)In [8]: c.save() (0.016) INSERT INTO "testappmodelc" ("fkid") VALUES (1); args=[1] GFK behaviour: In [9]: a2 = ModelA(name='Model A2') In [10]: b = ModelB(gfk=a2) In [11]: b.gfk Out[11]: <ModelA: ModelA object (None)In [12]: b.save() (0.000) INSERT INTO "testappmodelb" ("gfkctypeid", "gfkid") VALUES (9, NULL); args=[9, None] IntegrityError: NOT NULL constraint failed: In [14]: () (0.015) INSERT INTO "testappmodela" ("name") VALUES ('Model A2'); args=['Model A2'] In [15]: b.gfk (0.000) SELECT "testappmodela"."id", "testappmodela"."name" FROM "testappmodela" WHERE "testappmodela"."id" IS NULL LIMIT 21; args=() None In [17]: b.gfkctype Out[17]: <ContentType: testapp | model aTwo observations: No check on b.gfk and b.gfkid value during save() which could lead to silent data loss if b.gfkid is nullable. When a2 is saved, accessing b.gfk now does a redundant DB query to try and find ModelA instance with PK = None, then then returns None value (effectively un-assigning a2 model), while keeping b.gfkctype intact. This is because the new pk of a2 is different to the existing gfkid (pkval) of the GFK field (None) What should be done: Modify () or Model.preparerelatedfieldsforsave() to also perform verification check for GFK fields Modify () to handle case of pkval = None (update fkfield value using PK value of GFK model if present, do not perform redundant DB query on pk=None, return previously assigned (then saved) model instead of None)

def _prepare_related_fields_for_save(self, operation_name, fields=None):         # Ensure that a model instance without a PK hasn't been assigned to        # a ForeignKey or OneToOneField on this model. If the field is        # nullable, allowing the save would result in silent data loss.        # a ForeignKey, GenericForeignKey or OneToOneField on this model. If        # the field is nullable, allowing the save would result in silent data        # loss.         for field in self._meta.concrete_fields:             if fields and field not in fields:                 continue


Question: Inconsistent / Unexpected handling of assigning unsaved model to Generic Foreign Key Description addresses the issue of assigned an unsaved model to a ForeignKey or OneToOneField (raises error when save() called), however the same logic / pattern does not apply to GFKs. Given: class ModelA(models.Model): name = models.CharField(maxlength=20) class ModelB(models.Model): gfkctype = models.ForeignKey(ContentType, ondelete=models.PROTECT) gfkid = models.PositiveIntegerField() gfk = GenericForeignKey('gfkctype', 'gfkid') class ModelC(models.Model): fk = models.ForeignKey(ModelA, ondelete=models.CASCADE) Foreign Key Behaviour: In [2]: a = ModelA(name='Model A') In [3]: c = ModelC(fk=a) In [4]: c.fk Out[4]: <ModelA: ModelA object (None)In [5]: c.save() ... ValueError: save() prohibited to prevent data loss due to unsaved related object 'fk'. In [6]: a.save() (0.016) INSERT INTO "testappmodela" ("name") VALUES ('Model A'); args=['Model A'] In [7]: c.fk Out[7]: <ModelA: ModelA object (1)In [8]: c.save() (0.016) INSERT INTO "testappmodelc" ("fkid") VALUES (1); args=[1] GFK behaviour: In [9]: a2 = ModelA(name='Model A2') In [10]: b = ModelB(gfk=a2) In [11]: b.gfk Out[11]: <ModelA: ModelA object (None)In [12]: b.save() (0.000) INSERT INTO "testappmodelb" ("gfkctypeid", "gfkid") VALUES (9, NULL); args=[9, None] IntegrityError: NOT NULL constraint failed: In [14]: () (0.015) INSERT INTO "testappmodela" ("name") VALUES ('Model A2'); args=['Model A2'] In [15]: b.gfk (0.000) SELECT "testappmodela"."id", "testappmodela"."name" FROM "testappmodela" WHERE "testappmodela"."id" IS NULL LIMIT 21; args=() None In [17]: b.gfkctype Out[17]: <ContentType: testapp | model aTwo observations: No check on b.gfk and b.gfkid value during save() which could lead to silent data loss if b.gfkid is nullable. When a2 is saved, accessing b.gfk now does a redundant DB query to try and find ModelA instance with PK = None, then then returns None value (effectively un-assigning a2 model), while keeping b.gfkctype intact. This is because the new pk of a2 is different to the existing gfkid (pkval) of the GFK field (None) What should be done: Modify () or Model.preparerelatedfieldsforsave() to also perform verification check for GFK fields Modify () to handle case of pkval = None (update fkfield value using PK value of GFK model if present, do not perform redundant DB query on pk=None, return previously assigned (then saved) model instead of None)

self, field.attname                 ):                     field.delete_cached_value(self)<ins>        # GenericForeignKeys are private.        for field in self._meta.private_fields:            if fields and field not in fields:                continue            if (                field.is_relation                and field.is_cached(self)                and hasattr(field, "fk_field")            ):                obj = field.get_cached_value(self, default=None)                if obj and obj.pk is None:                    raise ValueError(                        f"{operation_name}() prohibited to prevent data loss due to "                        f"unsaved related object '{field.name}'."                    )</ins>      def delete(self, using=None, keep_parents=False):         if self.pk is None:


Question: Add SQL comment to describe deliberately no-op migration operations Description Currently when a field migration is a no-op, the operation description is output in SQL, but nothing else. This can be confusing as to which operations are no-ops. It could be clearer if we output an extra SQL comment when there are deliberately no statements to execute for a given migration operation. Take for example this output: BEGIN; -- Alter field name on Author ALTER ...; -- Alter field title on Book COMMIT; The field has an operation applied, whilst needs no changes to the database. This isn't exactly clear from the output - is the COMMIT part of the change? It could be clearer as: BEGIN; -- Alter field name on Author ALTER ...; -- Alter field name on Author -- (no-op) COMMIT; (Or perhaps more verbose wording, like "no SQL to execute") I think this can help especially when there are consecutive operations with no-op SQL: BEGIN; -- Alter field name on Author -- (no-op) -- Alter field name on Author -- (no-op) COMMIT; (Inspired by , where the OP suggested dropping such migration operation header comments.)

# there instead             if collect_sql:                 schema_editor.collected_sql.append("--")                if not operation.reduces_to_sql:                    schema_editor.collected_sql.append(                        "-- MIGRATION NOW PERFORMS OPERATION THAT CANNOT BE WRITTEN AS "                        "SQL:"                    )                 schema_editor.collected_sql.append("-- %s" % operation.describe())                 schema_editor.collected_sql.append("--")                 if not operation.reduces_to_sql:                    schema_editor.collected_sql.append(                        "-- THIS OPERATION CANNOT BE WRITTEN AS SQL"                    )                     continue                collected_sql_before = len(schema_editor.collected_sql)             # Save the state before the operation has run             old_state = project_state.clone()             operation.state_forwards(self.app_label, project_state)


Question: Add SQL comment to describe deliberately no-op migration operations Description Currently when a field migration is a no-op, the operation description is output in SQL, but nothing else. This can be confusing as to which operations are no-ops. It could be clearer if we output an extra SQL comment when there are deliberately no statements to execute for a given migration operation. Take for example this output: BEGIN; -- Alter field name on Author ALTER ...; -- Alter field title on Book COMMIT; The field has an operation applied, whilst needs no changes to the database. This isn't exactly clear from the output - is the COMMIT part of the change? It could be clearer as: BEGIN; -- Alter field name on Author ALTER ...; -- Alter field name on Author -- (no-op) COMMIT; (Or perhaps more verbose wording, like "no SQL to execute") I think this can help especially when there are consecutive operations with no-op SQL: BEGIN; -- Alter field name on Author -- (no-op) -- Alter field name on Author -- (no-op) COMMIT; (Inspired by , where the OP suggested dropping such migration operation header comments.)

operation.database_forwards(                     self.app_label, schema_editor, old_state, project_state                 )<ins>            if collect_sql and collected_sql_before == len(schema_editor.collected_sql):                schema_editor.collected_sql.append("-- (no-op)")</ins>         return project_state      def unapply(self, project_state, schema_editor, collect_sql=False):


Question: Add SQL comment to describe deliberately no-op migration operations Description Currently when a field migration is a no-op, the operation description is output in SQL, but nothing else. This can be confusing as to which operations are no-ops. It could be clearer if we output an extra SQL comment when there are deliberately no statements to execute for a given migration operation. Take for example this output: BEGIN; -- Alter field name on Author ALTER ...; -- Alter field title on Book COMMIT; The field has an operation applied, whilst needs no changes to the database. This isn't exactly clear from the output - is the COMMIT part of the change? It could be clearer as: BEGIN; -- Alter field name on Author ALTER ...; -- Alter field name on Author -- (no-op) COMMIT; (Or perhaps more verbose wording, like "no SQL to execute") I think this can help especially when there are consecutive operations with no-op SQL: BEGIN; -- Alter field name on Author -- (no-op) -- Alter field name on Author -- (no-op) COMMIT; (Inspired by , where the OP suggested dropping such migration operation header comments.)

for operation, to_state, from_state in to_run:             if collect_sql:                 schema_editor.collected_sql.append("--")                if not operation.reduces_to_sql:                    schema_editor.collected_sql.append(                        "-- MIGRATION NOW PERFORMS OPERATION THAT CANNOT BE WRITTEN AS "                        "SQL:"                    )                 schema_editor.collected_sql.append("-- %s" % operation.describe())                 schema_editor.collected_sql.append("--")                 if not operation.reduces_to_sql:                    schema_editor.collected_sql.append(                        "-- THIS OPERATION CANNOT BE WRITTEN AS SQL"                    )                     continue                collected_sql_before = len(schema_editor.collected_sql)             atomic_operation = operation.atomic or (                 self.atomic and operation.atomic is not False             )


Question: Add SQL comment to describe deliberately no-op migration operations Description Currently when a field migration is a no-op, the operation description is output in SQL, but nothing else. This can be confusing as to which operations are no-ops. It could be clearer if we output an extra SQL comment when there are deliberately no statements to execute for a given migration operation. Take for example this output: BEGIN; -- Alter field name on Author ALTER ...; -- Alter field title on Book COMMIT; The field has an operation applied, whilst needs no changes to the database. This isn't exactly clear from the output - is the COMMIT part of the change? It could be clearer as: BEGIN; -- Alter field name on Author ALTER ...; -- Alter field name on Author -- (no-op) COMMIT; (Or perhaps more verbose wording, like "no SQL to execute") I think this can help especially when there are consecutive operations with no-op SQL: BEGIN; -- Alter field name on Author -- (no-op) -- Alter field name on Author -- (no-op) COMMIT; (Inspired by , where the OP suggested dropping such migration operation header comments.)

operation.database_backwards(                     self.app_label, schema_editor, from_state, to_state                 )<ins>            if collect_sql and collected_sql_before == len(schema_editor.collected_sql):                schema_editor.collected_sql.append("-- (no-op)")</ins>         return project_state      def suggest_name(self):


Question: Errors with dbcollation – no propagation to foreignkeys Description (last modified by typonaut) Using dbcollation with a pk that also has referenced fks in other models causes foreign key constraint errors in MySQL. With the following models: class Account(models.Model): id = ShortUUIDField(primarykey=True, dbcollation='utf8bin', dbindex=True, maxlength=22) … class Address(models.Model): id = ShortUUIDField(primarykey=True, dbcollation='utf8bin', dbindex=True, maxlength=22) account = models.OneToOneField(Account, ondelete=models.CASCADE) … class Profile(models.Model): id = ShortUUIDField(primarykey=True, dbcollation='utf8bin', dbindex=True, maxlength=22) … account = models.ForeignKey('Account', verbosename=('account'), null=True, blank=True, ondelete=models.CASCADE) … etc Where has been changed from models.BigAutoField if makemigrations is run then it produces sqlmigrate output like this: ALTER TABLE MODIFY varchar(22) COLLATE ; ALTER TABLE MODIFY varchar(22) NOT NULL; ALTER TABLE MODIFY varchar(22) NULL; ALTER TABLE ADD CONSTRAINT FOREIGN KEY () REFERENCES (); ALTER TABLE ADD CONSTRAINT FOREIGN KEY () REFERENCES (); With this SQL the ADD CONSTRAINT queries fail. This is because the COLLATE should also be present in the bmanageaddress.accountid and bmanageprofile.accountid modification statements. Like this: ALTER TABLE MODIFY varchar(22) COLLATE ; ALTER TABLE MODIFY varchar(22) NOT NULL COLLATE ; ALTER TABLE MODIFY varchar(22) NULL COLLATE ; ALTER TABLE ADD CONSTRAINT FOREIGN KEY () REFERENCES (); ALTER TABLE ADD CONSTRAINT FOREIGN KEY () REFERENCES (); In the latter case the ADD CONSTRAINT statements run without error. The collation of the pk must match the collation of the fk otherwise an error will occur.

self.execute(self._delete_unique_sql(model, constraint_name))         # Drop incoming FK constraints if the field is a primary key or unique,         # which might be a to_field target, and things are going to change.        old_collation = old_db_params.get("collation")        new_collation = new_db_params.get("collation")         drop_foreign_keys = (             self.connection.features.supports_foreign_keys             and (                 (old_field.primary_key and new_field.primary_key)                 or (old_field.unique and new_field.unique)             )            and old_type != new_type            and ((old_type != new_type) or (old_collation != new_collation))         )         if drop_foreign_keys:             # '_meta.related_field' also contains M2M reverse fields, these


Question: Errors with dbcollation – no propagation to foreignkeys Description (last modified by typonaut) Using dbcollation with a pk that also has referenced fks in other models causes foreign key constraint errors in MySQL. With the following models: class Account(models.Model): id = ShortUUIDField(primarykey=True, dbcollation='utf8bin', dbindex=True, maxlength=22) … class Address(models.Model): id = ShortUUIDField(primarykey=True, dbcollation='utf8bin', dbindex=True, maxlength=22) account = models.OneToOneField(Account, ondelete=models.CASCADE) … class Profile(models.Model): id = ShortUUIDField(primarykey=True, dbcollation='utf8bin', dbindex=True, maxlength=22) … account = models.ForeignKey('Account', verbosename=('account'), null=True, blank=True, ondelete=models.CASCADE) … etc Where has been changed from models.BigAutoField if makemigrations is run then it produces sqlmigrate output like this: ALTER TABLE MODIFY varchar(22) COLLATE ; ALTER TABLE MODIFY varchar(22) NOT NULL; ALTER TABLE MODIFY varchar(22) NULL; ALTER TABLE ADD CONSTRAINT FOREIGN KEY () REFERENCES (); ALTER TABLE ADD CONSTRAINT FOREIGN KEY () REFERENCES (); With this SQL the ADD CONSTRAINT queries fail. This is because the COLLATE should also be present in the bmanageaddress.accountid and bmanageprofile.accountid modification statements. Like this: ALTER TABLE MODIFY varchar(22) COLLATE ; ALTER TABLE MODIFY varchar(22) NOT NULL COLLATE ; ALTER TABLE MODIFY varchar(22) NULL COLLATE ; ALTER TABLE ADD CONSTRAINT FOREIGN KEY () REFERENCES (); ALTER TABLE ADD CONSTRAINT FOREIGN KEY () REFERENCES (); In the latter case the ADD CONSTRAINT statements run without error. The collation of the pk must match the collation of the fk otherwise an error will occur.

old_type_suffix = old_field.db_type_suffix(connection=self.connection)         new_type_suffix = new_field.db_type_suffix(connection=self.connection)         # Collation change?<del>        old_collation = old_db_params.get("collation")        new_collation = new_db_params.get("collation")</del>         if old_collation != new_collation:             # Collation change handles also a type change.             fragment = self._alter_column_collation_sql(


Question: Errors with dbcollation – no propagation to foreignkeys Description (last modified by typonaut) Using dbcollation with a pk that also has referenced fks in other models causes foreign key constraint errors in MySQL. With the following models: class Account(models.Model): id = ShortUUIDField(primarykey=True, dbcollation='utf8bin', dbindex=True, maxlength=22) … class Address(models.Model): id = ShortUUIDField(primarykey=True, dbcollation='utf8bin', dbindex=True, maxlength=22) account = models.OneToOneField(Account, ondelete=models.CASCADE) … class Profile(models.Model): id = ShortUUIDField(primarykey=True, dbcollation='utf8bin', dbindex=True, maxlength=22) … account = models.ForeignKey('Account', verbosename=('account'), null=True, blank=True, ondelete=models.CASCADE) … etc Where has been changed from models.BigAutoField if makemigrations is run then it produces sqlmigrate output like this: ALTER TABLE MODIFY varchar(22) COLLATE ; ALTER TABLE MODIFY varchar(22) NOT NULL; ALTER TABLE MODIFY varchar(22) NULL; ALTER TABLE ADD CONSTRAINT FOREIGN KEY () REFERENCES (); ALTER TABLE ADD CONSTRAINT FOREIGN KEY () REFERENCES (); With this SQL the ADD CONSTRAINT queries fail. This is because the COLLATE should also be present in the bmanageaddress.accountid and bmanageprofile.accountid modification statements. Like this: ALTER TABLE MODIFY varchar(22) COLLATE ; ALTER TABLE MODIFY varchar(22) NOT NULL COLLATE ; ALTER TABLE MODIFY varchar(22) NULL COLLATE ; ALTER TABLE ADD CONSTRAINT FOREIGN KEY () REFERENCES (); ALTER TABLE ADD CONSTRAINT FOREIGN KEY () REFERENCES (); In the latter case the ADD CONSTRAINT statements run without error. The collation of the pk must match the collation of the fk otherwise an error will occur.

for old_rel, new_rel in rels_to_update:             rel_db_params = new_rel.field.db_parameters(connection=self.connection)             rel_type = rel_db_params["type"]            fragment, other_actions = self._alter_column_type_sql(                new_rel.related_model, old_rel.field, new_rel.field, rel_type            )            rel_collation = rel_db_params.get("collation")            old_rel_db_params = old_rel.field.db_parameters(connection=self.connection)            old_rel_collation = old_rel_db_params.get("collation")            if old_rel_collation != rel_collation:                # Collation change handles also a type change.                fragment = self._alter_column_collation_sql(                    new_rel.related_model,                    new_rel.field,                    rel_type,                    rel_collation,                )                other_actions = []            else:                fragment, other_actions = self._alter_column_type_sql(                    new_rel.related_model, old_rel.field, new_rel.field, rel_type                )             self.execute(                 self.sql_alter_column                 % {diff --git a/django/db/backends/oracle/features.py b/django/db/backends/oracle/features.py-- a/django/db/backends/oracle/features.py++ b/django/db/backends/oracle/features.py


Question: Errors with dbcollation – no propagation to foreignkeys Description (last modified by typonaut) Using dbcollation with a pk that also has referenced fks in other models causes foreign key constraint errors in MySQL. With the following models: class Account(models.Model): id = ShortUUIDField(primarykey=True, dbcollation='utf8bin', dbindex=True, maxlength=22) … class Address(models.Model): id = ShortUUIDField(primarykey=True, dbcollation='utf8bin', dbindex=True, maxlength=22) account = models.OneToOneField(Account, ondelete=models.CASCADE) … class Profile(models.Model): id = ShortUUIDField(primarykey=True, dbcollation='utf8bin', dbindex=True, maxlength=22) … account = models.ForeignKey('Account', verbosename=('account'), null=True, blank=True, ondelete=models.CASCADE) … etc Where has been changed from models.BigAutoField if makemigrations is run then it produces sqlmigrate output like this: ALTER TABLE MODIFY varchar(22) COLLATE ; ALTER TABLE MODIFY varchar(22) NOT NULL; ALTER TABLE MODIFY varchar(22) NULL; ALTER TABLE ADD CONSTRAINT FOREIGN KEY () REFERENCES (); ALTER TABLE ADD CONSTRAINT FOREIGN KEY () REFERENCES (); With this SQL the ADD CONSTRAINT queries fail. This is because the COLLATE should also be present in the bmanageaddress.accountid and bmanageprofile.accountid modification statements. Like this: ALTER TABLE MODIFY varchar(22) COLLATE ; ALTER TABLE MODIFY varchar(22) NOT NULL COLLATE ; ALTER TABLE MODIFY varchar(22) NULL COLLATE ; ALTER TABLE ADD CONSTRAINT FOREIGN KEY () REFERENCES (); ALTER TABLE ADD CONSTRAINT FOREIGN KEY () REFERENCES (); In the latter case the ADD CONSTRAINT statements run without error. The collation of the pk must match the collation of the fk otherwise an error will occur.

"Raises ORA-00600: internal error code.": {             "model_fields.test_jsonfield.TestQuerying.test_usage_in_subquery",         },        "Oracle doesn't support changing collations on indexed columns (#33671).": {            "migrations.test_operations.OperationTests."            "test_alter_field_pk_fk_db_collation",        },     }     django_test_expected_failures = {         # A bug in Django/cx_Oracle with respect to string handling (#23843).diff --git a/django/db/backends/sqlite3/schema.py b/django/db/backends/sqlite3/schema.py-- a/django/db/backends/sqlite3/schema.py++ b/django/db/backends/sqlite3/schema.py


Question: Errors with dbcollation – no propagation to foreignkeys Description (last modified by typonaut) Using dbcollation with a pk that also has referenced fks in other models causes foreign key constraint errors in MySQL. With the following models: class Account(models.Model): id = ShortUUIDField(primarykey=True, dbcollation='utf8bin', dbindex=True, maxlength=22) … class Address(models.Model): id = ShortUUIDField(primarykey=True, dbcollation='utf8bin', dbindex=True, maxlength=22) account = models.OneToOneField(Account, ondelete=models.CASCADE) … class Profile(models.Model): id = ShortUUIDField(primarykey=True, dbcollation='utf8bin', dbindex=True, maxlength=22) … account = models.ForeignKey('Account', verbosename=('account'), null=True, blank=True, ondelete=models.CASCADE) … etc Where has been changed from models.BigAutoField if makemigrations is run then it produces sqlmigrate output like this: ALTER TABLE MODIFY varchar(22) COLLATE ; ALTER TABLE MODIFY varchar(22) NOT NULL; ALTER TABLE MODIFY varchar(22) NULL; ALTER TABLE ADD CONSTRAINT FOREIGN KEY () REFERENCES (); ALTER TABLE ADD CONSTRAINT FOREIGN KEY () REFERENCES (); With this SQL the ADD CONSTRAINT queries fail. This is because the COLLATE should also be present in the bmanageaddress.accountid and bmanageprofile.accountid modification statements. Like this: ALTER TABLE MODIFY varchar(22) COLLATE ; ALTER TABLE MODIFY varchar(22) NOT NULL COLLATE ; ALTER TABLE MODIFY varchar(22) NULL COLLATE ; ALTER TABLE ADD CONSTRAINT FOREIGN KEY () REFERENCES (); ALTER TABLE ADD CONSTRAINT FOREIGN KEY () REFERENCES (); In the latter case the ADD CONSTRAINT statements run without error. The collation of the pk must match the collation of the fk otherwise an error will occur.

# Alter by remaking table         self._remake_table(model, alter_field=(old_field, new_field))         # Rebuild tables with FKs pointing to this field.        if new_field.unique and old_type != new_type:        old_collation = old_db_params.get("collation")        new_collation = new_db_params.get("collation")        if new_field.unique and (            old_type != new_type or old_collation != new_collation        ):             related_models = set()             opts = new_field.model._meta             for remote_field in opts.related_objects:diff --git a/django/db/models/fields/related.py b/django/db/models/fields/related.py-- a/django/db/models/fields/related.py++ b/django/db/models/fields/related.py


Question: Errors with dbcollation – no propagation to foreignkeys Description (last modified by typonaut) Using dbcollation with a pk that also has referenced fks in other models causes foreign key constraint errors in MySQL. With the following models: class Account(models.Model): id = ShortUUIDField(primarykey=True, dbcollation='utf8bin', dbindex=True, maxlength=22) … class Address(models.Model): id = ShortUUIDField(primarykey=True, dbcollation='utf8bin', dbindex=True, maxlength=22) account = models.OneToOneField(Account, ondelete=models.CASCADE) … class Profile(models.Model): id = ShortUUIDField(primarykey=True, dbcollation='utf8bin', dbindex=True, maxlength=22) … account = models.ForeignKey('Account', verbosename=('account'), null=True, blank=True, ondelete=models.CASCADE) … etc Where has been changed from models.BigAutoField if makemigrations is run then it produces sqlmigrate output like this: ALTER TABLE MODIFY varchar(22) COLLATE ; ALTER TABLE MODIFY varchar(22) NOT NULL; ALTER TABLE MODIFY varchar(22) NULL; ALTER TABLE ADD CONSTRAINT FOREIGN KEY () REFERENCES (); ALTER TABLE ADD CONSTRAINT FOREIGN KEY () REFERENCES (); With this SQL the ADD CONSTRAINT queries fail. This is because the COLLATE should also be present in the bmanageaddress.accountid and bmanageprofile.accountid modification statements. Like this: ALTER TABLE MODIFY varchar(22) COLLATE ; ALTER TABLE MODIFY varchar(22) NOT NULL COLLATE ; ALTER TABLE MODIFY varchar(22) NULL COLLATE ; ALTER TABLE ADD CONSTRAINT FOREIGN KEY () REFERENCES (); ALTER TABLE ADD CONSTRAINT FOREIGN KEY () REFERENCES (); In the latter case the ADD CONSTRAINT statements run without error. The collation of the pk must match the collation of the fk otherwise an error will occur.

return self.target_field.rel_db_type(connection=connection)      def db_parameters(self, connection):        return {"type": self.db_type(connection), "check": self.db_check(connection)}        target_db_parameters = self.target_field.db_parameters(connection)        return {            "type": self.db_type(connection),            "check": self.db_check(connection),            "collation": target_db_parameters.get("collation"),        }      def convert_empty_strings(self, value, expression, connection):         if (not value) and isinstance(value, str):


Question: Unnecessary column in a GROUP BY clause with () Description I've got a question about the interaction between exists() and group by. For example something like: ('field').annotate(cnt=Count('id')).filter(cnt_gt=1).exists() The corresp. query with PostgreSQL looks like this: SELECT (1) AS "a" FROM "appmodel" GROUP BY "appmodel"."field", (1) HAVING COUNT("appmodel"."id") 1 LIMIT 1 exists() (see ) clears the SELECT clause and replaces it by (if I understand correctly) a hardcoded value 1 (as "a"), along with a limit of 1, which makes sense to me. But getgroupby() (see ) pushes this hardcoded value to the GROUP BY clause and we end up with the query above. Now, on PostgreSQL, that works, but to me it sounds like it works by luck/lucky robustness... and certainly the same query without the , (1) in the GROUP BY clause yields the same result. The problem is that outside of PostgreSQL that GROUP BY clause can be downright invalid... Note that if I alter exists() to use {'a': 2} instead of {'a': 1}, it does not work anymore (on PostgreSQL), because (2) in the SELECT clause means the hardcoded number 2 while (2) in the GROUP BY clause presumably (??) refers to the 2nd expr of the SELECT clause, but we only got one... My feeling is that getgroupby() should avoid adding extra/raw sql (or at least extra/raw pure values, if we can detect that...) to the group by expressions? NB: the existing/old ticket that seems most related to my question would probably be:

OuterRef,     Ref,     ResolvedOuterRef,<ins>    Value,</ins> ) from django.db.models.fields import Field from django.db.models.fields.related_lookups import MultiColSource


Question: Unnecessary column in a GROUP BY clause with () Description I've got a question about the interaction between exists() and group by. For example something like: ('field').annotate(cnt=Count('id')).filter(cnt_gt=1).exists() The corresp. query with PostgreSQL looks like this: SELECT (1) AS "a" FROM "appmodel" GROUP BY "appmodel"."field", (1) HAVING COUNT("appmodel"."id") 1 LIMIT 1 exists() (see ) clears the SELECT clause and replaces it by (if I understand correctly) a hardcoded value 1 (as "a"), along with a limit of 1, which makes sense to me. But getgroupby() (see ) pushes this hardcoded value to the GROUP BY clause and we end up with the query above. Now, on PostgreSQL, that works, but to me it sounds like it works by luck/lucky robustness... and certainly the same query without the , (1) in the GROUP BY clause yields the same result. The problem is that outside of PostgreSQL that GROUP BY clause can be downright invalid... Note that if I alter exists() to use {'a': 2} instead of {'a': 1}, it does not work anymore (on PostgreSQL), because (2) in the SELECT clause means the hardcoded number 2 while (2) in the GROUP BY clause presumably (??) refers to the 2nd expr of the SELECT clause, but we only got one... My feeling is that getgroupby() should avoid adding extra/raw sql (or at least extra/raw pure values, if we can detect that...) to the group by expressions? NB: the existing/old ticket that seems most related to my question would probably be:

q.clear_ordering(force=True)         if limit:             q.set_limits(high=1)        q.add_extra({"a": 1}, None, None, None, None, None)        q.set_extra_mask(["a"])        q.add_annotation(Value(1), "a")         return q      def has_results(self, using):


Question: Migrations changes to implicit primary key when primary key is altered on SQLite. Description Migrations drops explicit primary key defined on a model when it's altered.

alter_field and getattr(alter_field[1], "primary_key", False)         ):             for name, field in list(body.items()):                if field.primary_key:                if field.primary_key and not (                    # Do not remove the old primary key when an altered field                    # that introduces a primary key is the same field.                    alter_field                    and name == alter_field[1].name                ):                     field.primary_key = False                     restore_pk_field = field                     if field.auto_created:


Question: views.Feed methods cannot be decorated Description If one applies a decorator on a method which is called by getdynamicattr a TypeError like this occurs: Exception Type: TypeError at /blog/feed/ Exception Value: itemlink() takes exactly 2 arguments (1 given) I think this is because getdynamicattr tries to count the function's arguments, but decorators usally get defined with the args, *kwargs syntax, so this trick does not work here. if code.coargcount == 2: # one argument is 'self' return attr(obj) else: return attr() I think the best approach would be to remove one of the two methods. IMHO We should have either attr(item) or attr() not both, as "there should be one, and preferably only one, obvious way to do it".

<ins>from inspect import getattr_static, unwrap</ins> from django.contrib.sites.shortcuts import get_current_site from django.core.exceptions import ImproperlyConfigured, ObjectDoesNotExist from django.http import Http404, HttpResponse


Question: views.Feed methods cannot be decorated Description If one applies a decorator on a method which is called by getdynamicattr a TypeError like this occurs: Exception Type: TypeError at /blog/feed/ Exception Value: itemlink() takes exactly 2 arguments (1 given) I think this is because getdynamicattr tries to count the function's arguments, but decorators usally get defined with the args, *kwargs syntax, so this trick does not work here. if code.coargcount == 2: # one argument is 'self' return attr(obj) else: return attr() I think the best approach would be to remove one of the two methods. IMHO We should have either attr(item) or attr() not both, as "there should be one, and preferably only one, obvious way to do it".

# Check co_argcount rather than try/excepting the function and             # catching the TypeError, because something inside the function             # may raise the TypeError. This technique is more accurate.            func = unwrap(attr)             try:                code = attr.__code__                code = func.__code__             except AttributeError:                code = attr.__call__.__code__                func = unwrap(attr.__call__)                code = func.__code__            # If function doesn't have arguments and it is not a static method,            # it was decorated without using @functools.wraps.            if not code.co_argcount and not isinstance(                getattr_static(self, func.__name__, None), staticmethod            ):                raise ImproperlyConfigured(                    f"Feed method {attname!r} decorated by {func.__name__!r} needs to "                    f"use @functools.wraps."                )             if code.co_argcount == 2:  # one argument is 'self'                 return attr(obj)             else:


Question: Implement RenameIndex in a backwards compatible way Description In order to eventually deprecate indextogether we need a way to deal with old projects that have unnamed indexes. This proves to be a non-trivial problem. Andrew and I came up with these things to consider. RenameIndex(model, newname, oldname=None, oldfields=None) where exactly one of oldname and oldfield is given (oldname ^ oldfields) If the oldname is given we use RENAME INDEX if available Otherwise look at the state and drop existing indexes and create new the index with new name On MySQL (or other DBs) that don't support RENAME INDEX, provide SQL query to look up index name from informationschema by field names and pass in to DROP INDEX. If more than one index is found while identifying with field names, migrations must error out with an AmbiguityError If the autodetector finds an old, unnamed index and a new, named one matching field signature, issue a RenameIndex operation For backwards operations with unnamed old indexes, RenameIndex is a noop.

import functools import re<ins>from collections import defaultdict</ins> from itertools import chain  from django.conf import settings


Question: Implement RenameIndex in a backwards compatible way Description In order to eventually deprecate indextogether we need a way to deal with old projects that have unnamed indexes. This proves to be a non-trivial problem. Andrew and I came up with these things to consider. RenameIndex(model, newname, oldname=None, oldfields=None) where exactly one of oldname and oldfield is given (oldname ^ oldfields) If the oldname is given we use RENAME INDEX if available Otherwise look at the state and drop existing indexes and create new the index with new name On MySQL (or other DBs) that don't support RENAME INDEX, provide SQL query to look up index name from informationschema by field names and pass in to DROP INDEX. If more than one index is found while identifying with field names, migrations must error out with an AmbiguityError If the autodetector finds an old, unnamed index and a new, named one matching field signature, issue a RenameIndex operation For backwards operations with unnamed old indexes, RenameIndex is a noop.

self.generated_operations = {}         self.altered_indexes = {}         self.altered_constraints = {}<ins>        self.renamed_fields = {}</ins>          # Prepare some old/new state and model lists, separating         # proxy models and ignoring unmigrated apps.


Question: Implement RenameIndex in a backwards compatible way Description In order to eventually deprecate indextogether we need a way to deal with old projects that have unnamed indexes. This proves to be a non-trivial problem. Andrew and I came up with these things to consider. RenameIndex(model, newname, oldname=None, oldfields=None) where exactly one of oldname and oldfield is given (oldname ^ oldfields) If the oldname is given we use RENAME INDEX if available Otherwise look at the state and drop existing indexes and create new the index with new name On MySQL (or other DBs) that don't support RENAME INDEX, provide SQL query to look up index name from informationschema by field names and pass in to DROP INDEX. If more than one index is found while identifying with field names, migrations must error out with an AmbiguityError If the autodetector finds an old, unnamed index and a new, named one matching field signature, issue a RenameIndex operation For backwards operations with unnamed old indexes, RenameIndex is a noop.

self.generate_altered_options()         self.generate_altered_managers() <ins>        # Create the renamed fields and store them in self.renamed_fields.        # They are used by create_altered_indexes(), generate_altered_fields(),        # generate_removed_altered_index/unique_together(), and        # generate_altered_index/unique_together().        self.create_renamed_fields()</ins>         # Create the altered indexes and store them in self.altered_indexes.         # This avoids the same computation in generate_removed_indexes()         # and generate_added_indexes().


Question: Implement RenameIndex in a backwards compatible way Description In order to eventually deprecate indextogether we need a way to deal with old projects that have unnamed indexes. This proves to be a non-trivial problem. Andrew and I came up with these things to consider. RenameIndex(model, newname, oldname=None, oldfields=None) where exactly one of oldname and oldfield is given (oldname ^ oldfields) If the oldname is given we use RENAME INDEX if available Otherwise look at the state and drop existing indexes and create new the index with new name On MySQL (or other DBs) that don't support RENAME INDEX, provide SQL query to look up index name from informationschema by field names and pass in to DROP INDEX. If more than one index is found while identifying with field names, migrations must error out with an AmbiguityError If the autodetector finds an old, unnamed index and a new, named one matching field signature, issue a RenameIndex operation For backwards operations with unnamed old indexes, RenameIndex is a noop.

),             )     def generate_renamed_fields(self):    def create_renamed_fields(self):         """Work out renamed fields."""        self.renamed_fields = {}        self.renamed_operations = []        old_field_keys = self.old_field_keys.copy()         for app_label, model_name, field_name in sorted(            self.new_field_keys - self.old_field_keys            self.new_field_keys - old_field_keys         ):             old_model_name = self.renamed_models.get(                 (app_label, model_name), model_name


Question: Implement RenameIndex in a backwards compatible way Description In order to eventually deprecate indextogether we need a way to deal with old projects that have unnamed indexes. This proves to be a non-trivial problem. Andrew and I came up with these things to consider. RenameIndex(model, newname, oldname=None, oldfields=None) where exactly one of oldname and oldfield is given (oldname ^ oldfields) If the oldname is given we use RENAME INDEX if available Otherwise look at the state and drop existing indexes and create new the index with new name On MySQL (or other DBs) that don't support RENAME INDEX, provide SQL query to look up index name from informationschema by field names and pass in to DROP INDEX. If more than one index is found while identifying with field names, migrations must error out with an AmbiguityError If the autodetector finds an old, unnamed index and a new, named one matching field signature, issue a RenameIndex operation For backwards operations with unnamed old indexes, RenameIndex is a noop.

# Scan to see if this is actually a rename!             field_dec = self.deep_deconstruct(field)             for rem_app_label, rem_model_name, rem_field_name in sorted(                self.old_field_keys - self.new_field_keys                old_field_keys - self.new_field_keys             ):                 if rem_app_label == app_label and rem_model_name == model_name:                     old_field = old_model_state.get_field(rem_field_name)


Question: Implement RenameIndex in a backwards compatible way Description In order to eventually deprecate indextogether we need a way to deal with old projects that have unnamed indexes. This proves to be a non-trivial problem. Andrew and I came up with these things to consider. RenameIndex(model, newname, oldname=None, oldfields=None) where exactly one of oldname and oldfield is given (oldname ^ oldfields) If the oldname is given we use RENAME INDEX if available Otherwise look at the state and drop existing indexes and create new the index with new name On MySQL (or other DBs) that don't support RENAME INDEX, provide SQL query to look up index name from informationschema by field names and pass in to DROP INDEX. If more than one index is found while identifying with field names, migrations must error out with an AmbiguityError If the autodetector finds an old, unnamed index and a new, named one matching field signature, issue a RenameIndex operation For backwards operations with unnamed old indexes, RenameIndex is a noop.

if self.questioner.ask_rename(                             model_name, rem_field_name, field_name, field                         ):                            # A db_column mismatch requires a prior noop                            # AlterField for the subsequent RenameField to be a                            # noop on attempts at preserving the old name.                            if old_field.db_column != field.db_column:                                altered_field = field.clone()                                altered_field.name = rem_field_name                                self.add_operation(                            self.renamed_operations.append(                                (                                    rem_app_label,                                    rem_model_name,                                    old_field.db_column,                                    rem_field_name,                                     app_label,                                    operations.AlterField(                                        model_name=model_name,                                        name=rem_field_name,                                        field=altered_field,                                    ),                                    model_name,                                    field,                                    field_name,                                 )                            self.add_operation(                                app_label,                                operations.RenameField(                                    model_name=model_name,                                    old_name=rem_field_name,                                    new_name=field_name,                                ),                             )                            self.old_field_keys.remove(                            old_field_keys.remove(                                 (rem_app_label, rem_model_name, rem_field_name)                             )                            self.old_field_keys.add((app_label, model_name, field_name))                            old_field_keys.add((app_label, model_name, field_name))                             self.renamed_fields[                                 app_label, model_name, field_name                             ] = rem_field_name                             break     def generate_renamed_fields(self):        """Generate RenameField operations."""        for (            rem_app_label,            rem_model_name,            rem_db_column,            rem_field_name,            app_label,            model_name,            field,            field_name,        ) in self.renamed_operations:            # A db_column mismatch requires a prior noop AlterField for the            # subsequent RenameField to be a noop on attempts at preserving the            # old name.            if rem_db_column != field.db_column:                altered_field = field.clone()                altered_field.name = rem_field_name                self.add_operation(                    app_label,                    operations.AlterField(                        model_name=model_name,                        name=rem_field_name,                        field=altered_field,                    ),                )            self.add_operation(                app_label,                operations.RenameField(                    model_name=model_name,                    old_name=rem_field_name,                    new_name=field_name,                ),            )            self.old_field_keys.remove((rem_app_label, rem_model_name, rem_field_name))            self.old_field_keys.add((app_label, model_name, field_name))     def generate_added_fields(self):         """Make AddField operations."""         for app_label, model_name, field_name in sorted(


Question: Implement RenameIndex in a backwards compatible way Description In order to eventually deprecate indextogether we need a way to deal with old projects that have unnamed indexes. This proves to be a non-trivial problem. Andrew and I came up with these things to consider. RenameIndex(model, newname, oldname=None, oldfields=None) where exactly one of oldname and oldfield is given (oldname ^ oldfields) If the oldname is given we use RENAME INDEX if available Otherwise look at the state and drop existing indexes and create new the index with new name On MySQL (or other DBs) that don't support RENAME INDEX, provide SQL query to look up index name from informationschema by field names and pass in to DROP INDEX. If more than one index is found while identifying with field names, migrations must error out with an AmbiguityError If the autodetector finds an old, unnamed index and a new, named one matching field signature, issue a RenameIndex operation For backwards operations with unnamed old indexes, RenameIndex is a noop.

def create_altered_indexes(self):         option_name = operations.AddIndex.option_name<ins>        self.renamed_index_together_values = defaultdict(list)</ins>         for app_label, model_name in sorted(self.kept_model_keys):             old_model_name = self.renamed_models.get(                 (app_label, model_name), model_name


Question: Implement RenameIndex in a backwards compatible way Description In order to eventually deprecate indextogether we need a way to deal with old projects that have unnamed indexes. This proves to be a non-trivial problem. Andrew and I came up with these things to consider. RenameIndex(model, newname, oldname=None, oldfields=None) where exactly one of oldname and oldfield is given (oldname ^ oldfields) If the oldname is given we use RENAME INDEX if available Otherwise look at the state and drop existing indexes and create new the index with new name On MySQL (or other DBs) that don't support RENAME INDEX, provide SQL query to look up index name from informationschema by field names and pass in to DROP INDEX. If more than one index is found while identifying with field names, migrations must error out with an AmbiguityError If the autodetector finds an old, unnamed index and a new, named one matching field signature, issue a RenameIndex operation For backwards operations with unnamed old indexes, RenameIndex is a noop.

renamed_indexes.append((old_index_name, new_index_name, None))                         remove_from_added.append(new_index)                         remove_from_removed.append(old_index)<ins>            # Find index_together changed to indexes.            for (                old_value,                new_value,                index_together_app_label,                index_together_model_name,                dependencies,            ) in self._get_altered_foo_together_operations(                operations.AlterIndexTogether.option_name            ):                if (                    app_label != index_together_app_label                    or model_name != index_together_model_name                ):                    continue                removed_values = old_value.difference(new_value)                for removed_index_together in removed_values:                    renamed_index_together_indexes = []                    for new_index in added_indexes:                        _, args, kwargs = new_index.deconstruct()                        # Ensure only 'fields' are defined in the Index.                        if (                            not args                            and new_index.fields == list(removed_index_together)                            and set(kwargs) == {"name", "fields"}                        ):                            renamed_index_together_indexes.append(new_index)                    if len(renamed_index_together_indexes) == 1:                        renamed_index = renamed_index_together_indexes[0]                        remove_from_added.append(renamed_index)                        renamed_indexes.append(                            (None, renamed_index.name, removed_index_together)                        )                        self.renamed_index_together_values[                            index_together_app_label, index_together_model_name                        ].append(removed_index_together)</ins>             # Remove renamed indexes from the lists of added and removed             # indexes.             added_indexes = [


Question: Implement RenameIndex in a backwards compatible way Description In order to eventually deprecate indextogether we need a way to deal with old projects that have unnamed indexes. This proves to be a non-trivial problem. Andrew and I came up with these things to consider. RenameIndex(model, newname, oldname=None, oldfields=None) where exactly one of oldname and oldfield is given (oldname ^ oldfields) If the oldname is given we use RENAME INDEX if available Otherwise look at the state and drop existing indexes and create new the index with new name On MySQL (or other DBs) that don't support RENAME INDEX, provide SQL query to look up index name from informationschema by field names and pass in to DROP INDEX. If more than one index is found while identifying with field names, migrations must error out with an AmbiguityError If the autodetector finds an old, unnamed index and a new, named one matching field signature, issue a RenameIndex operation For backwards operations with unnamed old indexes, RenameIndex is a noop.

model_name,             dependencies,         ) in self._get_altered_foo_together_operations(operation.option_name):<ins>            if operation == operations.AlterIndexTogether:                old_value = {                    value                    for value in old_value                    if value                    not in self.renamed_index_together_values[app_label, model_name]                }</ins>             removal_value = new_value.intersection(old_value)             if removal_value or old_value:                 self.add_operation(


Question: Query Expression in ordering of a related object fails Description (last modified by Tim Graham) Since 2.0, according to the doc (), we can use QueryExpression objects in the Model.Meta.ordering field. Using: from import models class Musician(models.Model): firstname = models.CharField(maxlength=50) lastname = models.CharField(maxlength=50) instrument = models.CharField(maxlength=100, null=True, blank=True) class Meta: ordering = [models.F('instrument').asc(nullslast=True)] class Album(models.Model): artist = models.ForeignKey(Musician, ondelete=models.CASCADE) name = models.CharField(maxlength=100) releasedate = models.DateField() numstars = models.IntegerField() class Meta: ordering = ['artist'] () ... TypeError: 'OrderBy' does not support indexing When reaching , the compiler tries to use the related model, but at line 679, item can be an OrderBy object. Thus the failure.

def copy(self):         return copy.copy(self)     def prefix_references(self, prefix):        clone = self.copy()        clone.set_source_expressions(            [                F(f"{prefix}{expr.name}")                if isinstance(expr, F)                else expr.prefix_references(prefix)                for expr in self.get_source_expressions()            ]        )        return clone     def get_group_by_cols(self, alias=None):         if not self.contains_aggregate:             return [self]diff --git a/django/db/models/sql/compiler.py b/django/db/models/sql/compiler.py-- a/django/db/models/sql/compiler.py++ b/django/db/models/sql/compiler.py


Question: Query Expression in ordering of a related object fails Description (last modified by Tim Graham) Since 2.0, according to the doc (), we can use QueryExpression objects in the Model.Meta.ordering field. Using: from import models class Musician(models.Model): firstname = models.CharField(maxlength=50) lastname = models.CharField(maxlength=50) instrument = models.CharField(maxlength=100, null=True, blank=True) class Meta: ordering = [models.F('instrument').asc(nullslast=True)] class Album(models.Model): artist = models.ForeignKey(Musician, ondelete=models.CASCADE) name = models.CharField(maxlength=100) releasedate = models.DateField() numstars = models.IntegerField() class Meta: ordering = ['artist'] () ... TypeError: 'OrderBy' does not support indexing When reaching , the compiler tries to use the related model, but at line 679, item can be an OrderBy object. Thus the failure.

):                     item = item.desc() if descending else item.asc()                 if isinstance(item, OrderBy):                    results.append((item, False))                    results.append(                        (item.prefix_references(f"{name}{LOOKUP_SEP}"), False)                    )                     continue                 results.extend(                    self.find_ordering_name(item, opts, alias, order, already_seen)                    (expr.prefix_references(f"{name}{LOOKUP_SEP}"), is_ref)                    for expr, is_ref in self.find_ordering_name(                        item, opts, alias, order, already_seen                    )                 )             return results         targets, alias, _ = self.query.trim_joins(targets, joins, path)


Question: Create --update flag for makemigrations management command, mimicking South's one. Description When I was developing with South I was able to use the --update flag for the schemamigration management command () to refine the latest migration. This was very convenient for iterative development. Could we have an equivalent of the --update flag for Django=1.7 makemigrations? (I've taken it from )

from django.db.migrations import Migration from django.db.migrations.autodetector import MigrationAutodetector from django.db.migrations.loader import MigrationLoader<ins>from django.db.migrations.migration import SwappableTuplefrom django.db.migrations.optimizer import MigrationOptimizer</ins> from django.db.migrations.questioner import (     InteractiveMigrationQuestioner,     MigrationQuestioner,


Question: Create --update flag for makemigrations management command, mimicking South's one. Description When I was developing with South I was able to use the --update flag for the schemamigration management command () to refine the latest migration. This was very convenient for iterative development. Could we have an equivalent of the --update flag for Django=1.7 makemigrations? (I've taken it from )

"paths of generated migration files to stdout."             ),         )<ins>        parser.add_argument(            "--update",            action="store_true",            dest="update",            help=(                "Merge model changes into the latest migration and optimize the "                "resulting operations."            ),        )</ins>      @property     def log_output(self):


Question: Create --update flag for makemigrations management command, mimicking South's one. Description When I was developing with South I was able to use the --update flag for the schemamigration management command () to refine the latest migration. This was very convenient for iterative development. Could we have an equivalent of the --update flag for Django=1.7 makemigrations? (I've taken it from )

self.include_header = options["include_header"]         check_changes = options["check_changes"]         self.scriptable = options["scriptable"]<ins>        self.update = options["update"]</ins>         # If logs and prompts are diverted to stderr, remove the ERROR style.         if self.scriptable:             self.stderr.style_func = None


Question: Create --update flag for makemigrations management command, mimicking South's one. Description When I was developing with South I was able to use the --update flag for the schemamigration management command () to refine the latest migration. This was very convenient for iterative development. Could we have an equivalent of the --update flag for Django=1.7 makemigrations? (I've taken it from )

else:                     self.log("No changes detected")         else:            self.write_migration_files(changes)            if self.update:                self.write_to_last_migration_files(changes)            else:                self.write_migration_files(changes)             if check_changes:                 sys.exit(1)     def write_migration_files(self, changes):    def write_to_last_migration_files(self, changes):        loader = MigrationLoader(connections[DEFAULT_DB_ALIAS])        new_changes = {}        update_previous_migration_paths = {}        for app_label, app_migrations in changes.items():            # Find last migration.            leaf_migration_nodes = loader.graph.leaf_nodes(app=app_label)            if len(leaf_migration_nodes) == 0:                raise CommandError(                    f"App {app_label} has no migration, cannot update last migration."                )            leaf_migration_node = leaf_migration_nodes[0]            # Multiple leaf nodes have already been checked earlier in command.            leaf_migration = loader.graph.nodes[leaf_migration_node]            # Updated migration cannot be a squash migration, a dependency of            # another migration, and cannot be already applied.            if leaf_migration.replaces:                raise CommandError(                    f"Cannot update squash migration '{leaf_migration}'."                )            if leaf_migration_node in loader.applied_migrations:                raise CommandError(                    f"Cannot update applied migration '{leaf_migration}'."                )            depending_migrations = [                migration                for migration in loader.disk_migrations.values()                if leaf_migration_node in migration.dependencies            ]            if depending_migrations:                formatted_migrations = ", ".join(                    [f"'{migration}'" for migration in depending_migrations]                )                raise CommandError(                    f"Cannot update migration '{leaf_migration}' that migrations "                    f"{formatted_migrations} depend on."                )            # Build new migration.            for migration in app_migrations:                leaf_migration.operations.extend(migration.operations)                for dependency in migration.dependencies:                    if isinstance(dependency, SwappableTuple):                        if settings.AUTH_USER_MODEL == dependency.setting:                            leaf_migration.dependencies.append(                                ("__setting__", "AUTH_USER_MODEL")                            )                        else:                            leaf_migration.dependencies.append(dependency)                    elif dependency[0] != migration.app_label:                        leaf_migration.dependencies.append(dependency)            # Optimize migration.            optimizer = MigrationOptimizer()            leaf_migration.operations = optimizer.optimize(                leaf_migration.operations, app_label            )            # Update name.            previous_migration_path = MigrationWriter(leaf_migration).path            suggested_name = (                leaf_migration.name[:4] + "_" + leaf_migration.suggest_name()            )            if leaf_migration.name == suggested_name:                new_name = leaf_migration.name + "_updated"            else:                new_name = suggested_name            leaf_migration.name = new_name            # Register overridden migration.            new_changes[app_label] = [leaf_migration]            update_previous_migration_paths[app_label] = previous_migration_path        self.write_migration_files(new_changes, update_previous_migration_paths)    def write_migration_files(self, changes, update_previous_migration_paths=None):         """         Take a changes dict and write them out as migration files.         """


Question: Create --update flag for makemigrations management command, mimicking South's one. Description When I was developing with South I was able to use the --update flag for the schemamigration management command () to refine the latest migration. This was very convenient for iterative development. Could we have an equivalent of the --update flag for Django=1.7 makemigrations? (I've taken it from )

if self.verbosity >= 1:                     # Display a relative path if it's below the current working                     # directory, or an absolute path otherwise.                    try:                        migration_string = os.path.relpath(writer.path)                    except ValueError:                        migration_string = writer.path                    if migration_string.startswith(".."):                        migration_string = writer.path                    migration_string = self.get_relative_path(writer.path)                     self.log("  %s\n" % self.style.MIGRATE_LABEL(migration_string))                     for operation in migration.operations:                         self.log("    - %s" % operation.describe())


Question: Create --update flag for makemigrations management command, mimicking South's one. Description When I was developing with South I was able to use the --update flag for the schemamigration management command () to refine the latest migration. This was very convenient for iterative development. Could we have an equivalent of the --update flag for Django=1.7 makemigrations? (I've taken it from )

with open(writer.path, "w", encoding="utf-8") as fh:                         fh.write(migration_string)                         self.written_files.append(writer.path)<ins>                    if update_previous_migration_paths:                        prev_path = update_previous_migration_paths[app_label]                        rel_prev_path = self.get_relative_path(prev_path)                        if writer.needs_manual_porting:                            migration_path = self.get_relative_path(writer.path)                            self.log(                                self.style.WARNING(                                    f"Updated migration {migration_path} requires "                                    f"manual porting.\n"                                    f"Previous migration {rel_prev_path} was kept and "                                    f"must be deleted after porting functions manually."                                )                            )                        else:                            os.remove(prev_path)                            self.log(f"Deleted {rel_prev_path}")</ins>                 elif self.verbosity == 3:                     # Alternatively, makemigrations --dry-run --verbosity 3                     # will log the migrations rather than saving the file to


Question: Create --update flag for makemigrations management command, mimicking South's one. Description When I was developing with South I was able to use the --update flag for the schemamigration management command () to refine the latest migration. This was very convenient for iterative development. Could we have an equivalent of the --update flag for Django=1.7 makemigrations? (I've taken it from )

self.log(writer.as_string())         run_formatters(self.written_files) <ins>    @staticmethod    def get_relative_path(path):        try:            migration_string = os.path.relpath(path)        except ValueError:            migration_string = path        if migration_string.startswith(".."):            migration_string = path        return migration_string</ins>     def handle_merge(self, loader, conflicts):         """         Handles merging together conflicted migrations interactively,


Question: Allow overriding FormSet's error messages for too few and too many forms. Description Moving the messages to defaulterrormessages enables overriding them via the errormessages argument when instantiating the FormSet. I would prefer being able to override them via inlineformsetfactory, but that is for another day.

from django.forms.widgets import CheckboxInput, HiddenInput, NumberInput from django.utils.functional import cached_property from django.utils.translation import gettext_lazy as _from django.utils.translation import ngettextfrom django.utils.translation import ngettext_lazy  __all__ = ("BaseFormSet", "formset_factory", "all_valid")


Question: Allow overriding FormSet's error messages for too few and too many forms. Description Moving the messages to defaulterrormessages enables overriding them via the errormessages argument when instantiating the FormSet. I would prefer being able to override them via inlineformsetfactory, but that is for another day.

"ManagementForm data is missing or has been tampered with. Missing fields: "             "%(field_names)s. You may need to file a bug report if the issue persists."         ),<ins>        "too_many_forms": ngettext_lazy(            "Please submit at most %(num)d form.",            "Please submit at most %(num)d forms.",            "num",        ),        "too_few_forms": ngettext_lazy(            "Please submit at least %(num)d form.",            "Please submit at least %(num)d forms.",            "num",        ),</ins>     }      template_name_div = "django/forms/formsets/div.html"


Question: Allow overriding FormSet's error messages for too few and too many forms. Description Moving the messages to defaulterrormessages enables overriding them via the errormessages argument when instantiating the FormSet. I would prefer being able to override them via inlineformsetfactory, but that is for another day.

TOTAL_FORM_COUNT             ] > self.absolute_max:                 raise ValidationError(                    ngettext(                        "Please submit at most %d form.",                        "Please submit at most %d forms.",                        self.max_num,                    )                    % self.max_num,                    self.error_messages["too_many_forms"] % {"num": self.max_num},                     code="too_many_forms",                 )             if (


Question: Allow overriding FormSet's error messages for too few and too many forms. Description Moving the messages to defaulterrormessages enables overriding them via the errormessages argument when instantiating the FormSet. I would prefer being able to override them via inlineformsetfactory, but that is for another day.

< self.min_num             ):                 raise ValidationError(                    ngettext(                        "Please submit at least %d form.",                        "Please submit at least %d forms.",                        self.min_num,                    )                    % self.min_num,                    self.error_messages["too_few_forms"] % {"num": self.min_num},                     code="too_few_forms",                 )             # Give self.clean() a chance to do cross-form validation.


Question: Deprecate CryptPasswordHasher. Description CryptPasswordHasher was 15 years ago mainly to support legacy UNIX apps. It's almost undocumented, not recommended, and supported only on UNIX. Moreover crypt module was deprecated in Python 3.11 (see ). We should deprecate it in Django 4.1 and remove in Django 5.0.

md5,     pbkdf2, )<ins>from django.utils.deprecation import RemovedInDjango50Warning</ins> from django.utils.module_loading import import_string from django.utils.translation import gettext_noop as _


Question: Deprecate CryptPasswordHasher. Description CryptPasswordHasher was 15 years ago mainly to support legacy UNIX apps. It's almost undocumented, not recommended, and supported only on UNIX. Moreover crypt module was deprecated in Python 3.11 (see ). We should deprecate it in Django 4.1 and remove in Django 5.0.

pass  <ins># RemovedInDjango50Warning.</ins> class CryptPasswordHasher(BasePasswordHasher):     """     Password hashing using UNIX crypt (not recommended)


Question: Deprecate CryptPasswordHasher. Description CryptPasswordHasher was 15 years ago mainly to support legacy UNIX apps. It's almost undocumented, not recommended, and supported only on UNIX. Moreover crypt module was deprecated in Python 3.11 (see ). We should deprecate it in Django 4.1 and remove in Django 5.0.

algorithm = "crypt"     library = "crypt" <ins>    def __init__(self, *args, **kwargs):        warnings.warn(            "django.contrib.auth.hashers.CryptPasswordHasher is deprecated.",            RemovedInDjango50Warning,            stacklevel=2,        )        super().__init__(*args, **kwargs)</ins>     def salt(self):         return get_random_string(2)


Question: Depracate passing False to OrderBy's nullsfirst and nullslast. Description Consider the following: In [11]: [tv.publishedat for tv in TemplateVersion.objects.orderby(F("publishedat").desc(nullsfirst=True))] Out[11]: [None, datetime.datetime(2022, 2, 25, 13, 0, 12, , tzinfo=<UTC), datetime.datetime(2022, 2, 21, 10, 18, 0, , tzinfo=<UTC)] In [12]: [tv.publishedat for tv in TemplateVersion.objects.orderby(F("publishedat").desc(nullsfirst=False))] Out[12]: [None, datetime.datetime(2022, 2, 25, 13, 0, 12, , tzinfo=<UTC), datetime.datetime(2022, 2, 21, 10, 18, 0, , tzinfo=<UTC)] In [13]: [tv.publishedat for tv in TemplateVersion.objects.orderby(F("publishedat").desc(nullslast=True))] Out[13]: [datetime.datetime(2022, 2, 25, 13, 0, 12, , tzinfo=<UTC), datetime.datetime(2022, 2, 21, 10, 18, 0, , tzinfo=<UTC), None] In [14]: [tv.publishedat for tv in TemplateVersion.objects.orderby(F("publishedat").desc(nullslast=False))] Out[14]: [None, datetime.datetime(2022, 2, 25, 13, 0, 12, , tzinfo=<UTC), datetime.datetime(2022, 2, 21, 10, 18, 0, , tzinfo=<UTC)] Observe how nulls_first=False still puts the nulls first. This happens because they both default False and when they are both False it lets the DB decide. This is surprising behaviour, it also makes changing the null positioning based on a variable more awkward than it needs to be. I think it would be better if they defaulted to None, let the DB decide when both are None and when one is not None do the ordering that implies.

import datetime import functools import inspect<ins>import warnings</ins> from collections import defaultdict from decimal import Decimal from uuid import UUID


Question: Depracate passing False to OrderBy's nullsfirst and nullslast. Description Consider the following: In [11]: [tv.publishedat for tv in TemplateVersion.objects.orderby(F("publishedat").desc(nullsfirst=True))] Out[11]: [None, datetime.datetime(2022, 2, 25, 13, 0, 12, , tzinfo=<UTC), datetime.datetime(2022, 2, 21, 10, 18, 0, , tzinfo=<UTC)] In [12]: [tv.publishedat for tv in TemplateVersion.objects.orderby(F("publishedat").desc(nullsfirst=False))] Out[12]: [None, datetime.datetime(2022, 2, 25, 13, 0, 12, , tzinfo=<UTC), datetime.datetime(2022, 2, 21, 10, 18, 0, , tzinfo=<UTC)] In [13]: [tv.publishedat for tv in TemplateVersion.objects.orderby(F("publishedat").desc(nullslast=True))] Out[13]: [datetime.datetime(2022, 2, 25, 13, 0, 12, , tzinfo=<UTC), datetime.datetime(2022, 2, 21, 10, 18, 0, , tzinfo=<UTC), None] In [14]: [tv.publishedat for tv in TemplateVersion.objects.orderby(F("publishedat").desc(nullslast=False))] Out[14]: [None, datetime.datetime(2022, 2, 25, 13, 0, 12, , tzinfo=<UTC), datetime.datetime(2022, 2, 21, 10, 18, 0, , tzinfo=<UTC)] Observe how nulls_first=False still puts the nulls first. This happens because they both default False and when they are both False it lets the DB decide. This is surprising behaviour, it also makes changing the null positioning based on a variable more awkward than it needs to be. I think it would be better if they defaulted to None, let the DB decide when both are None and when one is not None do the ordering that implies.

from django.db.models.constants import LOOKUP_SEP from django.db.models.query_utils import Q from django.utils.deconstruct import deconstructible<ins>from django.utils.deprecation import RemovedInDjango50Warning</ins> from django.utils.functional import cached_property from django.utils.hashable import make_hashable


Question: Depracate passing False to OrderBy's nullsfirst and nullslast. Description Consider the following: In [11]: [tv.publishedat for tv in TemplateVersion.objects.orderby(F("publishedat").desc(nullsfirst=True))] Out[11]: [None, datetime.datetime(2022, 2, 25, 13, 0, 12, , tzinfo=<UTC), datetime.datetime(2022, 2, 21, 10, 18, 0, , tzinfo=<UTC)] In [12]: [tv.publishedat for tv in TemplateVersion.objects.orderby(F("publishedat").desc(nullsfirst=False))] Out[12]: [None, datetime.datetime(2022, 2, 25, 13, 0, 12, , tzinfo=<UTC), datetime.datetime(2022, 2, 21, 10, 18, 0, , tzinfo=<UTC)] In [13]: [tv.publishedat for tv in TemplateVersion.objects.orderby(F("publishedat").desc(nullslast=True))] Out[13]: [datetime.datetime(2022, 2, 25, 13, 0, 12, , tzinfo=<UTC), datetime.datetime(2022, 2, 21, 10, 18, 0, , tzinfo=<UTC), None] In [14]: [tv.publishedat for tv in TemplateVersion.objects.orderby(F("publishedat").desc(nullslast=False))] Out[14]: [None, datetime.datetime(2022, 2, 25, 13, 0, 12, , tzinfo=<UTC), datetime.datetime(2022, 2, 21, 10, 18, 0, , tzinfo=<UTC)] Observe how nulls_first=False still puts the nulls first. This happens because they both default False and when they are both False it lets the DB decide. This is surprising behaviour, it also makes changing the null positioning based on a variable more awkward than it needs to be. I think it would be better if they defaulted to None, let the DB decide when both are None and when one is not None do the ordering that implies.

template = "%(expression)s %(ordering)s"     conditional = False     def __init__(        self, expression, descending=False, nulls_first=False, nulls_last=False    ):    def __init__(self, expression, descending=False, nulls_first=None, nulls_last=None):         if nulls_first and nulls_last:             raise ValueError("nulls_first and nulls_last are mutually exclusive")        if nulls_first is False or nulls_last is False:            # When the deprecation ends, replace with:            # raise ValueError(            #     "nulls_first and nulls_last values must be True or None."            # )            warnings.warn(                "Passing nulls_first=False or nulls_last=False is deprecated, use None "                "instead.",                RemovedInDjango50Warning,                stacklevel=2,            )         self.nulls_first = nulls_first         self.nulls_last = nulls_last         self.descending = descending


Question: Depracate passing False to OrderBy's nullsfirst and nullslast. Description Consider the following: In [11]: [tv.publishedat for tv in TemplateVersion.objects.orderby(F("publishedat").desc(nullsfirst=True))] Out[11]: [None, datetime.datetime(2022, 2, 25, 13, 0, 12, , tzinfo=<UTC), datetime.datetime(2022, 2, 21, 10, 18, 0, , tzinfo=<UTC)] In [12]: [tv.publishedat for tv in TemplateVersion.objects.orderby(F("publishedat").desc(nullsfirst=False))] Out[12]: [None, datetime.datetime(2022, 2, 25, 13, 0, 12, , tzinfo=<UTC), datetime.datetime(2022, 2, 21, 10, 18, 0, , tzinfo=<UTC)] In [13]: [tv.publishedat for tv in TemplateVersion.objects.orderby(F("publishedat").desc(nullslast=True))] Out[13]: [datetime.datetime(2022, 2, 25, 13, 0, 12, , tzinfo=<UTC), datetime.datetime(2022, 2, 21, 10, 18, 0, , tzinfo=<UTC), None] In [14]: [tv.publishedat for tv in TemplateVersion.objects.orderby(F("publishedat").desc(nullslast=False))] Out[14]: [None, datetime.datetime(2022, 2, 25, 13, 0, 12, , tzinfo=<UTC), datetime.datetime(2022, 2, 21, 10, 18, 0, , tzinfo=<UTC)] Observe how nulls_first=False still puts the nulls first. This happens because they both default False and when they are both False it lets the DB decide. This is surprising behaviour, it also makes changing the null positioning based on a variable more awkward than it needs to be. I think it would be better if they defaulted to None, let the DB decide when both are None and when one is not None do the ordering that implies.

def reverse_ordering(self):         self.descending = not self.descending        if self.nulls_first or self.nulls_last:            self.nulls_first = not self.nulls_first            self.nulls_last = not self.nulls_last        if self.nulls_first:            self.nulls_last = True            self.nulls_first = None        elif self.nulls_last:            self.nulls_first = True            self.nulls_last = None         return self      def asc(self):


Question: APPENDSLASH adds significant latency to all requests not ending in / (even if successful) Description Originally, APPENDSLASH worked by looking for 404 responses and replacing them with redirects, so as not to unnecessarily impact the performance of successful responses. However, commit in 1.9.5/1.10 changed this to check shouldredirectwithslash() on every request, resulting in a moderately expensive extra urlconf lookup for every request not ending with /, whether or not it succeeds as written. This performance impact was not considered in the commit message or the corresponding ticket , so I assume it was an oversight. That ticket asserted “This doesn't really make sense, since the two settings are not interdependent”, which is incorrect—performance was the reason for the interdependence. The overhead was found to be significant enough in Zulip to merit subclassing CommonMiddleware to skip it in certain conditions. Here’s a minimal test project with an exaggerated number of routes so the overhead can be easily observed. $ runserver $ wrk http://127.0.0.1:8000/url9999 Running 10s test @ http://127.0.0.1:8000/url9999 2 threads and 10 connections Thread Stats Avg Stdev Max +/- Stdev Latency 232.40ms 73.85ms 570.86ms 69.16% Req/Sec 21.70 9.47 40.00 63.35% 426 requests in 10.01s, 64.90KB read Requests/sec: 42.56 Transfer/sec: 6.48KB $ sed -i 's/# APPENDSLASH = False/APPENDSLASH = False/' $ wrk http://127.0.0.1:8000/url9999 Running 10s test @ http://127.0.0.1:8000/url9999 2 threads and 10 connections Thread Stats Avg Stdev Max +/- Stdev Latency 139.80ms 52.07ms 352.19ms 69.09% Req/Sec 36.46 12.23 60.00 58.12% 714 requests in 10.01s, 108.79KB read Requests/sec: 71.32 Transfer/sec: 10.87KB

# Check for a redirect based on settings.PREPEND_WWW         host = request.get_host()        must_prepend = settings.PREPEND_WWW and host and not host.startswith("www.")        redirect_url = ("%s://www.%s" % (request.scheme, host)) if must_prepend else ""         # Check if a slash should be appended        if self.should_redirect_with_slash(request):            path = self.get_full_path_with_slash(request)        else:            path = request.get_full_path()        # Return a redirect if necessary        if redirect_url or path != request.get_full_path():            redirect_url += path            return self.response_redirect_class(redirect_url)        if settings.PREPEND_WWW and host and not host.startswith("www."):            # Check if we also need to append a slash so we can do it all            # with a single redirect. (This check may be somewhat expensive,            # so we only do it if we already know we're sending a redirect,            # or in process_response if we get a 404.)            if self.should_redirect_with_slash(request):                path = self.get_full_path_with_slash(request)            else:                path = request.get_full_path()            return self.response_redirect_class(f"{request.scheme}://www.{host}{path}")      def should_redirect_with_slash(self, request):         """


Question: RenameIndex() crashes when unnamed index is moving backward and forward. Description RenameIndex() should restore the old auto-generated name when an unnamed index for uniquetogether is moving backward. Now re-applying RenameIndex() crashes. For example: diff --git index .. a b class OperationTests(OperationTestBase): with connection.schemaeditor() as editor, self.assertNumQueries(0): operation.databasebackwards(applabel, editor, newstate, projectstate) self.assertIndexNameExists(tablename, "newponytestidx") 2991 # Re-apply renaming. 2992 with connection.schemaeditor() as editor: 2993 operation.databaseforwards(applabel, editor, projectstate, newstate) 2994 self.assertIndexNameExists(tablename, "newponytestidx") # Deconstruction. definition = operation.deconstruct() self.assertEqual(definition[0], "RenameIndex") crashes on PostgreSQL: relation "newponytest_idx" already exists

else:             from_model_state = from_state.models[app_label, self.model_name_lower]             old_index = from_model_state.get_index_by_name(self.old_name)<ins>        # Don't alter when the index name is not changed.        if old_index.name == self.new_name:            return</ins>          to_model_state = to_state.models[app_label, self.model_name_lower]         new_index = to_model_state.get_index_by_name(self.new_name)


Question: Templates crash when calling methods for built-in types. Description (last modified by Daniel) Found during a 2.2 -3.2 upgrade: Given a template: {{ foo }} where foo is non-existant, it returns nothing, empty. (That's good) {{ }} also empty (Also good) {% include '' with bar=foo %} and then in having: {{ }} results in File "/Users/daniel/src/django-bug-test/.v/lib/python3.8/site-", line 861, in resolvelookup signature = inspect.signature(current) File "", line 3093, in signature return Signature.fromcallable(obj, followwrapped=followwrapped) File "", line 2842, in fromcallable return signaturefromcallable(obj, sigcls=cls, File "", line 2296, in signaturefromcallable return signaturefrombuiltin(sigcls, obj, File "", line 2107, in signaturefrombuiltin raise ValueError("no signature found for builtin {!r}".format(func)) Exception Type: ValueError at / Exception Value: no signature found for builtin <built-in method count of str object at 0x1100ff2f0On django 2.2, this would not crash, but resulted in empty (as I expected). this seems to fix it for me: diff --git index .. --- +++ -913,15 +913,19 def resolvelookup(self, context): try: # method call (assuming no args required) current = current() except TypeError: signature = inspect.signature(current) try: () except TypeError: # arguments were required current = ( ) # invalid method call signature = inspect.signature(current) except ValueError: # python builtins might not have signature current = else: raise try: () except TypeError: # arguments were required current = ( ) # invalid method call else: raise except Exception as e: templatename = getattr(context, "templatename", None) or "unknown" (

try:  # method call (assuming no args required)                             current = current()                         except TypeError:                            signature = inspect.signature(current)                             try:                                signature.bind()                            except TypeError:  # arguments *were* required                                current = (                                    context.template.engine.string_if_invalid                                )  # invalid method call                                signature = inspect.signature(current)                            except ValueError:  # No signature found.                                current = context.template.engine.string_if_invalid                             else:                                raise                                try:                                    signature.bind()                                except TypeError:  # Arguments *were* required.                                    # Invalid method call.                                    current = context.template.engine.string_if_invalid                                else:                                    raise         except Exception as e:             template_name = getattr(context, "template_name", None) or "unknown"             logger.debug(


Question: Deprecate Model.Meta.indextogether in favour of Model.Meta.indexes Description Anything that indextogether does can be done by indexes and the Index class.

option_name = "index_together"     system_check_deprecated_details = {        "msg": (            "AlterIndexTogether is deprecated. Support for it (except in historical "            "migrations) will be removed in Django 5.1."        ),        "id": "migrations.W001",    }     def __init__(self, name, index_together):         super().__init__(name, index_together) diff --git a/django/db/models/options.py b/django/db/models/options.py-- a/django/db/models/options.py++ b/django/db/models/options.py


Question: Deprecate Model.Meta.indextogether in favour of Model.Meta.indexes Description Anything that indextogether does can be done by indexes and the Index class.

import bisect import copy import inspect<ins>import warnings</ins> from collections import defaultdict  from django.apps import apps


Question: Deprecate Model.Meta.indextogether in favour of Model.Meta.indexes Description Anything that indextogether does can be done by indexes and the Index class.

from django.db.models import AutoField, Manager, OrderWrt, UniqueConstraint from django.db.models.query_utils import PathInfo from django.utils.datastructures import ImmutableList, OrderedSet<ins>from django.utils.deprecation import RemovedInDjango51Warning</ins> from django.utils.functional import cached_property from django.utils.module_loading import import_string from django.utils.text import camel_case_to_spaces, format_lazy