Question: and hasattr(default_manager, "get_by_natural_key")         and hasattr(Model, "natural_key")     ):        natural_key = Model(**data).natural_key()        obj = Model(**data)        obj._state.db = db        natural_key = obj.natural_key()         try:             data[Model._meta.pk.attname] = Model._meta.pk.to_python(                 default_manager.db_manager(db).get_by_natural_key(*natural_key).pk

loaddata fails on non-default database when natural keys uses foreign keys. Description (last modified by François Granade) I've got a one-to-many relationship between two models Book and Author, that define a natural keys in both models. I'm loading some data from a fixture. It works in the default database, but when I use it a second database, then I get an exception. I'm relatively new to natural keys and to serializers, but I wouldn't expect things to work differently in the default DB and others ?


Question: and hasattr(default_manager, "get_by_natural_key")         and hasattr(Model, "natural_key")     ):        natural_key = Model(**data).natural_key()        obj = Model(**data)        obj._state.db = db        natural_key = obj.natural_key()         try:             data[Model._meta.pk.attname] = Model._meta.pk.to_python(                 default_manager.db_manager(db).get_by_natural_key(*natural_key).pk

I've committed a test project here:  (The problem doesn't appear if the data is already present in the default DB) The error: % cat | loaddata --database other --format json - Traceback (most recent call last): File "/Users/francois/Library/Caches/pypoetry/virtualenvs/exportbug-PGt-cwXF-py3.9/lib/python3.9/site-", line 187, in get relobj = (instance) File "/Users/francois/Library/Caches/pypoetry/virtualenvs/exportbug-PGt-cwXF-py3.9/lib/python3.9/site-", line 15, in getcachedvalue return [cachename] KeyError: 'author' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/Users/francois/Library/Caches/pypoetry/virtualenvs/exportbug-PGt-cwXF-py3.9/lib/python3.9/site-", line 70, in Deserializer yield from PythonDeserializer(objects, options) File "/Users/francois/Library/Caches/pypoetry/virtualenvs/exportbug-PGt-cwXF-py3.9/lib/python3.9/site-", line 174, in Deserializer obj = base.buildinstance(Model, data, using) File "/Users/francois/Library/Caches/pypoetry/virtualenvs/exportbug-PGt-cwXF-py3.9/lib/python3.9/site-", line 332, in buildinstance naturalkey = Model(data).naturalkey() File "/Users/francois/lmad/src/django-", line 33, in naturalkey return (,) + () File "/Users/francois/Library/Caches/pypoetry/virtualenvs/exportbug-PGt-cwXF-py3.9/lib/python3.9/site-", line 205, in get relobj = self.getobject(instance) File "/Users/francois/Library/Caches/pypoetry/virtualenvs/exportbug-PGt-cwXF-py3.9/lib/python3.9/site-", line 168, in getobject return ((instance)) File "/Users/francois/Library/Caches/pypoetry/virtualenvs/exportbug-PGt-cwXF-py3.9/lib/python3.9/site-", line 496, in get raise ( Author matching query does not exist.


Question: and hasattr(default_manager, "get_by_natural_key")         and hasattr(Model, "natural_key")     ):        natural_key = Model(**data).natural_key()        obj = Model(**data)        obj._state.db = db        natural_key = obj.natural_key()         try:             data[Model._meta.pk.attname] = Model._meta.pk.to_python(                 default_manager.db_manager(db).get_by_natural_key(*natural_key).pk

the model: from import models class AuthorManager(models.Manager): def getbynaturalkey(self, name): return (name=name) class Author(models.Model): id = models.AutoField(primarykey=True) name = models.CharField(maxlength=255, unique=True) objects = AuthorManager() def naturalkey(self): return (,) def str(self): return f"{} {}" class BookManager(models.Manager): def getbynaturalkey(self, title, author): # OR title, author ?? return (title=title, authorname=author) class Book(models.Model): id = models.AutoField(primarykey=True) title = models.CharField(maxlength=255) author = models.ForeignKey(Author, models.DONOTHING, relatedname="books") objects = BookManager() def naturalkey(self): return (,) + () naturalkey.dependencies = ["testbug.Author"] class Meta: uniquetogether = [["title", "author"]] def str_(self): return f"{}: '{}' by {}" the data (generated with from import serializers; from import Book, Author; print(serializers.serialize("json", list(()) + list(()), indent=2, usenaturalforeignkeys=True, usenaturalprimary_keys=True)) in the shell): [ { "model": "", "fields": { "name": "JR Tolkien" } }, { "model": "", "fields": { "title": "The Ring", "author": [ "JR Tolkien" ] } } ]


Question: "content_type_id": get_content_type_for_model(obj).pk,                     "object_id": obj.pk,                 },<ins>                current_app=self.admin_site.name,</ins>             )      def get_empty_value_display(self):

viewonsite redirect does not work for custom admin site. Description All reverse calls in have a currentapp keyword argument, so they resolve with the current (possibly custom) admin site – except the reverse call in getviewonsiteurl. This lead to custom admin sites using the default admin to redirect via `viewon_site. This is clearly a bug. When the default admin is handled with extra protections outside of Django (e.g. additional basic auth), users of the custom admin cannot use the "view on site" link because it routes through the default admin. Patch is easy and already here:  I'll try to provide a regression test as well.


Question: """         return len([1 for count in self.alias_refcount.values() if count])     def join(self, join, reuse=None):    def join(self, join, reuse=None, reuse_with_filtered_relation=False):         """         Return an alias for the 'join', either reusing an existing alias for         that join or creating a new one. 'join' is either a base_table_class or

Using multiple FilteredRelation with different filters but for same relation is ignored. Description (last modified by lind-marcus) I have a relation that ALWAYS have at least 1 entry with isall=True and then I have an optional entry that could have isall=False but instead have zone set. I'm trying to use FilteredRelation together with Case(When()) to ensure that it use the zone level one (if exist) and fall back on "All" if zone do not exist. from import FilteredRelation ( relationzone=FilteredRelation( "myrelationnested", condition=Q(myrelationnestedzone=F("zone")) ), relationall=FilteredRelation( "myrelationnested", condition=Q(myrelationnestedisall=True) ), pricezone=F("relationzoneprice") ).annotate( pricefinal=Case( When( pricezoneisnull=True, then=F("relationallprice"), ), default=F("pricezone") ) ) I noticed that when using multiple FilteredRelation with the same relation (myrelationnested) it actually just generates a single SQL JOIN (if inspecting the raw SQL) and ignores the other. So in this case if I do print(str()) I would only see a join for relationzone. Not for relation_all. Is this intended behavior or should I be able to do the thing above?


Question: The 'reuse' parameter can be either None which means all joins are         reusable, or it can be a set containing the aliases that can be reused.         The 'reuse_with_filtered_relation' parameter is used when computing        FilteredRelation instances.         A join is always created as LOUTER if the lhs alias is LOUTER to make         sure chains like t1 LOUTER t2 INNER t3 aren't generated. All new         joins are created as LOUTER if the join is nullable.         """        reuse_aliases = [            a            for a, j in self.alias_map.items()            if (reuse is None or a in reuse) and j.equals(join)        ]        if reuse_with_filtered_relation and reuse:            reuse_aliases = [                a for a, j in self.alias_map.items() if a in reuse and j.equals(join)            ]        else:            reuse_aliases = [                a                for a, j in self.alias_map.items()                if (reuse is None or a in reuse) and j == join            ]         if reuse_aliases:             if join.table_alias in reuse_aliases:                 reuse_alias = join.table_alias


Question: can_reuse=None,         allow_joins=True,         split_subq=True,<ins>        reuse_with_filtered_relation=False,</ins>         check_filterable=True,     ):         """


Question: The 'can_reuse' is a set of reusable joins for multijoins. <ins>        If 'reuse_with_filtered_relation' is True, then only joins in can_reuse        will be reused.</ins>         The method will create a filter clause that can be added to the current         query. However, if the filter isn't added to the query then the caller         is responsible for unreffing the joins used.


Question: alias,                 can_reuse=can_reuse,                 allow_many=allow_many,<ins>                reuse_with_filtered_relation=reuse_with_filtered_relation,</ins>             )              # Prevent iterator from being consumed by check_related_objects()


Question: current_negated=current_negated,                     allow_joins=True,                     split_subq=False,<ins>                    reuse_with_filtered_relation=True,</ins>                 )             target_clause.add(child_clause, connector)         return target_clause


Question: break         return path, final_field, targets, names[pos + 1 :]     def setup_joins(self, names, opts, alias, can_reuse=None, allow_many=True):    def setup_joins(        self,        names,        opts,        alias,        can_reuse=None,        allow_many=True,        reuse_with_filtered_relation=False,    ):         """         Compute the necessary table joins for the passage through the fields         given in 'names'. 'opts' is the Options class for the current model


Question: that can be reused. Note that non-reverse foreign keys are always         reusable when using setup_joins(). <ins>        The 'reuse_with_filtered_relation' can be used to force 'can_reuse'        parameter and force the relation on the given connections.</ins>         If 'allow_many' is False, then any reverse foreign key seen will         generate a MultiJoin exception.


Question: nullable,                 filtered_relation=filtered_relation,             )            reuse = can_reuse if join.m2m else None            alias = self.join(connection, reuse=reuse)            reuse = can_reuse if join.m2m or reuse_with_filtered_relation else None            alias = self.join(                connection,                reuse=reuse,                reuse_with_filtered_relation=reuse_with_filtered_relation,            )             joins.append(alias)             if filtered_relation:                 filtered_relation.path = joins[:]


Question: from django.core import exceptions from django.core.management.base import BaseCommand, CommandError from django.db import DEFAULT_DB_ALIAS<ins>from django.utils.functional import cached_property</ins> from django.utils.text import capfirst

createsuperuser does not validate usernames that use a UniqueConstraint. Description With a custom User model that uses a UniqueConstraint instead of unique=True, the createsuperuser command does not validate usernames at all. class CustomUser(AbstractBaseUser): customusername = models.CharField(maxlength=255) USERNAMEFIELD = 'customusername' class Meta: constraints = [ models.UniqueConstraint(fields=['customusername'], name='uniqueusernames'), ] Running createsuperuser to create a user with a username that already exists then results in an IntegrityError: IntegrityError: duplicate key value violates unique constraint "uniqueusernames" DETAIL: Key (customusername)=(foo) already exists.


Question: else "",         )     @cached_property    def username_is_unique(self):        if self.username_field.unique:            return True        for unique_constraint in self.UserModel._meta.total_unique_constraints:            if (                len(unique_constraint.fields) == 1                and unique_constraint.fields[0] == self.username_field.name            ):                return True        return False     def _validate_username(self, username, verbose_field_name, database):         """Validate username. If invalid, return a string error message."""        if self.username_field.unique:        if self.username_is_unique:             try:                 self.UserModel._default_manager.db_manager(database).get_by_natural_key(                     username


Question: # - changing only a field name         # - changing an attribute that doesn't affect the schema         # - adding only a db_column and the column name is not changed        non_database_attrs = [            "blank",            "db_column",            "editable",            "error_messages",            "help_text",            "limit_choices_to",            # Database-level options are not supported, see #21961.            "on_delete",            "related_name",            "related_query_name",            "validators",            "verbose_name",        ]        for attr in non_database_attrs:        for attr in old_field.non_db_attrs:             old_kwargs.pop(attr, None)        for attr in new_field.non_db_attrs:             new_kwargs.pop(attr, None)         return self.quote_name(old_field.column) != self.quote_name(             new_field.columndiff --git a/django/db/models/fields/__init__.py b/django/db/models/fields/__init__.py-- a/django/db/models/fields/__init__.py++ b/django/db/models/fields/__init__.py

AlterField operation should be noop when adding/changing choices on SQLite. Description while writing a test case for i found that for sqlite, even a seemingly db-transparent change like adding choices still generates sql (new table + insert + drop + rename) even though this shouldn't be needed. on e.g. postgres the same migration generates no sql


Question: system_check_deprecated_details = None     system_check_removed_details = None <ins>    # Attributes that don't affect a column definition.    # These attributes are ignored when altering the field.    non_db_attrs = (        "blank",        "choices",        "db_column",        "editable",        "error_messages",        "help_text",        "limit_choices_to",        # Database-level options are not supported, see #21961.        "on_delete",        "related_name",        "related_query_name",        "validators",        "verbose_name",    )</ins>     # Field flags     hidden = False


Question: query.clear_ordering(force=True)         query.extra = {}         query.select = []        query.add_fields([query.get_meta().pk.name])        meta = query.get_meta()        fields = [meta.pk.name]        related_ids_index = []        for related in self.query.related_updates:            if all(                path.join_field.primary_key for path in meta.get_path_to_parent(related)            ):                # If a primary key chain exists to the targeted related update,                # then the meta.pk value can be used for it.                related_ids_index.append((related, 0))            else:                # This branch will only be reached when updating a field of an                # ancestor that is not part of the primary key chain of a MTI                # tree.                related_ids_index.append((related, len(fields)))                fields.append(related._meta.pk.name)        query.add_fields(fields)         super().pre_sql_setup()          must_pre_select = (

Wrong behavior on queryset update when multiple inheritance Description Queryset update has a wrong behavior when queryset class inherits multiple classes. The update happens not on child class but on other parents class instances. Here an easy example to show the problem: class Base(models.Model): baseid = models.AutoField(primarykey=True) fieldbase = models.IntegerField() class OtherBase(models.Model): otherbaseid = models.AutoField(primarykey=True) fieldotherbase = models.IntegerField() class Child(Base, OtherBase): pass Then in django shell: In [1]: (fieldotherbase=100) <QuerySet [{'otherbaseid': 1, 'fieldotherbase': 100}]In [2]: (fieldotherbase=101) <QuerySet [{'otherbaseid': 2, 'fieldotherbase': 101}]In [3]: (fieldbase=0, fieldotherbase=0) <Child: Child object (1)In [4]: (fieldbase=1, fieldotherbase=1) <Child: Child object (2)In [5]: (fieldotherbase=55) SELECT "applianceschild"."baseptrid" FROM "applianceschild" Execution time: 0.000647s [Database: default] UPDATE "appliancesotherbase" SET "fieldotherbase" = 55 WHERE "appliancesotherbase"."otherbaseid" IN (1, 2) Execution time: 0.001414s [Database: default] Out[5]: 2 In [6]: ('fieldotherbase') <QuerySet [{'fieldotherbase': 0}, {'fieldotherbase': 1}]In [7]: (otherbaseidin=[1,2]).values('fieldotherbase') <QuerySet [{'fieldotherbase': 55}, {'fieldotherbase': 55}]As seen on the above code, updating Child fields from second parent has no effect. Worse is that OtherBase fields where modifed because query is using primiary keys from Base class.


Question: # don't want them to change), or the db backend doesn't support             # selecting from the updating table (e.g. MySQL).             idents = []            related_ids = collections.defaultdict(list)             for rows in query.get_compiler(self.using).execute_sql(MULTI):                 idents.extend(r[0] for r in rows)                for parent, index in related_ids_index:                    related_ids[parent].extend(r[index] for r in rows)             self.query.add_filter("pk__in", idents)            self.query.related_ids = idents            self.query.related_ids = related_ids         else:             # The fast path. Filters and updates in one query.             self.query.add_filter("pk__in", query)diff --git a/django/db/models/sql/subqueries.py b/django/db/models/sql/subqueries.py-- a/django/db/models/sql/subqueries.py++ b/django/db/models/sql/subqueries.py


Question: query = UpdateQuery(model)             query.values = values             if self.related_ids is not None:                query.add_filter("pk__in", self.related_ids)                query.add_filter("pk__in", self.related_ids[model])             result.append(query)         return result


Question: if lookup_name is None:             lookup_name = lookup.lookup_name         del cls.class_lookups[lookup_name]<ins>        cls._clear_cached_lookups()</ins>   def select_related_descend(field, restricted, requested, load_fields, reverse=False):

RegisterLookupMixin.unregisterlookup() should clear the lookup cache. Description (last modified by Himanshu Balasamanta) In current source code, in the unregisterlookup method, , the cache is not cleared, which should be done, as it is done in registerlookup,  Corresponding to this change, minor changes need to be brought in the test. The PR generated is


Question: if not isinstance(backend, DjangoTemplates):             continue         items.update(cwd / to_path(dir) for dir in backend.engine.dirs)        items.update(cwd / to_path(dir) for dir in backend.engine.dirs if dir)          for loader in backend.engine.template_loaders:             if not hasattr(loader, "get_dirs"):

Django 3.2.4+ autoreload breaks on empty string in TEMPLATES DIRS. Description Django versions 3.2.3 changes the way template dirs are handled, they are now normalized using pathlib.Path. People having an invalid value in TEMPLATESDIRS? will notice that autoreload stops working. "DIRS": ("TEMPLATESDIRS", "").split(",") # wrong, should be filter(None, ("TEMPLATESDIRS", "").split(",")) or anything else that produces this: "DIRS": [''] # wrong will break autoreload. This happens because was previously comparing the empty string to a directory, and would never match. Now the normalization transforms the empty string into the root of the project. The result is that templatechanged() will now always return True, preventing the autoreload when the app code changes Change that produced the regression Commits in main and stable/3.2.x:   Previous reports


Question: items.update(                 cwd / to_path(directory)                 for directory in loader.get_dirs()                if not is_django_path(directory)                if directory and not is_django_path(directory)             )     return items


Question: def exists(self, using, limit=True):         q = self.clone()        if not q.distinct:        if not (q.distinct and q.is_sliced):             if q.group_by is True:                 q.add_fields(                     (f.attname for f in self.model._meta.concrete_fields), False

clear select fields for of distinct, non-sliced querysets Description The fix () for bug introduced an inefficiency in .exists() on distinct querysets in that now all distinct querysets are selecting all their fields. I propose the fields should be left on the queryset only if (q.distinct and (q.highmark is not None or q.lowmark is not None)). As I started writing the test for this, I realized I would have to inspect the query itself, which seems... like a bad idea for a test. Would this be reasonable to submit the (one-line-change) patch without a test, or does someone have a suggestion for a way to make the test that's less brittle?


Question: loaders = ["django.template.loaders.filesystem.Loader"]             if app_dirs:                 loaders += ["django.template.loaders.app_directories.Loader"]            if not debug:                loaders = [("django.template.loaders.cached.Loader", loaders)]            loaders = [("django.template.loaders.cached.Loader", loaders)]         else:             if app_dirs:                 raise ImproperlyConfigured(

Enable cached template loader in development. Description (last modified by Carlton Gibson) Following the changes to allow auto-reloading cached templates in , there was discussion on PR to enable the cached template loader by default even in development. (Folks not wanting that would specify loaders.)


Question: # Security check -- Ensure the user-originating redirection URL is             # safe.             if not url_is_safe:                next_page = self.request.path                if settings.LOGOUT_REDIRECT_URL:                    next_page = resolve_url(settings.LOGOUT_REDIRECT_URL)                else:                    next_page = self.request.path         return next_page      def get_context_data(self, **kwargs):

Unnecessary redirect in LogoutView when ?next=... contains "unsafe" URL Description Reproduction instructions Set LOGOUTREDIRECTURL Wire () at /logout/ in the URLconf Add this form to any template: <form action="/logout/?next=http://evil/" method="POST"<input type="submit" value="Logout"{% csrftoken %}</formLog in, then use the form to log out Expected result You are logged out; the next parameter is ignored; you are redirected to LOGOUTREDIRECTURL Actual result There is an intermediary, useless redirect; see the logs of the development server: [16/Apr/2022 19:05:38] "POST /logout/?next=http://evil/ HTTP/1.1" 302 0 [16/Apr/2022 19:05:38] "GET /logout/ HTTP/1.1" 302 0 [16/Apr/2022 19:05:38] "GET /en/ HTTP/1.1" 200 I noticed this via code inspection. The implementation of LogoutView.getnextpage seemed a bit weird to me. This stems from  which predates the introduction of LOGOUTREDIRECTURL. From the user's perspective, the behavior is correct. There's just an extra round-trip and needlessly complicated code.


Question: def _prepare_related_fields_for_save(self, operation_name, fields=None):         # Ensure that a model instance without a PK hasn't been assigned to        # a ForeignKey or OneToOneField on this model. If the field is        # nullable, allowing the save would result in silent data loss.        # a ForeignKey, GenericForeignKey or OneToOneField on this model. If        # the field is nullable, allowing the save would result in silent data        # loss.         for field in self._meta.concrete_fields:             if fields and field not in fields:                 continue

Inconsistent / Unexpected handling of assigning unsaved model to Generic Foreign Key Description addresses the issue of assigned an unsaved model to a ForeignKey or OneToOneField (raises error when save() called), however the same logic / pattern does not apply to GFKs. Given: class ModelA(models.Model): name = models.CharField(maxlength=20) class ModelB(models.Model): gfkctype = models.ForeignKey(ContentType, ondelete=models.PROTECT) gfkid = models.PositiveIntegerField() gfk = GenericForeignKey('gfkctype', 'gfkid') class ModelC(models.Model): fk = models.ForeignKey(ModelA, ondelete=models.CASCADE) Foreign Key Behaviour: In [2]: a = ModelA(name='Model A') In [3]: c = ModelC(fk=a) In [4]: c.fk Out[4]: <ModelA: ModelA object (None)In [5]: c.save() ... ValueError: save() prohibited to prevent data loss due to unsaved related object 'fk'. In [6]: a.save() (0.016) INSERT INTO "testappmodela" ("name") VALUES ('Model A'); args=['Model A'] In [7]: c.fk Out[7]: <ModelA: ModelA object (1)In [8]: c.save() (0.016) INSERT INTO "testappmodelc" ("fkid") VALUES (1); args=[1] GFK behaviour: In [9]: a2 = ModelA(name='Model A2') In [10]: b = ModelB(gfk=a2) In [11]: b.gfk Out[11]: <ModelA: ModelA object (None)In [12]: b.save() (0.000) INSERT INTO "testappmodelb" ("gfkctypeid", "gfkid") VALUES (9, NULL); args=[9, None] IntegrityError: NOT NULL constraint failed: In [14]: () (0.015) INSERT INTO "testappmodela" ("name") VALUES ('Model A2'); args=['Model A2'] In [15]: b.gfk (0.000) SELECT "testappmodela"."id", "testappmodela"."name" FROM "testappmodela" WHERE "testappmodela".


Question: def _prepare_related_fields_for_save(self, operation_name, fields=None):         # Ensure that a model instance without a PK hasn't been assigned to        # a ForeignKey or OneToOneField on this model. If the field is        # nullable, allowing the save would result in silent data loss.        # a ForeignKey, GenericForeignKey or OneToOneField on this model. If        # the field is nullable, allowing the save would result in silent data        # loss.         for field in self._meta.concrete_fields:             if fields and field not in fields:                 continue

"id" IS NULL LIMIT 21; args=() None In [17]: b.gfkctype Out[17]: <ContentType: testapp | model aTwo observations: No check on b.gfk and b.gfkid value during save() which could lead to silent data loss if b.gfkid is nullable. When a2 is saved, accessing b.gfk now does a redundant DB query to try and find ModelA instance with PK = None, then then returns None value (effectively un-assigning a2 model), while keeping b.gfkctype intact. This is because the new pk of a2 is different to the existing gfkid (pkval) of the GFK field (None) What should be done: Modify () or Model.preparerelatedfieldsforsave() to also perform verification check for GFK fields Modify () to handle case of pkval = None (update fkfield value using PK value of GFK model if present, do not perform redundant DB query on pk=None, return previously assigned (then saved) model instead of None)


Question: self, field.attname                 ):                     field.delete_cached_value(self)<ins>        # GenericForeignKeys are private.        for field in self._meta.private_fields:            if fields and field not in fields:                continue            if (                field.is_relation                and field.is_cached(self)                and hasattr(field, "fk_field")            ):                obj = field.get_cached_value(self, default=None)                if obj and obj.pk is None:                    raise ValueError(                        f"{operation_name}() prohibited to prevent data loss due to "                        f"unsaved related object '{field.name}'."                    )</ins>      def delete(self, using=None, keep_parents=False):         if self.pk is None:


Question: self, field.attname                 ):                     field.delete_cached_value(self)<ins>        # GenericForeignKeys are private.        for field in self._meta.private_fields:            if fields and field not in fields:                continue            if (                field.is_relation                and field.is_cached(self)                and hasattr(field, "fk_field")            ):                obj = field.get_cached_value(self, default=None)                if obj and obj.pk is None:                    raise ValueError(                        f"{operation_name}() prohibited to prevent data loss due to "                        f"unsaved related object '{field.name}'."                    )</ins>      def delete(self, using=None, keep_parents=False):         if self.pk is None:


Question: # there instead             if collect_sql:                 schema_editor.collected_sql.append("--")                if not operation.reduces_to_sql:                    schema_editor.collected_sql.append(                        "-- MIGRATION NOW PERFORMS OPERATION THAT CANNOT BE WRITTEN AS "                        "SQL:"                    )                 schema_editor.collected_sql.append("-- %s" % operation.describe())                 schema_editor.collected_sql.append("--")                 if not operation.reduces_to_sql:                    schema_editor.collected_sql.append(                        "-- THIS OPERATION CANNOT BE WRITTEN AS SQL"                    )                     continue                collected_sql_before = len(schema_editor.collected_sql)             # Save the state before the operation has run             old_state = project_state.clone()             operation.state_forwards(self.app_label, project_state)

Add SQL comment to describe deliberately no-op migration operations Description Currently when a field migration is a no-op, the operation description is output in SQL, but nothing else. This can be confusing as to which operations are no-ops. It could be clearer if we output an extra SQL comment when there are deliberately no statements to execute for a given migration operation. Take for example this output: BEGIN; -- Alter field name on Author ALTER ...; -- Alter field title on Book COMMIT; The field has an operation applied, whilst needs no changes to the database. This isn't exactly clear from the output - is the COMMIT part of the change? It could be clearer as: BEGIN; -- Alter field name on Author ALTER ...; -- Alter field name on Author -- (no-op) COMMIT; (Or perhaps more verbose wording, like "no SQL to execute") I think this can help especially when there are consecutive operations with no-op SQL: BEGIN; -- Alter field name on Author -- (no-op) -- Alter field name on Author -- (no-op) COMMIT; (Inspired by , where the OP suggested dropping such migration operation header comments.)


Question: operation.database_forwards(                     self.app_label, schema_editor, old_state, project_state                 )<ins>            if collect_sql and collected_sql_before == len(schema_editor.collected_sql):                schema_editor.collected_sql.append("-- (no-op)")</ins>         return project_state      def unapply(self, project_state, schema_editor, collect_sql=False):


Question: for operation, to_state, from_state in to_run:             if collect_sql:                 schema_editor.collected_sql.append("--")                if not operation.reduces_to_sql:                    schema_editor.collected_sql.append(                        "-- MIGRATION NOW PERFORMS OPERATION THAT CANNOT BE WRITTEN AS "                        "SQL:"                    )                 schema_editor.collected_sql.append("-- %s" % operation.describe())                 schema_editor.collected_sql.append("--")                 if not operation.reduces_to_sql:                    schema_editor.collected_sql.append(                        "-- THIS OPERATION CANNOT BE WRITTEN AS SQL"                    )                     continue                collected_sql_before = len(schema_editor.collected_sql)             atomic_operation = operation.atomic or (                 self.atomic and operation.atomic is not False             )


Question: operation.database_backwards(                     self.app_label, schema_editor, from_state, to_state                 )<ins>            if collect_sql and collected_sql_before == len(schema_editor.collected_sql):                schema_editor.collected_sql.append("-- (no-op)")</ins>         return project_state      def suggest_name(self):


Question: self.execute(self._delete_unique_sql(model, constraint_name))         # Drop incoming FK constraints if the field is a primary key or unique,         # which might be a to_field target, and things are going to change.        old_collation = old_db_params.get("collation")        new_collation = new_db_params.get("collation")         drop_foreign_keys = (             self.connection.features.supports_foreign_keys             and (                 (old_field.primary_key and new_field.primary_key)                 or (old_field.unique and new_field.unique)             )            and old_type != new_type            and ((old_type != new_type) or (old_collation != new_collation))         )         if drop_foreign_keys:             # '_meta.related_field' also contains M2M reverse fields, these

Errors with dbcollation – no propagation to foreignkeys Description (last modified by typonaut) Using dbcollation with a pk that also has referenced fks in other models causes foreign key constraint errors in MySQL. With the following models: class Account(models.Model): id = ShortUUIDField(primarykey=True, dbcollation='utf8bin', dbindex=True, maxlength=22) … class Address(models.Model): id = ShortUUIDField(primarykey=True, dbcollation='utf8bin', dbindex=True, maxlength=22) account = models.OneToOneField(Account, ondelete=models.CASCADE) … class Profile(models.Model): id = ShortUUIDField(primarykey=True, dbcollation='utf8bin', dbindex=True, maxlength=22) … account = models.ForeignKey('Account', verbosename=('account'), null=True, blank=True, ondelete=models.CASCADE) … etc Where has been changed from models.BigAutoField if makemigrations is run then it produces sqlmigrate output like this: ALTER TABLE MODIFY varchar(22) COLLATE ; ALTER TABLE MODIFY varchar(22) NOT NULL; ALTER TABLE MODIFY varchar(22) NULL; ALTER TABLE ADD CONSTRAINT FOREIGN KEY () REFERENCES (); ALTER TABLE ADD CONSTRAINT FOREIGN KEY () REFERENCES (); With this SQL the ADD CONSTRAINT queries fail. This is because the COLLATE should also be present in the bmanageaddress.accountid and bmanageprofile.accountid modification statements. Like this: ALTER TABLE MODIFY varchar(22) COLLATE ; ALTER TABLE MODIFY varchar(22) NOT NULL COLLATE ; ALTER TABLE MODIFY varchar(22) NULL COLLATE ; ALTER TABLE ADD CONSTRAINT FOREIGN KEY () REFERENCES (); ALTER TABLE ADD CONSTRAINT FOREIGN KEY () REFERENCES (); In the latter case the ADD CONSTRAINT statements run without error. The collation of the pk must match the collation of the fk otherwise an error will occur.


Question: old_type_suffix = old_field.db_type_suffix(connection=self.connection)         new_type_suffix = new_field.db_type_suffix(connection=self.connection)         # Collation change?<del>        old_collation = old_db_params.get("collation")        new_collation = new_db_params.get("collation")</del>         if old_collation != new_collation:             # Collation change handles also a type change.             fragment = self._alter_column_collation_sql(


Question: for old_rel, new_rel in rels_to_update:             rel_db_params = new_rel.field.db_parameters(connection=self.connection)             rel_type = rel_db_params["type"]            fragment, other_actions = self._alter_column_type_sql(                new_rel.related_model, old_rel.field, new_rel.field, rel_type            )            rel_collation = rel_db_params.get("collation")            old_rel_db_params = old_rel.field.db_parameters(connection=self.connection)            old_rel_collation = old_rel_db_params.get("collation")            if old_rel_collation != rel_collation:                # Collation change handles also a type change.                fragment = self._alter_column_collation_sql(                    new_rel.related_model,                    new_rel.field,                    rel_type,                    rel_collation,                )                other_actions = []            else:                fragment, other_actions = self._alter_column_type_sql(                    new_rel.related_model, old_rel.field, new_rel.field, rel_type                )             self.execute(                 self.sql_alter_column                 % {diff --git a/django/db/backends/oracle/features.py b/django/db/backends/oracle/features.py-- a/django/db/backends/oracle/features.py++ b/django/db/backends/oracle/features.py


Question: "Raises ORA-00600: internal error code.": {             "model_fields.test_jsonfield.TestQuerying.test_usage_in_subquery",         },        "Oracle doesn't support changing collations on indexed columns (#33671).": {            "migrations.test_operations.OperationTests."            "test_alter_field_pk_fk_db_collation",        },     }     django_test_expected_failures = {         # A bug in Django/cx_Oracle with respect to string handling (#23843).diff --git a/django/db/backends/sqlite3/schema.py b/django/db/backends/sqlite3/schema.py-- a/django/db/backends/sqlite3/schema.py++ b/django/db/backends/sqlite3/schema.py


Question: # Alter by remaking table         self._remake_table(model, alter_field=(old_field, new_field))         # Rebuild tables with FKs pointing to this field.        if new_field.unique and old_type != new_type:        old_collation = old_db_params.get("collation")        new_collation = new_db_params.get("collation")        if new_field.unique and (            old_type != new_type or old_collation != new_collation        ):             related_models = set()             opts = new_field.model._meta             for remote_field in opts.related_objects:diff --git a/django/db/models/fields/related.py b/django/db/models/fields/related.py-- a/django/db/models/fields/related.py++ b/django/db/models/fields/related.py


Question: return self.target_field.rel_db_type(connection=connection)      def db_parameters(self, connection):        return {"type": self.db_type(connection), "check": self.db_check(connection)}        target_db_parameters = self.target_field.db_parameters(connection)        return {            "type": self.db_type(connection),            "check": self.db_check(connection),            "collation": target_db_parameters.get("collation"),        }      def convert_empty_strings(self, value, expression, connection):         if (not value) and isinstance(value, str):


Question: OuterRef,     Ref,     ResolvedOuterRef,<ins>    Value,</ins> ) from django.db.models.fields import Field from django.db.models.fields.related_lookups import MultiColSource

Unnecessary column in a GROUP BY clause with () Description I've got a question about the interaction between exists() and group by. For example something like: ('field').annotate(cnt=Count('id')).filter(cnt_gt=1).exists() The corresp. query with PostgreSQL looks like this: SELECT (1) AS "a" FROM "appmodel" GROUP BY "appmodel"."field", (1) HAVING COUNT("appmodel"."id") 1 LIMIT 1 exists() (see ) clears the SELECT clause and replaces it by (if I understand correctly) a hardcoded value 1 (as "a"), along with a limit of 1, which makes sense to me. But getgroupby() (see ) pushes this hardcoded value to the GROUP BY clause and we end up with the query above. Now, on PostgreSQL, that works, but to me it sounds like it works by luck/lucky robustness... and certainly the same query without the , (1) in the GROUP BY clause yields the same result. The problem is that outside of PostgreSQL that GROUP BY clause can be downright invalid... Note that if I alter exists() to use {'a': 2} instead of {'a': 1}, it does not work anymore (on PostgreSQL), because (2) in the SELECT clause means the hardcoded number 2 while (2) in the GROUP BY clause presumably (??) refers to the 2nd expr of the SELECT clause, but we only got one... My feeling is that getgroupby() should avoid adding extra/raw sql (or at least extra/raw pure values, if we can detect that...) to the group by expressions? NB: the existing/old ticket that seems most related to my question would probably be:


Question: q.clear_ordering(force=True)         if limit:             q.set_limits(high=1)        q.add_extra({"a": 1}, None, None, None, None, None)        q.set_extra_mask(["a"])        q.add_annotation(Value(1), "a")         return q      def has_results(self, using):


Question: alter_field and getattr(alter_field[1], "primary_key", False)         ):             for name, field in list(body.items()):                if field.primary_key:                if field.primary_key and not (                    # Do not remove the old primary key when an altered field                    # that introduces a primary key is the same field.                    alter_field                    and name == alter_field[1].name                ):                     field.primary_key = False                     restore_pk_field = field                     if field.auto_created:

Migrations changes to implicit primary key when primary key is altered on SQLite. Description Migrations drops explicit primary key defined on a model when it's altered.


Question: <ins>from inspect import getattr_static, unwrap</ins> from django.contrib.sites.shortcuts import get_current_site from django.core.exceptions import ImproperlyConfigured, ObjectDoesNotExist from django.http import Http404, HttpResponse

views.Feed methods cannot be decorated Description If one applies a decorator on a method which is called by getdynamicattr a TypeError like this occurs: Exception Type: TypeError at /blog/feed/ Exception Value: itemlink() takes exactly 2 arguments (1 given) I think this is because getdynamicattr tries to count the function's arguments, but decorators usally get defined with the args, *kwargs syntax, so this trick does not work here. if code.coargcount == 2: # one argument is 'self' return attr(obj) else: return attr() I think the best approach would be to remove one of the two methods. IMHO We should have either attr(item) or attr() not both, as "there should be one, and preferably only one, obvious way to do it".


Question: # Check co_argcount rather than try/excepting the function and             # catching the TypeError, because something inside the function             # may raise the TypeError. This technique is more accurate.            func = unwrap(attr)             try:                code = attr.__code__                code = func.__code__             except AttributeError:                code = attr.__call__.__code__                func = unwrap(attr.__call__)                code = func.__code__            # If function doesn't have arguments and it is not a static method,            # it was decorated without using @functools.wraps.            if not code.co_argcount and not isinstance(                getattr_static(self, func.__name__, None), staticmethod            ):                raise ImproperlyConfigured(                    f"Feed method {attname!r} decorated by {func.__name__!r} needs to "                    f"use @functools.wraps."                )             if code.co_argcount == 2:  # one argument is 'self'                 return attr(obj)             else:


Question: import functools import re<ins>from collections import defaultdict</ins> from itertools import chain  from django.conf import settings

Implement RenameIndex in a backwards compatible way Description In order to eventually deprecate indextogether we need a way to deal with old projects that have unnamed indexes. This proves to be a non-trivial problem. Andrew and I came up with these things to consider. RenameIndex(model, newname, oldname=None, oldfields=None) where exactly one of oldname and oldfield is given (oldname ^ oldfields) If the oldname is given we use RENAME INDEX if available Otherwise look at the state and drop existing indexes and create new the index with new name On MySQL (or other DBs) that don't support RENAME INDEX, provide SQL query to look up index name from informationschema by field names and pass in to DROP INDEX. If more than one index is found while identifying with field names, migrations must error out with an AmbiguityError If the autodetector finds an old, unnamed index and a new, named one matching field signature, issue a RenameIndex operation For backwards operations with unnamed old indexes, RenameIndex is a noop.


Question: self.generated_operations = {}         self.altered_indexes = {}         self.altered_constraints = {}<ins>        self.renamed_fields = {}</ins>          # Prepare some old/new state and model lists, separating         # proxy models and ignoring unmigrated apps.


Question: self.generate_altered_options()         self.generate_altered_managers() <ins>        # Create the renamed fields and store them in self.renamed_fields.        # They are used by create_altered_indexes(), generate_altered_fields(),        # generate_removed_altered_index/unique_together(), and        # generate_altered_index/unique_together().        self.create_renamed_fields()</ins>         # Create the altered indexes and store them in self.altered_indexes.         # This avoids the same computation in generate_removed_indexes()         # and generate_added_indexes().


Question: ),             )     def generate_renamed_fields(self):    def create_renamed_fields(self):         """Work out renamed fields."""        self.renamed_fields = {}        self.renamed_operations = []        old_field_keys = self.old_field_keys.copy()         for app_label, model_name, field_name in sorted(            self.new_field_keys - self.old_field_keys            self.new_field_keys - old_field_keys         ):             old_model_name = self.renamed_models.get(                 (app_label, model_name), model_name


Question: # Scan to see if this is actually a rename!             field_dec = self.deep_deconstruct(field)             for rem_app_label, rem_model_name, rem_field_name in sorted(                self.old_field_keys - self.new_field_keys                old_field_keys - self.new_field_keys             ):                 if rem_app_label == app_label and rem_model_name == model_name:                     old_field = old_model_state.get_field(rem_field_name)


Question: if self.questioner.ask_rename(                             model_name, rem_field_name, field_name, field                         ):                            # A db_column mismatch requires a prior noop                            # AlterField for the subsequent RenameField to be a                            # noop on attempts at preserving the old name.                            if old_field.db_column != field.db_column:                                altered_field = field.clone()                                altered_field.name = rem_field_name                                self.add_operation(                            self.renamed_operations.append(                                (                                    rem_app_label,                                    rem_model_name,                                    old_field.db_column,                                    rem_field_name,                                     app_label,                                    operations.AlterField(                                        model_name=model_name,                                        name=rem_field_name,                                        field=altered_field,                                    ),                                    model_name,                                    field,                                    field_name,                                 )                            self.add_operation(                                app_label,                                operations.RenameField(                                    model_name=model_name,                                    old_name=rem_field_name,                                    new_name=field_name,                                ),                             )                            self.old_field_keys.remove(                            old_field_keys.remove(                                 (rem_app_label, rem_model_name, rem_field_name)                             )                            self.old_field_keys.add((app_label, model_name, field_name))                            old_field_keys.add((app_label, model_name, field_name))                             self.renamed_fields[                                 app_label, model_name, field_name                             ] = rem_field_name                             break     def generate_renamed_fields(self):        """Generate RenameField operations."""        for (            rem_app_label,            rem_model_name,            rem_db_column,            rem_field_name,            app_label,            model_name,            field,            field_name,        ) in self.renamed_operations:            # A db_column mismatch requires a prior noop AlterField for the            # subsequent RenameField to be a noop on attempts at preserving the            # old name.            if rem_db_column != field.db_column:                altered_field = field.clone()                altered_field.name = rem_field_name                self.add_operation(                    app_label,                    operations.AlterField(                        model_name=model_name,                        name=rem_field_name,                        field=altered_field,                    ),                )            self.add_operation(                app_label,                operations.RenameField(                    model_name=model_name,                    old_name=rem_field_name,                    new_name=field_name,                ),            )            self.old_field_keys.remove((rem_app_label, rem_model_name, rem_field_name))            self.old_field_keys.add((app_label, model_name, field_name))     def generate_added_fields(self):         """Make AddField operations."""         for app_label, model_name, field_name in sorted(


Question: def create_altered_indexes(self):         option_name = operations.AddIndex.option_name<ins>        self.renamed_index_together_values = defaultdict(list)</ins>         for app_label, model_name in sorted(self.kept_model_keys):             old_model_name = self.renamed_models.get(                 (app_label, model_name), model_name


Question: renamed_indexes.append((old_index_name, new_index_name, None))                         remove_from_added.append(new_index)                         remove_from_removed.append(old_index)<ins>            # Find index_together changed to indexes.            for (                old_value,                new_value,                index_together_app_label,                index_together_model_name,                dependencies,            ) in self._get_altered_foo_together_operations(                operations.AlterIndexTogether.option_name            ):                if (                    app_label != index_together_app_label                    or model_name != index_together_model_name                ):                    continue                removed_values = old_value.difference(new_value)                for removed_index_together in removed_values:                    renamed_index_together_indexes = []                    for new_index in added_indexes:                        _, args, kwargs = new_index.deconstruct()                        # Ensure only 'fields' are defined in the Index.                        if (                            not args                            and new_index.fields == list(removed_index_together)                            and set(kwargs) == {"name", "fields"}                        ):                            renamed_index_together_indexes.append(new_index)                    if len(renamed_index_together_indexes) == 1:                        renamed_index = renamed_index_together_indexes[0]                        remove_from_added.append(renamed_index)                        renamed_indexes.append(                            (None, renamed_index.name, removed_index_together)                        )                        self.renamed_index_together_values[                            index_together_app_label, index_together_model_name                        ].append(removed_index_together)</ins>             # Remove renamed indexes from the lists of added and removed             # indexes.             added_indexes = [


Question: model_name,             dependencies,         ) in self._get_altered_foo_together_operations(operation.option_name):<ins>            if operation == operations.AlterIndexTogether:                old_value = {                    value                    for value in old_value                    if value                    not in self.renamed_index_together_values[app_label, model_name]                }</ins>             removal_value = new_value.intersection(old_value)             if removal_value or old_value:                 self.add_operation(


Question: def copy(self):         return copy.copy(self)     def prefix_references(self, prefix):        clone = self.copy()        clone.set_source_expressions(            [                F(f"{prefix}{expr.name}")                if isinstance(expr, F)                else expr.prefix_references(prefix)                for expr in self.get_source_expressions()            ]        )        return clone     def get_group_by_cols(self, alias=None):         if not self.contains_aggregate:             return [self]diff --git a/django/db/models/sql/compiler.py b/django/db/models/sql/compiler.py-- a/django/db/models/sql/compiler.py++ b/django/db/models/sql/compiler.py

Query Expression in ordering of a related object fails Description (last modified by Tim Graham) Since 2.0, according to the doc (), we can use QueryExpression objects in the Model.Meta.ordering field. Using: from import models class Musician(models.Model): firstname = models.CharField(maxlength=50) lastname = models.CharField(maxlength=50) instrument = models.CharField(maxlength=100, null=True, blank=True) class Meta: ordering = [models.F('instrument').asc(nullslast=True)] class Album(models.Model): artist = models.ForeignKey(Musician, ondelete=models.CASCADE) name = models.CharField(maxlength=100) releasedate = models.DateField() numstars = models.IntegerField() class Meta: ordering = ['artist'] () ... TypeError: 'OrderBy' does not support indexing When reaching , the compiler tries to use the related model, but at line 679, item can be an OrderBy object. Thus the failure.


Question: ):                     item = item.desc() if descending else item.asc()                 if isinstance(item, OrderBy):                    results.append((item, False))                    results.append(                        (item.prefix_references(f"{name}{LOOKUP_SEP}"), False)                    )                     continue                 results.extend(                    self.find_ordering_name(item, opts, alias, order, already_seen)                    (expr.prefix_references(f"{name}{LOOKUP_SEP}"), is_ref)                    for expr, is_ref in self.find_ordering_name(                        item, opts, alias, order, already_seen                    )                 )             return results         targets, alias, _ = self.query.trim_joins(targets, joins, path)


Question: from django.db.migrations import Migration from django.db.migrations.autodetector import MigrationAutodetector from django.db.migrations.loader import MigrationLoader<ins>from django.db.migrations.migration import SwappableTuplefrom django.db.migrations.optimizer import MigrationOptimizer</ins> from django.db.migrations.questioner import (     InteractiveMigrationQuestioner,     MigrationQuestioner,

Create --update flag for makemigrations management command, mimicking South's one. Description When I was developing with South I was able to use the --update flag for the schemamigration management command () to refine the latest migration. This was very convenient for iterative development. Could we have an equivalent of the --update flag for Django=1.7 makemigrations? (I've taken it from )


Question: "paths of generated migration files to stdout."             ),         )<ins>        parser.add_argument(            "--update",            action="store_true",            dest="update",            help=(                "Merge model changes into the latest migration and optimize the "                "resulting operations."            ),        )</ins>      @property     def log_output(self):


Question: self.include_header = options["include_header"]         check_changes = options["check_changes"]         self.scriptable = options["scriptable"]<ins>        self.update = options["update"]</ins>         # If logs and prompts are diverted to stderr, remove the ERROR style.         if self.scriptable:             self.stderr.style_func = None


Question: else:                     self.log("No changes detected")         else:            self.write_migration_files(changes)            if self.update:                self.write_to_last_migration_files(changes)            else:                self.write_migration_files(changes)             if check_changes:                 sys.exit(1)     def write_migration_files(self, changes):    def write_to_last_migration_files(self, changes):        loader = MigrationLoader(connections[DEFAULT_DB_ALIAS])        new_changes = {}        update_previous_migration_paths = {}        for app_label, app_migrations in changes.items():            # Find last migration.            leaf_migration_nodes = loader.graph.leaf_nodes(app=app_label)            if len(leaf_migration_nodes) == 0:                raise CommandError(                    f"App {app_label} has no migration, cannot update last migration."                )            leaf_migration_node = leaf_migration_nodes[0]            # Multiple leaf nodes have already been checked earlier in command.            leaf_migration = loader.graph.nodes[leaf_migration_node]            # Updated migration cannot be a squash migration, a dependency of            # another migration, and cannot be already applied.            if leaf_migration.replaces:                raise CommandError(                    f"Cannot update squash migration '{leaf_migration}'."                )            if leaf_migration_node in loader.applied_migrations:                raise CommandError(                    f"Cannot update applied migration '{leaf_migration}'."                )            depending_migrations = [                migration                for migration in loader.disk_migrations.values()                if leaf_migration_node in migration.dependencies            ]            if depending_migrations:                formatted_migrations = ", ".join(                    [f"'{migration}'" for migration in depending_migrations]                )                raise CommandError(                    f"Cannot update migration '{leaf_migration}' that migrations "                    f"{formatted_migrations} depend on."                )            # Build new migration.            for migration in app_migrations:                leaf_migration.operations.extend(migration.operations)                for dependency in migration.dependencies:                    if isinstance(dependency, SwappableTuple):                        if settings.AUTH_USER_MODEL == dependency.setting:                            leaf_migration.dependencies.append(                                ("__setting__", "AUTH_USER_MODEL")                            )                        else:                            leaf_migration.dependencies.append(dependency)                    elif dependency[0] != migration.app_label:                        leaf_migration.dependencies.append(dependency)            # Optimize migration.            optimizer = MigrationOptimizer()            leaf_migration.operations = optimizer.optimize(                leaf_migration.operations, app_label            )            # Update name.            previous_migration_path = MigrationWriter(leaf_migration).path            suggested_name = (                leaf_migration.name[:4] + "_" + leaf_migration.suggest_name()            )            if leaf_migration.name == suggested_name:                new_name = leaf_migration.name + "_updated"            else:                new_name = suggested_name            leaf_migration.name = new_name            # Register overridden migration.            new_changes[app_label] = [leaf_migration]            update_previous_migration_paths[app_label] = previous_migration_path        self.write_migration_files(new_changes, update_previous_migration_paths)    def write_migration_files(self, changes, update_previous_migration_paths=None):         """         Take a changes dict and write them out as migration files.         """


Question: if self.verbosity >= 1:                     # Display a relative path if it's below the current working                     # directory, or an absolute path otherwise.                    try:                        migration_string = os.path.relpath(writer.path)                    except ValueError:                        migration_string = writer.path                    if migration_string.startswith(".."):                        migration_string = writer.path                    migration_string = self.get_relative_path(writer.path)                     self.log("  %s\n" % self.style.MIGRATE_LABEL(migration_string))                     for operation in migration.operations:                         self.log("    - %s" % operation.describe())


Question: with open(writer.path, "w", encoding="utf-8") as fh:                         fh.write(migration_string)                         self.written_files.append(writer.path)<ins>                    if update_previous_migration_paths:                        prev_path = update_previous_migration_paths[app_label]                        rel_prev_path = self.get_relative_path(prev_path)                        if writer.needs_manual_porting:                            migration_path = self.get_relative_path(writer.path)                            self.log(                                self.style.WARNING(                                    f"Updated migration {migration_path} requires "                                    f"manual porting.\n"                                    f"Previous migration {rel_prev_path} was kept and "                                    f"must be deleted after porting functions manually."                                )                            )                        else:                            os.remove(prev_path)                            self.log(f"Deleted {rel_prev_path}")</ins>                 elif self.verbosity == 3:                     # Alternatively, makemigrations --dry-run --verbosity 3                     # will log the migrations rather than saving the file to


Question: self.log(writer.as_string())         run_formatters(self.written_files) <ins>    @staticmethod    def get_relative_path(path):        try:            migration_string = os.path.relpath(path)        except ValueError:            migration_string = path        if migration_string.startswith(".."):            migration_string = path        return migration_string</ins>     def handle_merge(self, loader, conflicts):         """         Handles merging together conflicted migrations interactively,


Question: from django.forms.widgets import CheckboxInput, HiddenInput, NumberInput from django.utils.functional import cached_property from django.utils.translation import gettext_lazy as _from django.utils.translation import ngettextfrom django.utils.translation import ngettext_lazy  __all__ = ("BaseFormSet", "formset_factory", "all_valid")

Allow overriding FormSet's error messages for too few and too many forms. Description Moving the messages to defaulterrormessages enables overriding them via the errormessages argument when instantiating the FormSet. I would prefer being able to override them via inlineformsetfactory, but that is for another day.


Question: "ManagementForm data is missing or has been tampered with. Missing fields: "             "%(field_names)s. You may need to file a bug report if the issue persists."         ),<ins>        "too_many_forms": ngettext_lazy(            "Please submit at most %(num)d form.",            "Please submit at most %(num)d forms.",            "num",        ),        "too_few_forms": ngettext_lazy(            "Please submit at least %(num)d form.",            "Please submit at least %(num)d forms.",            "num",        ),</ins>     }      template_name_div = "django/forms/formsets/div.html"


Question: TOTAL_FORM_COUNT             ] > self.absolute_max:                 raise ValidationError(                    ngettext(                        "Please submit at most %d form.",                        "Please submit at most %d forms.",                        self.max_num,                    )                    % self.max_num,                    self.error_messages["too_many_forms"] % {"num": self.max_num},                     code="too_many_forms",                 )             if (


Question: < self.min_num             ):                 raise ValidationError(                    ngettext(                        "Please submit at least %d form.",                        "Please submit at least %d forms.",                        self.min_num,                    )                    % self.min_num,                    self.error_messages["too_few_forms"] % {"num": self.min_num},                     code="too_few_forms",                 )             # Give self.clean() a chance to do cross-form validation.


Question: md5,     pbkdf2, )<ins>from django.utils.deprecation import RemovedInDjango50Warning</ins> from django.utils.module_loading import import_string from django.utils.translation import gettext_noop as _

Deprecate CryptPasswordHasher. Description CryptPasswordHasher was 15 years ago mainly to support legacy UNIX apps. It's almost undocumented, not recommended, and supported only on UNIX. Moreover crypt module was deprecated in Python 3.11 (see ). We should deprecate it in Django 4.1 and remove in Django 5.0.


Question: pass  <ins># RemovedInDjango50Warning.</ins> class CryptPasswordHasher(BasePasswordHasher):     """     Password hashing using UNIX crypt (not recommended)


Question: algorithm = "crypt"     library = "crypt" <ins>    def __init__(self, *args, **kwargs):        warnings.warn(            "django.contrib.auth.hashers.CryptPasswordHasher is deprecated.",            RemovedInDjango50Warning,            stacklevel=2,        )        super().__init__(*args, **kwargs)</ins>     def salt(self):         return get_random_string(2)


Question: import datetime import functools import inspect<ins>import warnings</ins> from collections import defaultdict from decimal import Decimal from uuid import UUID

Depracate passing False to OrderBy's nullsfirst and nullslast. Description Consider the following: In [11]: [tv.publishedat for tv in TemplateVersion.objects.orderby(F("publishedat").desc(nullsfirst=True))] Out[11]: [None, datetime.datetime(2022, 2, 25, 13, 0, 12, , tzinfo=<UTC), datetime.datetime(2022, 2, 21, 10, 18, 0, , tzinfo=<UTC)] In [12]: [tv.publishedat for tv in TemplateVersion.objects.orderby(F("publishedat").desc(nullsfirst=False))] Out[12]: [None, datetime.datetime(2022, 2, 25, 13, 0, 12, , tzinfo=<UTC), datetime.datetime(2022, 2, 21, 10, 18, 0, , tzinfo=<UTC)] In [13]: [tv.publishedat for tv in TemplateVersion.objects.orderby(F("publishedat").desc(nullslast=True))] Out[13]: [datetime.datetime(2022, 2, 25, 13, 0, 12, , tzinfo=<UTC), datetime.datetime(2022, 2, 21, 10, 18, 0, , tzinfo=<UTC), None] In [14]: [tv.publishedat for tv in TemplateVersion.objects.orderby(F("publishedat").desc(nullslast=False))] Out[14]: [None, datetime.datetime(2022, 2, 25, 13, 0, 12, , tzinfo=<UTC), datetime.datetime(2022, 2, 21, 10, 18, 0, , tzinfo=<UTC)] Observe how nulls_first=False still puts the nulls first. This happens because they both default False and when they are both False it lets the DB decide. This is surprising behaviour, it also makes changing the null positioning based on a variable more awkward than it needs to be. I think it would be better if they defaulted to None, let the DB decide when both are None and when one is not None do the ordering that implies.


Question: from django.db.models.constants import LOOKUP_SEP from django.db.models.query_utils import Q from django.utils.deconstruct import deconstructible<ins>from django.utils.deprecation import RemovedInDjango50Warning</ins> from django.utils.functional import cached_property from django.utils.hashable import make_hashable


Question: template = "%(expression)s %(ordering)s"     conditional = False     def __init__(        self, expression, descending=False, nulls_first=False, nulls_last=False    ):    def __init__(self, expression, descending=False, nulls_first=None, nulls_last=None):         if nulls_first and nulls_last:             raise ValueError("nulls_first and nulls_last are mutually exclusive")        if nulls_first is False or nulls_last is False:            # When the deprecation ends, replace with:            # raise ValueError(            #     "nulls_first and nulls_last values must be True or None."            # )            warnings.warn(                "Passing nulls_first=False or nulls_last=False is deprecated, use None "                "instead.",                RemovedInDjango50Warning,                stacklevel=2,            )         self.nulls_first = nulls_first         self.nulls_last = nulls_last         self.descending = descending


Question: def reverse_ordering(self):         self.descending = not self.descending        if self.nulls_first or self.nulls_last:            self.nulls_first = not self.nulls_first            self.nulls_last = not self.nulls_last        if self.nulls_first:            self.nulls_last = True            self.nulls_first = None        elif self.nulls_last:            self.nulls_first = True            self.nulls_last = None         return self      def asc(self):


Question: # Check for a redirect based on settings.PREPEND_WWW         host = request.get_host()        must_prepend = settings.PREPEND_WWW and host and not host.startswith("www.")        redirect_url = ("%s://www.%s" % (request.scheme, host)) if must_prepend else ""         # Check if a slash should be appended        if self.should_redirect_with_slash(request):            path = self.get_full_path_with_slash(request)        else:            path = request.get_full_path()        # Return a redirect if necessary        if redirect_url or path != request.get_full_path():            redirect_url += path            return self.response_redirect_class(redirect_url)        if settings.PREPEND_WWW and host and not host.startswith("www."):            # Check if we also need to append a slash so we can do it all            # with a single redirect. (This check may be somewhat expensive,            # so we only do it if we already know we're sending a redirect,            # or in process_response if we get a 404.)            if self.should_redirect_with_slash(request):                path = self.get_full_path_with_slash(request)            else:                path = request.get_full_path()            return self.response_redirect_class(f"{request.scheme}://www.{host}{path}")      def should_redirect_with_slash(self, request):         """

APPENDSLASH adds significant latency to all requests not ending in / (even if successful) Description Originally, APPENDSLASH worked by looking for 404 responses and replacing them with redirects, so as not to unnecessarily impact the performance of successful responses. However, commit in 1.9.5/1.10 changed this to check shouldredirectwithslash() on every request, resulting in a moderately expensive extra urlconf lookup for every request not ending with /, whether or not it succeeds as written. This performance impact was not considered in the commit message or the corresponding ticket , so I assume it was an oversight. That ticket asserted “This doesn't really make sense, since the two settings are not interdependent”, which is incorrect—performance was the reason for the interdependence. The overhead was found to be significant enough in Zulip to merit subclassing CommonMiddleware to skip it in certain conditions. Here’s a minimal test project with an exaggerated number of routes so the overhead can be easily observed. $ runserver $ wrk http://127.0.0.1:8000/url9999 Running 10s test @ http://127.0.0.1:8000/url9999 2 threads and 10 connections Thread Stats Avg Stdev Max +/- Stdev Latency 232.40ms 73.85ms 570.86ms 69.16% Req/Sec 21.70 9.47 40.00 63.35% 426 requests in 10.01s, 64.90KB read Requests/sec: 42.56 Transfer/sec: 6.48KB $ sed -i 's/# APPENDSLASH = False/APPENDSLASH = False/' $ wrk http://127.0.0.1:8000/url9999 Running 10s test @ http://127.0.0.1:8000/url9999 2 threads and 10 connections Thread Stats Avg Stdev Max +/- Stdev Latency 139.80ms 52.07ms 352.19ms 69.09% Req/Sec 36.46 12.23 60.00 58.12% 714 requests in 10.01s, 108.79KB read Requests/sec: 71.32 Transfer/sec: 10.87KB


Question: else:             from_model_state = from_state.models[app_label, self.model_name_lower]             old_index = from_model_state.get_index_by_name(self.old_name)<ins>        # Don't alter when the index name is not changed.        if old_index.name == self.new_name:            return</ins>          to_model_state = to_state.models[app_label, self.model_name_lower]         new_index = to_model_state.get_index_by_name(self.new_name)

RenameIndex() crashes when unnamed index is moving backward and forward. Description RenameIndex() should restore the old auto-generated name when an unnamed index for uniquetogether is moving backward. Now re-applying RenameIndex() crashes. For example: diff --git index .. a b class OperationTests(OperationTestBase): with connection.schemaeditor() as editor, self.assertNumQueries(0): operation.databasebackwards(applabel, editor, newstate, projectstate) self.assertIndexNameExists(tablename, "newponytestidx") 2991 # Re-apply renaming. 2992 with connection.schemaeditor() as editor: 2993 operation.databaseforwards(applabel, editor, projectstate, newstate) 2994 self.assertIndexNameExists(tablename, "newponytestidx") # Deconstruction. definition = operation.deconstruct() self.assertEqual(definition[0], "RenameIndex") crashes on PostgreSQL: relation "newponytest_idx" already exists


Question: try:  # method call (assuming no args required)                             current = current()                         except TypeError:                            signature = inspect.signature(current)                             try:                                signature.bind()                            except TypeError:  # arguments *were* required                                current = (                                    context.template.engine.string_if_invalid                                )  # invalid method call                                signature = inspect.signature(current)                            except ValueError:  # No signature found.                                current = context.template.engine.string_if_invalid                             else:                                raise                                try:                                    signature.bind()                                except TypeError:  # Arguments *were* required.                                    # Invalid method call.                                    current = context.template.engine.string_if_invalid                                else:                                    raise         except Exception as e:             template_name = getattr(context, "template_name", None) or "unknown"             logger.debug(

Templates crash when calling methods for built-in types. Description (last modified by Daniel) Found during a 2.2 -3.2 upgrade: Given a template: {{ foo }} where foo is non-existant, it returns nothing, empty. (That's good) {{ }} also empty (Also good) {% include '' with bar=foo %} and then in having: {{ }} results in File "/Users/daniel/src/django-bug-test/.v/lib/python3.8/site-", line 861, in resolvelookup signature = inspect.signature(current) File "", line 3093, in signature return Signature.fromcallable(obj, followwrapped=followwrapped) File "", line 2842, in fromcallable return signaturefromcallable(obj, sigcls=cls, File "", line 2296, in signaturefromcallable return signaturefrombuiltin(sigcls, obj, File "", line 2107, in signaturefrombuiltin raise ValueError("no signature found for builtin {!r}".format(func)) Exception Type: ValueError at / Exception Value: no signature found for builtin <built-in method count of str object at 0x1100ff2f0On django 2.2, this would not crash, but resulted in empty (as I expected). this seems to fix it for me: diff --git index .. --- +++ -913,15 +913,19 def resolvelookup(self, context): try: # method call (assuming no args required) current = current() except TypeError: signature = inspect.signature(current) try: () except TypeError: # arguments were required current = ( ) # invalid method call signature = inspect.signature(current) except ValueError: # python builtins might not have signature current = else: raise try: () except TypeError: # arguments were required current = ( ) # invalid method call else: raise except Exception as e: templatename = getattr(context, "templatename", None) or "unknown" (


Question: option_name = "index_together"     system_check_deprecated_details = {        "msg": (            "AlterIndexTogether is deprecated. Support for it (except in historical "            "migrations) will be removed in Django 5.1."        ),        "id": "migrations.W001",    }     def __init__(self, name, index_together):         super().__init__(name, index_together) diff --git a/django/db/models/options.py b/django/db/models/options.py-- a/django/db/models/options.py++ b/django/db/models/options.py

Deprecate Model.Meta.indextogether in favour of Model.Meta.indexes Description Anything that indextogether does can be done by indexes and the Index class.


Question: import bisect import copy import inspect<ins>import warnings</ins> from collections import defaultdict  from django.apps import apps


Question: from django.db.models import AutoField, Manager, OrderWrt, UniqueConstraint from django.db.models.query_utils import PathInfo from django.utils.datastructures import ImmutableList, OrderedSet<ins>from django.utils.deprecation import RemovedInDjango51Warning</ins> from django.utils.functional import cached_property from django.utils.module_loading import import_string from django.utils.text import camel_case_to_spaces, format_lazy


Question: self.unique_together = normalize_together(self.unique_together)             self.index_together = normalize_together(self.index_together)<ins>            if self.index_together:                warnings.warn(                    f"'index_together' is deprecated. Use 'Meta.indexes' in "                    f"{self.label!r} instead.",                    RemovedInDjango51Warning,                )</ins>             # App label/class name interpolation for names of constraints and             # indexes.             if not getattr(cls._meta, "abstract", False):


Question: import copy import inspect<ins>from functools import wraps</ins> from importlib import import_module  from django.db import router

inspect.signature() returns incorrect signature on manager methods. Description (last modified by Shiva Kumar) inspect.signature returns incorrect signature information when used on queryset methods import inspect from import models class Person(models.Model): name = models.CharField(maxlength=100) print(inspect.signature(Person.objects.bulkcreate)) ipython and jupyter seem to internally use inspect.signature to show documentation when using the <obj? command and they too show incorrect signature information: The issue is due to the code at  Although we are ensuring the decorated method has the right name and docstring on lines 87 and 88, complete metadata is not copied. The fix is to use instead of manually assigning name and docstring. wraps will take care of all the metadata and inspect.signature will return the expected output. If the bug is acknowledged please assign the ticket to me, I would like to raise a PR for this.


Question: @classmethod     def _get_queryset_methods(cls, queryset_class):         def create_method(name, method):            @wraps(method)             def manager_method(self, *args, **kwargs):                 return getattr(self.get_queryset(), name)(*args, **kwargs)             manager_method.__name__ = method.__name__            manager_method.__doc__ = method.__doc__             return manager_method          new_methods = {}


Question: # Deleted uniques         for fields in olds.difference(news):             self._delete_composed_index(                model, fields, {"unique": True}, self.sql_delete_unique                model,                fields,                {"unique": True, "primary_key": False},                self.sql_delete_unique,             )         # Created uniques         for field_names in news.difference(olds):

Cannot drop uniquetogether constraint on a single field with its own unique=True constraint Description I have an erroneous uniquetogether constraint on a model's primary key (uniquetogether = (('id',),)) that cannot be dropped by a migration. Apparently the migration tries to find all unique constraints on the column and expects there to be only one, but I've got two — the primary key and the uniquetogether constraint: Indexes: "foobarpkey" PRIMARY KEY, btree (id) "foobarid1c3b3088c74c3b17uniq" UNIQUE CONSTRAINT, btree (id) Database is PostgreSQL, if that makes any difference.


Question: exclude=meta_constraint_names | meta_index_names,             **constraint_kwargs,         )<ins>        if (            constraint_kwargs.get("unique") is True            and constraint_names            and self.connection.features.allows_multiple_constraints_on_same_fields        ):            # Constraint matching the unique_together name.            default_name = str(                self._unique_constraint_name(model._meta.db_table, columns, quote=False)            )            if default_name in constraint_names:                constraint_names = [default_name]</ins>         if len(constraint_names) != 1:             raise ValueError(                 "Found wrong number (%s) of constraints for %s(%s)"


Question: ):             return None         def create_unique_name(*args, **kwargs):            return self.quote_name(self._create_index_name(*args, **kwargs))         compiler = Query(model, alias_cols=False).get_compiler(             connection=self.connection         )         table = model._meta.db_table         columns = [field.column for field in fields]         if name is None:            name = IndexName(table, columns, "_uniq", create_unique_name)            name = self._unique_constraint_name(table, columns, quote=True)         else:             name = self.quote_name(name)         if condition or include or opclasses or expressions:


Question: include=self._index_include_sql(model, include),         ) <ins>    def _unique_constraint_name(self, table, columns, quote=True):        if quote:            def create_unique_name(*args, **kwargs):                return self.quote_name(self._create_index_name(*args, **kwargs))        else:            create_unique_name = self._create_index_name        return IndexName(table, columns, "_uniq", create_unique_name)</ins>     def _delete_unique_sql(         self,         model,


Question: "related object '%s'." % (operation_name, field.name)                     )                 elif getattr(self, field.attname) in field.empty_values:                    # Use pk from related object if it has been saved after                    # an assignment.                    setattr(self, field.attname, obj.pk)                    # Set related object if it has been saved after an                    # assignment.                    setattr(self, field.name, obj)                 # If the relationship's pk/to_field was changed, clear the                 # cached relationship.                 if getattr(obj, field.target_field.attname) != getattr(

Avoid unnecessary clear of cached reference Description (last modified by Barry Johnson) Consider this case of ORM models "Parent" and "Child", where Child has a foreign key reference to Parent (and the database can return generated IDs following insert operations): parent = Parent(name='parentobject') child = Child(parent=parent) () () print() The print statement will cause an unnecessary lazy read of the parent object. In the application where this behavior was first observed, the application was creating thousands of parent and child objects using bulkcreate(). The subsequent lazy reads occurred when creating log entries to record the action, and thousands of unwanted SELECT queries. Closed ticket solved a problem with potential data loss in this situation by essentially executing child.parentid = while preparing the child object to be saved. However, when the child's ForeignKeyDeferredAttrbute "parentid" changes value from None to the parent's ID, the child's internal cache containing the reference to "parent" is cleared. The subsequent reference to then must do a lazy read and reload parent from the database. A workaround to avoid this lazy read is to explicitly update both the "parentid" and "parent" cache entry by adding this non-intuitive statement: = after executing () But it appears that a simple change could avoid clearing the cache in this narrow case. Within Model.preparerelatedfieldsforsave(), replace setattr(self, field.attname, ) with setattr(self, , obj) This suggested code has -not- been tested. This change would set the associated "parent_id" attribute while ensuring that the currently referenced object remains referenced.


Question: def _generate_added_field(self, app_label, model_name, field_name):         field = self.to_state.models[app_label, model_name].get_field(field_name)        # Fields that are foreignkeys/m2ms depend on stuff        dependencies = []        # Adding a field always depends at least on its removal.        dependencies = [(app_label, model_name, field_name, False)]        # Fields that are foreignkeys/m2ms depend on stuff.         if field.remote_field and field.remote_field.model:             dependencies.extend(                 self._get_dependencies_for_foreign_key(

Models migration with change field foreign to many and deleting unique together. Description (last modified by Simon Charette) I have models like class Authors(models.Model): projectdataset = models.ForeignKey( ProjectDataSet, ondelete=models.PROTECT ) state = models.IntegerField() startdate = models.DateField() class Meta: uniquetogether = (('projectdataset', 'state', 'startdate'),) and class DataSet(models.Model): name = models.TextField(maxlength=50) class Project(models.Model): datasets = models.ManyToManyField( DataSet, through='ProjectDataSet', ) name = models.TextField(maxlength=50) class ProjectDataSet(models.Model): """ Cross table of data set and project """ dataset = models.ForeignKey(DataSet, ondelete=models.PROTECT) project = models.ForeignKey(Project, ondelete=models.PROTECT) class Meta: uniquetogether = (('dataset', 'project'),) when i want to change field projectdataset in Authors model from foreign key field to many to many field I must delete a uniquetogether, cause it can't be on many to many field. Then my model should be like: class Authors(models.Model): projectdataset = models.ManyToManyField( ProjectDataSet, ) state = models.IntegerField() startdate = models.DateField() But when I want to do a migrations. python3 makemigrations python3 migrate I have error: ValueError: Found wrong number (0) of constraints for appauthors(projectdataset, state, startdate) The database is on production, so I can't delete previous initial migrations, and this error isn't depending on database, cause I delete it and error is still the same. My solve is to first delete unique_together, then do a makemigrations and then migrate. After that change the field from foreign key to many to many field, then do a makemigrations and then migrate. But in this way I have 2 migrations instead of one. I attachment with this project, download it and then do makemigrations and then migrate to see this error.


Question: use_l10n = settings.USE_L10N     if use_l10n and lang is None:         lang = get_language()<ins>    format_type = str(format_type)  # format_type may be lazy.</ins>     cache_key = (format_type, lang)     try:         return _format_cache[cache_key]

should allow lazy parameter Description Commit [] (fixing ) triggered a regression when the date template filter (possibly others are affected too) receives a lazy string, like in somedate|date:('Y-m-d'). This fails with: TypeError: getattr(): attribute name must be string in


Question: from django.template.base import TokenType, render_value_in_context from django.template.defaulttags import token_kwargs from django.utils import translationfrom django.utils.safestring import SafeData, mark_safefrom django.utils.safestring import SafeData, SafeString, mark_safe  register = Library()

Blocktranslate asvar escapes variables, but stores the result as str instance, leading to double escaping Description In the docs, this snippet is given as an example usage of blocktranslate with the asvar argument (here:  {% blocktranslate asvar thetitle %}The title is {{ title }}.{% endblocktranslate %} <title{{ thetitle }}</title<meta name="description" content="{{ thetitle }}"However, this template is buggy when title is a string, which I'd argue is a common use case. title will be escaped when formatting the content of the blocktranslate block, but the "was escaped" information is discarded, and thetitle will be a str instance with escaped content. When later using the thetitle variable, it will be conditionally escaped. Since it is a str, it will be escaped, so control characters are escaped again, breaking their display on the final page. Minimal example to reproduce (can be put in any view): from django.template import Template, Context templatecontent = """ {% blocktranslate asvar thetitle %}The title is {{ title }}.{% endblocktranslate %} <title{{ thetitle }}</title<meta name="description" content="{{ thetitle }}"""" rendered = Template(templatecontent).render(Context({"title": "<& Title"})) assert "&lt;" not in rendered, "was escaped two times" I'd argue that blocktranslate should: Either assign a SafeString instance to prevent future escaping or not escape the variables used within the translation, and store them marked as unsafe (= as str instance)


Question: with translation.override(None):                 result = self.render(context, nested=True)         if self.asvar:            context[self.asvar] = result            context[self.asvar] = SafeString(result)             return ""         else:             return result


Question: )             # Get the request and check for basic issues.             request, error_response = self.create_request(scope, body_file)            if request is None:                await self.send_response(error_response, send)                return            # Get the response, using the async mode of BaseHandler.            response = await self.get_response_async(request)            response._handler_class = self.__class__         finally:             body_file.close()        if request is None:            await self.send_response(error_response, send)            return        # Get the response, using the async mode of BaseHandler.        response = await self.get_response_async(request)        response._handler_class = self.__class__         # Increase chunk size on file responses (ASGI servers handles low-level         # chunking).         if isinstance(response, FileResponse):

Request body is closed prematurely by the ASGI handler Description The following PR  introduces I/O operation on closed file., due to closing the ASGI request body before getting the response, e.g. accessing in a view.


Question: "scalar value but it's not implemented (#25287).": {                 "expressions.tests.FTimeDeltaTests.test_durationfield_multiply_divide",             },            "UPDATE ... ORDER BY syntax on MySQL/MariaDB does not support ordering by"            "related fields.": {                "update.tests.AdvancedTests."                "test_update_ordered_by_inline_m2m_annotation",                "update.tests.AdvancedTests.test_update_ordered_by_m2m_annotation",            },         }         if "ONLY_FULL_GROUP_BY" in self.connection.sql_mode:             skips.update(diff --git a/django/db/models/query.py b/django/db/models/query.py-- a/django/db/models/query.py++ b/django/db/models/query.py

() raises FieldError on queryset ordered by an annotated field. Description A FieldError results if I try to perform a custom a custom action on data in the admin interface IF that data is sorted by an annotated field. The action is obviously not acting on an annotated field as those fields do not exist in the database, but raises a FieldError saying it 'Cannot resolve keyword 'fieldthatdataiscurrentlysortedby' into field.' and then lists the choices for fields (all of which are the raw model fields, not the annotated fields in the custom queryset). My admin model: class PositionAdmin(admin.ModelAdmin): def init(self, args, kwargs): super().init(args, kwargs) def getqueryset(self, request): qs = super(PositionAdmin, self).getqueryset(request).prefetchrelated( 'orders').selectrelated( 'portfolio', 'currentposition').selectrelated( 'currentposition').annotate( Count('orders', distinct=True), Count('shortrequests'), totalpositionvalue=ExpressionWrapper( F('price') F('quantity'), outputfield=IntegerField()), diffshares=Coalesce( F('quantity') F('currentpositionquantity'), F('quantity')), diffvalue=ExpressionWrapper( F('diffshares') F('price'), outputfield=IntegerField()), orderswitherrors=Count( Case(When(~Q(orderserror=''), then=1))), nonacceptedorders=Count( Case(When(Q(ordersstatus=''), then=1))), ordertoday=Count( Case(When( orderscreatedat_gte=brokeragetodaystart(), then=1)))) return qs # Custom Action def approvepositionfortrading(self, request, queryset): try: numposapproved = (approved=True) except FieldError: self.messageuser( request, "You cannot perform actions when you have sorted by " "this column. Please remove your column sortings and then try " "your action again.", level=messages.ERROR) else: if numposapproved == 1: messagebit = "1 position was was" else: messagebit = "%s positions were" % numposapproved self.messageuser( request, "%s successfully approved for trading." % messagebit) I had to write code to handle the error so that the user will know how to proceed. However, this seems like bad behavior.


Question: "scalar value but it's not implemented (#25287).": {                 "expressions.tests.FTimeDeltaTests.test_durationfield_multiply_divide",             },            "UPDATE ... ORDER BY syntax on MySQL/MariaDB does not support ordering by"            "related fields.": {                "update.tests.AdvancedTests."                "test_update_ordered_by_inline_m2m_annotation",                "update.tests.AdvancedTests.test_update_ordered_by_m2m_annotation",            },         }         if "ONLY_FULL_GROUP_BY" in self.connection.sql_mode:             skips.update(diff --git a/django/db/models/query.py b/django/db/models/query.py-- a/django/db/models/query.py++ b/django/db/models/query.py

Django can successfully perform that action (changing the 'Approved' field to 'True') on the data if it is sorted by any core model field, or if sorted by no field at all. However, if the data is sorted by a field resulting from the annotation of my queryset, then I get the FieldError when I try to perform the action. This seems like a bug.


Question: self._for_write = True         query = self.query.chain(sql.UpdateQuery)         query.add_update_values(kwargs)<ins>        # Inline annotations in order_by(), if possible.        new_order_by = []        for col in query.order_by:            if annotation := query.annotations.get(col):                if getattr(annotation, "contains_aggregate", False):                    raise exceptions.FieldError(                        f"Cannot update when ordering by an aggregate: {annotation}"                    )                new_order_by.append(annotation)            else:                new_order_by.append(col)        query.order_by = tuple(new_order_by)</ins>         # Clear any annotations so that they won't be present in subqueries.         query.annotations = {}         with transaction.mark_for_rollback_on_error(using=self.db):


Question: self._for_write = True         query = self.query.chain(sql.UpdateQuery)         query.add_update_values(kwargs)<ins>        # Inline annotations in order_by(), if possible.        new_order_by = []        for col in query.order_by:            if annotation := query.annotations.get(col):                if getattr(annotation, "contains_aggregate", False):                    raise exceptions.FieldError(                        f"Cannot update when ordering by an aggregate: {annotation}"                    )                new_order_by.append(annotation)            else:                new_order_by.append(col)        query.order_by = tuple(new_order_by)</ins>         # Clear any annotations so that they won't be present in subqueries.         query.annotations = {}         with transaction.mark_for_rollback_on_error(using=self.db):


Question: not in INSTALLED_APPS. """ import unicodedata<ins>import warnings</ins>  from django.contrib.auth import password_validation from django.contrib.auth.hashers import (

Deprecate makerandompassword(). Description (last modified by Carlton Gibson) It's unused since , see also Carlton's comment.


Question: ) from django.db import models from django.utils.crypto import get_random_string, salted_hmac<ins>from django.utils.deprecation import RemovedInDjango51Warning</ins> from django.utils.translation import gettext_lazy as _


Question: allowed_chars. The default value of allowed_chars does not have "I" or         "O" or letters and digits that look similar -- just to avoid confusion.         """<ins>        warnings.warn(            "BaseUserManager.make_random_password() is deprecated.",            category=RemovedInDjango51Warning,            stacklevel=2,        )</ins>         return get_random_string(length, allowed_chars)      def get_by_natural_key(self, username):