Question: Serialization dependency sorting disallows circular references unneccesarily. Description The core.serialization.sortdependencies() function takes a list of apps and/or models, and resolves this into a sorted flat list of models, ready to be serialized in that order. This function is intended to make natural foreign keys work, by serializing models referenced by a natural key before the referencing models. When deserializing, this guarantees that natural key references can be resolved, because there are no "forward references". Furthermore, when a circular reference using natural keys is present, this function raises an exception (e.g. "Can't resolve dependencies for someapp.SomeModel in serialized app list") and prevents serialization from completing, since there is no way to guarantee a model ordering that will have no forward references. Note that this ordering is only needed when natural keys are involved, since data is intended to be loaded in a transaction without constraint checks, so numerical foreign keys can be in the wrong order, as long as all referenced data is present at the end of the transaction. This does not work with natural keys, since those are resolved by Python code that needs the referenced objects present in the database to resolve them. However, this sorting is not actually strictly necessary in all cases where it is applied. When circular references are involved, this then actually prevents serialization for no good reason. In particular, this is the case: When running dumpdata without natural keys enabled (which is the default). Even though natural keys might be defined in the models (which causes the sorting and exception), no natural keys will be present in the dumped data, so no ordering is needed. When dumping data intended for loading with loaddata (which I think is the primary usecase for dumpdata?). loaddata will (since 17 months ago in v2.2, see ) automatically handle forward references by deferring setting fields that reference natural keys that are not yet. In this case, sorting is still useful, to prevent forward references where possible, but when there are circular references, it is acceptable to ignore some dependencies rather than preventing serialization from happening alltogether. When serializing data for tests for serializedrollback=True (in ). This is a serialization that does not use natural keys, so no ordering is needed at all. Note that this serialization happens always (unlike deserialization only happens with serializedrollback=True), so AFAIU this effectively prevents any tests from working on a database with circular references with natural keys defined. The fix for these issues seems to be rather simple: For dumpdata without usenaturalforeignkeys, skip the ordering and just serialize all models in arbitrary order. AFAICS usenaturalprimarykeys is not relevant here, since that only controls omitting the numerical primary key. For dumpdata with usenaturalforeignkeys, do the ordering but do not bail out when there are circular references (instead just ignore some dependencies and produce a best-effort ordering). For test database serialization, also skip the ordering and serialize in arbitrary order. Note that this would remove two of the three calls to sortdependencies() and allow loops in the last remaining instance. This means that sort_dependencies could be modified to allow loops unconditionally, or we could add an argument and default to disallowing loops in case any code outside of django is using this function? Note that is a related, but different issue, concerning the deserialization of data in testcases. I've been working on fixing this and that related issue today and have a basic version working, with testcases (which proved to be quite a challenge, since testing the test subsystem is a bit tricky...). I'll do some additional testing and cleanup and submit a PR soon. Also note that the circular-reference exception was already disabled for self-referencing models in . The fix for that issue simply ignores self-referencing models for sorting, without taking any additional measures to sort instances to prevent problems in deserialization (this code was when the deferred deserialization did not exist yet), so I wonder how much value this exception still has.

_serializers = serializers  def sort_dependencies(app_list):def sort_dependencies(app_list, allow_cycles=False):     """Sort a list of (app_config, models) pairs into a single list of models.      The single list of models is sorted so that any model with a natural key     is serialized before a normal model, and any model with a natural key     dependency has it's dependencies serialized first.    If allow_cycles is True, return the best-effort ordering that will respect    most of dependencies but ignore some of them to break the cycles.     """     # Process the list of models, and get the list of dependencies     model_dependencies = []


Question: Serialization dependency sorting disallows circular references unneccesarily. Description The core.serialization.sortdependencies() function takes a list of apps and/or models, and resolves this into a sorted flat list of models, ready to be serialized in that order. This function is intended to make natural foreign keys work, by serializing models referenced by a natural key before the referencing models. When deserializing, this guarantees that natural key references can be resolved, because there are no "forward references". Furthermore, when a circular reference using natural keys is present, this function raises an exception (e.g. "Can't resolve dependencies for someapp.SomeModel in serialized app list") and prevents serialization from completing, since there is no way to guarantee a model ordering that will have no forward references. Note that this ordering is only needed when natural keys are involved, since data is intended to be loaded in a transaction without constraint checks, so numerical foreign keys can be in the wrong order, as long as all referenced data is present at the end of the transaction. This does not work with natural keys, since those are resolved by Python code that needs the referenced objects present in the database to resolve them. However, this sorting is not actually strictly necessary in all cases where it is applied. When circular references are involved, this then actually prevents serialization for no good reason. In particular, this is the case: When running dumpdata without natural keys enabled (which is the default). Even though natural keys might be defined in the models (which causes the sorting and exception), no natural keys will be present in the dumped data, so no ordering is needed. When dumping data intended for loading with loaddata (which I think is the primary usecase for dumpdata?). loaddata will (since 17 months ago in v2.2, see ) automatically handle forward references by deferring setting fields that reference natural keys that are not yet. In this case, sorting is still useful, to prevent forward references where possible, but when there are circular references, it is acceptable to ignore some dependencies rather than preventing serialization from happening alltogether. When serializing data for tests for serializedrollback=True (in ). This is a serialization that does not use natural keys, so no ordering is needed at all. Note that this serialization happens always (unlike deserialization only happens with serializedrollback=True), so AFAIU this effectively prevents any tests from working on a database with circular references with natural keys defined. The fix for these issues seems to be rather simple: For dumpdata without usenaturalforeignkeys, skip the ordering and just serialize all models in arbitrary order. AFAICS usenaturalprimarykeys is not relevant here, since that only controls omitting the numerical primary key. For dumpdata with usenaturalforeignkeys, do the ordering but do not bail out when there are circular references (instead just ignore some dependencies and produce a best-effort ordering). For test database serialization, also skip the ordering and serialize in arbitrary order. Note that this would remove two of the three calls to sortdependencies() and allow loops in the last remaining instance. This means that sort_dependencies could be modified to allow loops unconditionally, or we could add an argument and default to disallowing loops in case any code outside of django is using this function? Note that is a related, but different issue, concerning the deserialization of data in testcases. I've been working on fixing this and that related issue today and have a basic version working, with testcases (which proved to be quite a challenge, since testing the test subsystem is a bit tricky...). I'll do some additional testing and cleanup and submit a PR soon. Also note that the circular-reference exception was already disabled for self-referencing models in . The fix for that issue simply ignores self-referencing models for sorting, without taking any additional measures to sort instances to prevent problems in deserialization (this code was when the deferred deserialization did not exist yet), so I wonder how much value this exception still has.

else:                 skipped.append((model, deps))         if not changed:            raise RuntimeError(                "Can't resolve dependencies for %s in serialized app list." %                ', '.join(                    model._meta.label                    for model, deps in sorted(skipped, key=lambda obj: obj[0].__name__)            if allow_cycles:                # If cycles are allowed, add the last skipped model and ignore                # its dependencies. This could be improved by some graph                # analysis to ignore as few dependencies as possible.                model, _ = skipped.pop()                model_list.append(model)            else:                raise RuntimeError(                    "Can't resolve dependencies for %s in serialized app list."                    % ', '.join(                        model._meta.label                        for model, deps in sorted(skipped, key=lambda obj: obj[0].__name__)                    ),                 )            )         model_dependencies = skipped      return model_list


Question: DISTINCT with GROUPCONCAT() and multiple expressions raises NotSupportedError on SQLite. Description Contrary to what is suggested in lines 60-64 of , SQLite does support DISTINCT on aggregate functions. One such example is GROUPCONCAT, which is quite similar to PostgreSQL's STRINGAGG. I can't find any canonical links which provide a useful explanation of GROUPCONCAT, but this should be good enough:  I came across this issue when trying to create my own GroupConcat function subclassing Aggregate (similar to the StringAgg implementation from postgres) and noticed that skipping the check in would allow my queries to run as advertised. My code for GroupConcat is: from import Value from import Aggregate class GroupConcat(Aggregate): function = 'GROUPCONCAT' template = '%(function)s(%(distinct)s %(expressions)s)' allowdistinct = True def init(self, expression, delimiter=None, extra): if delimiter is not None: self.allowdistinct = False delimiterexpr = Value(str(delimiter)) super().init(expression, delimiterexpr, extra) else: super().init(expression, extra) def assqlite(self, compiler, connection, extracontext): return super().assql( compiler, connection, function=self.function, template=self.template, **extracontext ) For the record, as the code above suggests, a separate issue is that GROUP_CONCAT only allows DISTINCT when a delimiter isn't specified. After some discussion on #django, an argument was raised in favor of changing the message to say "Django doesn't support...", but I would imagine that skipping the check entirely would simply result in an OperationalError for malformed queries while still allowing users to extend the ORM as needed.

'aggregations on date/time fields in sqlite3 '                             'since date/time is saved as text.'                         )        if isinstance(expression, models.Aggregate) and len(expression.source_expressions) > 1:        if (            isinstance(expression, models.Aggregate) and            expression.distinct and            len(expression.source_expressions) > 1        ):             raise NotSupportedError(                 "SQLite doesn't support DISTINCT on aggregate functions "                 "accepting multiple arguments."


Question: Admin datehierarchy filter by month displays an extra day at timezone boundary. Description (last modified by Lavrenov Ivan) When I authorized by user with not-UTC timezone, like America/LosAngeles , and open filter by date in month, I see one extra day, that follows to the first day of the previous month

from django.contrib.admin.templatetags.admin_urls import add_preserved_filters from django.contrib.admin.utils import (    display_for_field, display_for_value, label_for_field, lookup_field,    display_for_field, display_for_value, get_fields_from_path,    label_for_field, lookup_field, ) from django.contrib.admin.views.main import (     ALL_VAR, ORDER_VAR, PAGE_VAR, SEARCH_VAR,


Question: Admin datehierarchy filter by month displays an extra day at timezone boundary. Description (last modified by Lavrenov Ivan) When I authorized by user with not-UTC timezone, like America/LosAngeles , and open filter by date in month, I see one extra day, that follows to the first day of the previous month

from django.template.loader import get_template from django.templatetags.static import static from django.urls import NoReverseMatchfrom django.utils import formatsfrom django.utils import formats, timezone from django.utils.html import format_html from django.utils.safestring import mark_safe from django.utils.text import capfirst


Question: Admin datehierarchy filter by month displays an extra day at timezone boundary. Description (last modified by Lavrenov Ivan) When I authorized by user with not-UTC timezone, like America/LosAngeles , and open filter by date in month, I see one extra day, that follows to the first day of the previous month

"""     if cl.date_hierarchy:         field_name = cl.date_hierarchy<ins>        field = get_fields_from_path(cl.model, field_name)[-1]        if isinstance(field, models.DateTimeField):            dates_or_datetimes = 'datetimes'            qs_kwargs = {'is_dst': True}        else:            dates_or_datetimes = 'dates'            qs_kwargs = {}</ins>         year_field = '%s__year' % field_name         month_field = '%s__month' % field_name         day_field = '%s__day' % field_name


Question: Admin datehierarchy filter by month displays an extra day at timezone boundary. Description (last modified by Lavrenov Ivan) When I authorized by user with not-UTC timezone, like America/LosAngeles , and open filter by date in month, I see one extra day, that follows to the first day of the previous month

# select appropriate start level             date_range = cl.queryset.aggregate(first=models.Min(field_name),                                                last=models.Max(field_name))<ins>            if dates_or_datetimes == 'datetimes':                date_range = {                    k: timezone.localtime(v) if timezone.is_aware(v) else v                    for k, v in date_range.items()                }</ins>             if date_range['first'] and date_range['last']:                 if date_range['first'].year == date_range['last'].year:                     year_lookup = date_range['first'].year


Question: Admin datehierarchy filter by month displays an extra day at timezone boundary. Description (last modified by Lavrenov Ivan) When I authorized by user with not-UTC timezone, like America/LosAngeles , and open filter by date in month, I see one extra day, that follows to the first day of the previous month

'choices': [{'title': capfirst(formats.date_format(day, 'MONTH_DAY_FORMAT'))}]             }         elif year_lookup and month_lookup:            days = getattr(cl.queryset, 'dates')(field_name, 'day')            days = getattr(cl.queryset, dates_or_datetimes)(field_name, 'day', **qs_kwargs)             return {                 'show': True,                 'back': {


Question: Admin datehierarchy filter by month displays an extra day at timezone boundary. Description (last modified by Lavrenov Ivan) When I authorized by user with not-UTC timezone, like America/LosAngeles , and open filter by date in month, I see one extra day, that follows to the first day of the previous month

} for day in days]             }         elif year_lookup:            months = getattr(cl.queryset, 'dates')(field_name, 'month')            months = getattr(cl.queryset, dates_or_datetimes)(field_name, 'month', **qs_kwargs)             return {                 'show': True,                 'back': {


Question: Admin datehierarchy filter by month displays an extra day at timezone boundary. Description (last modified by Lavrenov Ivan) When I authorized by user with not-UTC timezone, like America/LosAngeles , and open filter by date in month, I see one extra day, that follows to the first day of the previous month

} for month in months]             }         else:            years = getattr(cl.queryset, 'dates')(field_name, 'year')            years = getattr(cl.queryset, dates_or_datetimes)(field_name, 'year', **qs_kwargs)             return {                 'show': True,                 'back': None,


Question: Inherited model doesn't correctly order by "-pk" when specified on Parent.Meta.ordering Description Given the following model definition: from import models class Parent(models.Model): class Meta: ordering = ["-pk"] class Child(Parent): pass Querying the Child class results in the following: print(().query) SELECT "myappparent"."id", "myappchild"."parentptrid" FROM "myappchild" INNER JOIN "myappparent" ON ("myappchild"."parentptrid" = "myappparent"."id") ORDER BY "myapp_parent"."id" ASC The query is ordered ASC but I expect the order to be DESC.

field, targets, alias, joins, path, opts, transform_function = self._setup_joins(pieces, opts, alias)          # If we get to this point and the field is a relation to another model,        # append the default ordering for that model unless the attribute name        # of the field is specified.        if field.is_relation and opts.ordering and getattr(field, 'attname', None) != name:        # append the default ordering for that model unless it is the pk        # shortcut or the attribute name of the field that is specified.        if field.is_relation and opts.ordering and getattr(field, 'attname', None) != name and name != 'pk':             # Firstly, avoid infinite loops.             already_seen = already_seen or set()             join_tuple = tuple(getattr(self.query.alias_map[j], 'join_cols', None) for j in joins)


Question: fields.E310-E311 should take into account UniqueConstraints without conditions. Description Hello, I'm trying to create migration with this kind of model. class AppUsers(models.Model): name = models.CharField(...) uid = models.CharField(...) source = models.ForeignKey(...) class Meta: constraints = [models.UniqueConstraint(fields=['uid', 'source'], name='appusersuniqueness')] When I start makemigrations command in I've fields.E310  error It says that I should add uniquetogether field in Meta options: (fields.E310) No subset of the fields 'uid', 'source' on model 'AppUsers' is unique. HINT: Add unique=True on any of those fields or add at least a subset of them to a uniquetogether constraint. If I change Meta options to uniquetogether constraint migration passes with no errors. class AppUsers(models.Model): name = models.CharField(...) uid = models.CharField(...) source = models.ForeignKey(...) class Meta: uniquetogether = [['uid', 'source']] As mentioned in docs  unique_together may be deprecated in the future. So I think nobody wants to face this issue when this will be deprecated :) Thanks, Pavel

frozenset(ut)             for ut in self.remote_field.model._meta.unique_together         })<ins>        unique_foreign_fields.update({            frozenset(uc.fields)            for uc in self.remote_field.model._meta.total_unique_constraints        })</ins>         foreign_fields = {f.name for f in self.foreign_related_fields}         has_unique_constraint = any(u <= foreign_fields for u in unique_foreign_fields)


Question: fields.E310-E311 should take into account UniqueConstraints without conditions. Description Hello, I'm trying to create migration with this kind of model. class AppUsers(models.Model): name = models.CharField(...) uid = models.CharField(...) source = models.ForeignKey(...) class Meta: constraints = [models.UniqueConstraint(fields=['uid', 'source'], name='appusersuniqueness')] When I start makemigrations command in I've fields.E310  error It says that I should add uniquetogether field in Meta options: (fields.E310) No subset of the fields 'uid', 'source' on model 'AppUsers' is unique. HINT: Add unique=True on any of those fields or add at least a subset of them to a uniquetogether constraint. If I change Meta options to uniquetogether constraint migration passes with no errors. class AppUsers(models.Model): name = models.CharField(...) uid = models.CharField(...) source = models.ForeignKey(...) class Meta: uniquetogether = [['uid', 'source']] As mentioned in docs  unique_together may be deprecated in the future. So I think nobody wants to face this issue when this will be deprecated :) Thanks, Pavel

"No subset of the fields %s on model '%s' is unique."                     % (field_combination, model_name),                     hint=(                        "Add unique=True on any of those fields or add at "                        "least a subset of them to a unique_together constraint."                        'Mark a single field as unique=True or add a set of '                        'fields to a unique constraint (via unique_together '                        'or a UniqueConstraint (without condition) in the '                        'model Meta.constraints).'                     ),                     obj=self,                     id='fields.E310',


Question: fields.E310-E311 should take into account UniqueConstraints without conditions. Description Hello, I'm trying to create migration with this kind of model. class AppUsers(models.Model): name = models.CharField(...) uid = models.CharField(...) source = models.ForeignKey(...) class Meta: constraints = [models.UniqueConstraint(fields=['uid', 'source'], name='appusersuniqueness')] When I start makemigrations command in I've fields.E310  error It says that I should add uniquetogether field in Meta options: (fields.E310) No subset of the fields 'uid', 'source' on model 'AppUsers' is unique. HINT: Add unique=True on any of those fields or add at least a subset of them to a uniquetogether constraint. If I change Meta options to uniquetogether constraint migration passes with no errors. class AppUsers(models.Model): name = models.CharField(...) uid = models.CharField(...) source = models.ForeignKey(...) class Meta: uniquetogether = [['uid', 'source']] As mentioned in docs  unique_together may be deprecated in the future. So I think nobody wants to face this issue when this will be deprecated :) Thanks, Pavel

model_name = self.remote_field.model.__name__             return [                 checks.Error(                    "'%s.%s' must set unique=True because it is referenced by "                    "'%s.%s' must be unique because it is referenced by "                     "a foreign key." % (model_name, field_name),                    hint=(                        'Add unique=True to this field or add a '                        'UniqueConstraint (without condition) in the model '                        'Meta.constraints.'                    ),                     obj=self,                     id='fields.E311',                 )


Question: system checks: admin.E002 could provide a hint but doesn't Description Currently the output is: myapp.MyCustomUserModel: (auth.E002) The field named as the 'USERNAMEFIELD' for a custom user model must not be included in 'REQUIREDFIELDS'. because I accidentally had: USERNAMEFIELD = "email" EMAILFIELD = "email" REQUIREDFIELDS = (USERNAMEFIELD, "full_name",) Ignoring the fact that Django knows it's wrong to have it in there, and could easily just skip it or warn if it's not set, it doesn't make use of the hints available in system checks. I'd like to suggest that a hint could be provided which says which field it's moaning about, something like (open to better wording): HINT: Your username field is currently set to "email", you should remove "email" from your required fields definition. It's a stupidly little thing, but having not had to make a custom user from scratch for a while, my eyes glazed over the not in must not be and I was confused for all of 2 minutes before actually reading it properly.

checks.Error(                 "The field named as the 'USERNAME_FIELD' "                 "for a custom user model must not be included in 'REQUIRED_FIELDS'.",<ins>                hint=(                    "The 'USERNAME_FIELD' is currently set to '%s', you "                    "should remove '%s' from the 'REQUIRED_FIELDS'."                    % (cls.USERNAME_FIELD, cls.USERNAME_FIELD)                ),</ins>                 obj=cls,                 id='auth.E002',             )


Question: MultiPartParser support double quotes Description Although the rfc2231 document does not indicate that values can be wrapped in double quotes. However, some third-party tools wrap the value in double quotation marks when wrapping HTTP requests (such as the filename of the file uploaded by PostmanCanary). This results in double quotes for the filename at the end of Django parsing. Postman request body: ---------------------------- Content-Disposition: form-data; name="file"; filename=".txt"; filename*="UTF-8''" Content-Type: text/plain test ------------------------------ django got filename is .txt" not .txt This is not a bug of Django. But I suggest Django do some compatibility processing. CPython is also compatible with this situation.

if p.count(b"'") == 2:                     has_encoding = True             value = p[i + 1:].strip()            if has_encoding:                encoding, lang, value = value.split(b"'")                value = unquote(value.decode(), encoding=encoding.decode())             if len(value) >= 2 and value[:1] == value[-1:] == b'"':                 value = value[1:-1]                 value = value.replace(b'\\\\', b'\\').replace(b'\\"', b'"')            if has_encoding:                encoding, lang, value = value.split(b"'")                value = unquote(value.decode(), encoding=encoding.decode())             pdict[name] = value     return key, pdict


Question: () incorrectly formats large/tiny floats in scientific notation Description (last modified by Tim Graham) For floats with values larger than 1e16 or smaller than 1e-5, their string representation uses scientific notation in Python, which causes to return an erroneous output. from import format format(0., '.', 2) '9e-19.00' format(1e16, '.', 2, thousandsep=',', grouping=3, forcegrouping=True) '1e,+16.00' This is similar to but that was only fixed for Decimal types.

return mark_safe(number)     # sign     sign = ''<ins>    # Treat potentially very large/small floats as Decimals.    if isinstance(number, float) and 'e' in str(number).lower():        number = Decimal(str(number))</ins>     if isinstance(number, Decimal):          if decimal_pos is not None:


Question: Child model updates parent model with empty fields making an extra query in multi-inheritance when parent model has custom PK Description While creating a new model object (using multi-inheritance model =Child(Parent)), Django does an extra update query setting parent model fields to empty values. This situation occurs only if we define a custom primary key in a parent model (eg. as an UUID field). An example without custom primary key (correct behavior): class Parent(models.Model): title = models.TextField() class Child(Parent): body = models.TextField() () INTO "appparent" ("title") VALUES ('') RETURNING "appparent"."id" INTO "appchild" ("parentptrid", "body") VALUES (1, '') An example with custom primary key (incorrect behavior): class Parent(models.Model): id = models.UUIDField( primarykey=True, default=uuid.uuid4, editable=False ) title = models.TextField() class Child(Parent): body = models.TextField() () "appparent" SET "title" = '' WHERE "appparent"."id" = '-ae7b-48a6-a2e0-'::uuid INTO "appparent" ("id", "title") VALUES ('-ae7b-48a6-a2e0-'::uuid, '') INTO "appchild" ("parentptrid", "body") VALUES ('-ae7b-48a6-a2e0-'::uuid, '') Python 3.6, PostgreSQL 9.6

not raw and             not force_insert and             self._state.adding and            self._meta.pk.default and            self._meta.pk.default is not NOT_PROVIDED            meta.pk.default and            meta.pk.default is not NOT_PROVIDED         ):             force_insert = True         # If possible, try an UPDATE. If that doesn't update anything, do an INSERT.


Question: Wrong hint about recursive relationship. Description (last modified by Matheus Cunha Motta) When there's more than 2 ForeignKeys in an intermediary model of a m2m field and no through_fields have been set, Django will show an error with the following hint: hint=( 'If you want to create a recursive relationship, ' 'use ForeignKey("%s", symmetrical=False, through="%s").' But 'symmetrical' and 'through' are m2m keyword arguments, not ForeignKey. This was probably a small mistake where the developer thought ManyToManyField but typed ForeignKey instead. And the symmetrical=False is an outdated requirement to recursive relationships with intermediary model to self, not required since 3.0. I'll provide a PR with a proposed correction shortly after. Edit: fixed description.

"through_fields keyword argument.") % (self, from_model_name),                             hint=(                                 'If you want to create a recursive relationship, '                                'use ForeignKey("%s", symmetrical=False, through="%s").'                                'use ManyToManyField("%s", through="%s").'                             ) % (                                 RECURSIVE_RELATIONSHIP_CONSTANT,                                 relationship_model_name,


Question: Wrong hint about recursive relationship. Description (last modified by Matheus Cunha Motta) When there's more than 2 ForeignKeys in an intermediary model of a m2m field and no through_fields have been set, Django will show an error with the following hint: hint=( 'If you want to create a recursive relationship, ' 'use ForeignKey("%s", symmetrical=False, through="%s").' But 'symmetrical' and 'through' are m2m keyword arguments, not ForeignKey. This was probably a small mistake where the developer thought ManyToManyField but typed ForeignKey instead. And the symmetrical=False is an outdated requirement to recursive relationships with intermediary model to self, not required since 3.0. I'll provide a PR with a proposed correction shortly after. Edit: fixed description.

"through_fields keyword argument." % (self, to_model_name),                             hint=(                                 'If you want to create a recursive relationship, '                                'use ForeignKey("%s", symmetrical=False, through="%s").'                                'use ManyToManyField("%s", through="%s").'                             ) % (                                 RECURSIVE_RELATIONSHIP_CONSTANT,                                 relationship_model_name,


Question: makemessages doesn't provide feedback when no locale is specified Description (last modified by Cristóbal Mackenzie) makemessages requires that one of three flags be passed to specify locales for message building: --locale to explicitly specify locales, --exclude to specify locales to exclude, or --all to build message files for all locales. When non of these flags are present, the command doesn't show any errors for the user. According to the source code, it should raise CommandError, but that never happens because of a bug in an if statement that checks if a locale has been specified. I've already fixed this in my fork and have submitted a small PR.  Please point out if there are any other necessary steps to move this forward. Thanks!

exts = extensions or ['html', 'txt', 'py']         self.extensions = handle_extensions(exts)         if (locale is None and not exclude and not process_all) or self.domain is None:        if (not locale and not exclude and not process_all) or self.domain is None:             raise CommandError(                 "Type '%s help %s' for usage information."                 % (os.path.basename(sys.argv[0]), sys.argv[1])


Question: Logout link should be protected Description There is a logout link in admin app. It is link, not a form. Therefore it is not CSRF-protected. Probably it is not so important to protect logout from CSRF attack, because this fact cannot be used to do anything harmful. So this is just a request for purity. Another reason is that GET request should never change invernal state of the system.

<ins>import warnings</ins> from urllib.parse import urlparse, urlunparse  from django.conf import settings


Question: Logout link should be protected Description There is a logout link in admin app. It is link, not a form. Therefore it is not CSRF-protected. Probably it is not so important to protect logout from CSRF attack, because this fact cannot be used to do anything harmful. So this is just a request for purity. Another reason is that GET request should never change invernal state of the system.

from django.shortcuts import resolve_url from django.urls import reverse_lazy from django.utils.decorators import method_decorator<ins>from django.utils.deprecation import RemovedInDjango50Warning</ins> from django.utils.http import url_has_allowed_host_and_scheme, urlsafe_base64_decode from django.utils.translation import gettext_lazy as _ from django.views.decorators.cache import never_cache


Question: Logout link should be protected Description There is a logout link in admin app. It is link, not a form. Therefore it is not CSRF-protected. Probably it is not so important to protect logout from CSRF attack, because this fact cannot be used to do anything harmful. So this is just a request for purity. Another reason is that GET request should never change invernal state of the system.

Log out the user and display the 'You are logged out' message.     """     # RemovedInDjango50Warning: when the deprecation ends, remove "get" and    # "head" from http_method_names.    http_method_names = ["get", "head", "post", "options"]     next_page = None     redirect_field_name = REDIRECT_FIELD_NAME     template_name = "registration/logged_out.html"     extra_context = None     # RemovedInDjango50Warning: when the deprecation ends, move    # @method_decorator(csrf_protect) from post() to dispatch().     @method_decorator(never_cache)     def dispatch(self, request, *args, **kwargs):        if request.method.lower() == "get":            warnings.warn(                "Log out via GET requests is deprecated and will be removed in Django "                "5.0. Use POST requests for logging out.",                RemovedInDjango50Warning,            )        return super().dispatch(request, *args, **kwargs)    @method_decorator(csrf_protect)    def post(self, request, *args, **kwargs):        """Logout may be done via POST."""         auth_logout(request)         next_page = self.get_next_page()         if next_page:             # Redirect to this page until the session has been cleared.             return HttpResponseRedirect(next_page)        return super().dispatch(request, *args, **kwargs)        return super().get(request, *args, **kwargs)     def post(self, request, *args, **kwargs):        """Logout may be done via POST."""        return self.get(request, *args, **kwargs)    # RemovedInDjango50Warning.    get = post      def get_next_page(self):         if self.next_page is not None:


Question: Deprecate providingargs argument from Signal Description The argument is and always has been purely documentational. It provides no functionality or checking. Therefore, these values are stored in memory for no real use. Documentation can be handled just as easily by a code comment or real documentation articles. On a more practical level, I rarely signals, so I recently had to look up their interface. Seeing this providingargs argument required me to think about how to best use it, only to realize it actually goes unused. We can remove this cognitive distraction from the docs. It has caused other small confusion in the past: .

from django.dispatch import Signal user_logged_in = Signal(providing_args=['request', 'user'])user_login_failed = Signal(providing_args=['credentials', 'request'])user_logged_out = Signal(providing_args=['request', 'user'])user_logged_in = Signal()user_login_failed = Signal()user_logged_out = Signal()diff --git a/django/core/signals.py b/django/core/signals.py-- a/django/core/signals.py++ b/django/core/signals.py


Question: Deprecate providingargs argument from Signal Description The argument is and always has been purely documentational. It provides no functionality or checking. Therefore, these values are stored in memory for no real use. Documentation can be handled just as easily by a code comment or real documentation articles. On a more practical level, I rarely signals, so I recently had to look up their interface. Seeing this providingargs argument required me to think about how to best use it, only to realize it actually goes unused. We can remove this cognitive distraction from the docs. It has caused other small confusion in the past: .

from django.dispatch import Signal request_started = Signal(providing_args=["environ", "scope"])request_started = Signal() request_finished = Signal()got_request_exception = Signal(providing_args=["request"])setting_changed = Signal(providing_args=["setting", "value", "enter"])got_request_exception = Signal()setting_changed = Signal()diff --git a/django/db/backends/signals.py b/django/db/backends/signals.py-- a/django/db/backends/signals.py++ b/django/db/backends/signals.py


Question: Deprecate providingargs argument from Signal Description The argument is and always has been purely documentational. It provides no functionality or checking. Therefore, these values are stored in memory for no real use. Documentation can be handled just as easily by a code comment or real documentation articles. On a more practical level, I rarely signals, so I recently had to look up their interface. Seeing this providingargs argument required me to think about how to best use it, only to realize it actually goes unused. We can remove this cognitive distraction from the docs. It has caused other small confusion in the past: .

from django.dispatch import Signal connection_created = Signal(providing_args=["connection"])connection_created = Signal()diff --git a/django/db/models/signals.py b/django/db/models/signals.py-- a/django/db/models/signals.py++ b/django/db/models/signals.py


Question: Deprecate providingargs argument from Signal Description The argument is and always has been purely documentational. It provides no functionality or checking. Therefore, these values are stored in memory for no real use. Documentation can be handled just as easily by a code comment or real documentation articles. On a more practical level, I rarely signals, so I recently had to look up their interface. Seeing this providingargs argument required me to think about how to best use it, only to realize it actually goes unused. We can remove this cognitive distraction from the docs. It has caused other small confusion in the past: .

from django.db.models.utils import make_model_tuple from django.dispatch import Signal class_prepared = Signal(providing_args=["class"])class_prepared = Signal()   class ModelSignal(Signal):


Question: Deprecate providingargs argument from Signal Description The argument is and always has been purely documentational. It provides no functionality or checking. Therefore, these values are stored in memory for no real use. Documentation can be handled just as easily by a code comment or real documentation articles. On a more practical level, I rarely signals, so I recently had to look up their interface. Seeing this providingargs argument required me to think about how to best use it, only to realize it actually goes unused. We can remove this cognitive distraction from the docs. It has caused other small confusion in the past: .

)  pre_init = ModelSignal(providing_args=["instance", "args", "kwargs"], use_caching=True)post_init = ModelSignal(providing_args=["instance"], use_caching=True)pre_init = ModelSignal(use_caching=True)post_init = ModelSignal(use_caching=True) pre_save = ModelSignal(providing_args=["instance", "raw", "using", "update_fields"],                       use_caching=True)post_save = ModelSignal(providing_args=["instance", "raw", "created", "using", "update_fields"], use_caching=True)pre_save = ModelSignal(use_caching=True)post_save = ModelSignal(use_caching=True) pre_delete = ModelSignal(providing_args=["instance", "using"], use_caching=True)post_delete = ModelSignal(providing_args=["instance", "using"], use_caching=True)pre_delete = ModelSignal(use_caching=True)post_delete = ModelSignal(use_caching=True) m2m_changed = ModelSignal(    providing_args=["action", "instance", "reverse", "model", "pk_set", "using"],    use_caching=True,)m2m_changed = ModelSignal(use_caching=True) pre_migrate = Signal(providing_args=["app_config", "verbosity", "interactive", "using", "apps", "plan"])post_migrate = Signal(providing_args=["app_config", "verbosity", "interactive", "using", "apps", "plan"])pre_migrate = Signal()post_migrate = Signal()diff --git a/django/dispatch/dispatcher.py b/django/dispatch/dispatcher.py-- a/django/dispatch/dispatcher.py++ b/django/dispatch/dispatcher.py


Question: Deprecate providingargs argument from Signal Description The argument is and always has been purely documentational. It provides no functionality or checking. Therefore, these values are stored in memory for no real use. Documentation can be handled just as easily by a code comment or real documentation articles. On a more practical level, I rarely signals, so I recently had to look up their interface. Seeing this providingargs argument required me to think about how to best use it, only to realize it actually goes unused. We can remove this cognitive distraction from the docs. It has caused other small confusion in the past: .

import threadingimport warnings import weakref from django.utils.deprecation import RemovedInDjango40Warning from django.utils.inspect import func_accepts_kwargs


Question: Deprecate providingargs argument from Signal Description The argument is and always has been purely documentational. It provides no functionality or checking. Therefore, these values are stored in memory for no real use. Documentation can be handled just as easily by a code comment or real documentation articles. On a more practical level, I rarely signals, so I recently had to look up their interface. Seeing this providingargs argument required me to think about how to best use it, only to realize it actually goes unused. We can remove this cognitive distraction from the docs. It has caused other small confusion in the past: .

def __init__(self, providing_args=None, use_caching=False):         """         Create a new signal.        providing_args            A list of the arguments this signal can pass along in a send() call.         """         self.receivers = []        if providing_args is None:            providing_args = []        self.providing_args = set(providing_args)        if providing_args is not None:            warnings.warn(                'The providing_args argument is deprecated. As it is purely '                'documentational, it has no replacement. If you rely on this '                'argument as documentation, you can move the text to a code '                'comment or docstring.',                RemovedInDjango40Warning, stacklevel=2,            )         self.lock = threading.Lock()         self.use_caching = use_caching         # For convenience we create empty caches even if they are not used.


Question: Deprecate providingargs argument from Signal Description The argument is and always has been purely documentational. It provides no functionality or checking. Therefore, these values are stored in memory for no real use. Documentation can be handled just as easily by a code comment or real documentation articles. On a more practical level, I rarely signals, so I recently had to look up their interface. Seeing this providingargs argument required me to think about how to best use it, only to realize it actually goes unused. We can remove this cognitive distraction from the docs. It has caused other small confusion in the past: .

occur).              named                Named arguments which will be passed to receivers. These                arguments must be a subset of the argument names defined in                providing_args.                Named arguments which will be passed to receivers.          Return a list of tuple pairs [(receiver, response), ... ]. diff --git a/django/utils/autoreload.py b/django/utils/autoreload.py-- a/django/utils/autoreload.py++ b/django/utils/autoreload.py


Question: Deprecate providingargs argument from Signal Description The argument is and always has been purely documentational. It provides no functionality or checking. Therefore, these values are stored in memory for no real use. Documentation can be handled just as easily by a code comment or real documentation articles. On a more practical level, I rarely signals, so I recently had to look up their interface. Seeing this providingargs argument required me to think about how to best use it, only to realize it actually goes unused. We can remove this cognitive distraction from the docs. It has caused other small confusion in the past: .

from django.utils.version import get_version_tuple  autoreload_started = Signal()file_changed = Signal(providing_args=['file_path', 'kind'])file_changed = Signal()  DJANGO_AUTORELOAD_ENV = 'RUN_MAIN'


Question: Inconsistent datetime logging from runserver. Description In Django 1.11 and higher, the runserver logging can sometimes be inconsistent. [16/Apr/2018 13:32:35] "GET /some/local/url HTTP/1.1" 200 7927 [2018-04-16 13:32:35,745] - Broken pipe from ('127.0.0.1', ) This is because logging from WSGIRequestHandler uses servertime as calculated using BaseHTTPServer.logdatetimestring. On the other hand, WSGIServer uses logging without providing a servertime. This gets "fixed" in using self.formatTime(record, self.datefmt), which uses a completely different format. Currently we make this at least consistent by providing the datefmt parameter when constructing the logger, but it would be better if they were coded to be in sync (and configurable?). (Looking into it further, it looks like we should be using %(asctime)s instead of %(servertime)s, but would be good if that were the suggested default. In  we see %(server_time)s.)

class ServerFormatter(logging.Formatter):<ins>    default_time_format = '%d/%b/%Y %H:%M:%S'</ins>     def __init__(self, *args, **kwargs):         self.style = color_style()         super().__init__(*args, **kwargs)


Question: sqlmigrate doesn't allow inspecting migrations that have been squashed Description This project for another ticket can be used to reproduce:  When running sqlmigrate to pick up migration 0001 in this project, it complains that two migrations have that prefix: $ python sqlmigrate testapp 0001 CommandError: More than one migration matches '0001' in app 'testapp'. Please be more specific. But when trying to be more specific, it's not possible to load it: $ python sqlmigrate testapp 0001initial Traceback (most recent call last): File "", line 21, in <modulemain() File "", line 17, in main executefromcommandline() File "", line 401, in executefromcommandline utility.execute() File "", line 395, in execute self.fetchcommand(subcommand).runfromargv() File "", line 328, in runfromargv self.execute(args, cmdoptions) File "", line 30, in execute return super().execute(args, options) File "", line 369, in execute output = (args, *options) File "", line 64, in handle plan = [([targets[0]], options['backwards'])] KeyError: ('testapp', '0001initial') It would be nice to: A) catch this error and report a nice message as well B) allow inspection of individual migrations that have been involved in a squash. Normally the workflow is to remove the individual migrations some time after committing the squash, but until that is done it could be useful to see their sql.

from django.apps import apps from django.core.management.base import BaseCommand, CommandError from django.db import DEFAULT_DB_ALIAS, connectionsfrom django.db.migrations.executor import MigrationExecutorfrom django.db.migrations.loader import AmbiguityErrorfrom django.db.migrations.loader import AmbiguityError, MigrationLoader   class Command(BaseCommand):


Question: sqlmigrate doesn't allow inspecting migrations that have been squashed Description This project for another ticket can be used to reproduce:  When running sqlmigrate to pick up migration 0001 in this project, it complains that two migrations have that prefix: $ python sqlmigrate testapp 0001 CommandError: More than one migration matches '0001' in app 'testapp'. Please be more specific. But when trying to be more specific, it's not possible to load it: $ python sqlmigrate testapp 0001initial Traceback (most recent call last): File "", line 21, in <modulemain() File "", line 17, in main executefromcommandline() File "", line 401, in executefromcommandline utility.execute() File "", line 395, in execute self.fetchcommand(subcommand).runfromargv() File "", line 328, in runfromargv self.execute(args, cmdoptions) File "", line 30, in execute return super().execute(args, options) File "", line 369, in execute output = (args, *options) File "", line 64, in handle plan = [([targets[0]], options['backwards'])] KeyError: ('testapp', '0001initial') It would be nice to: A) catch this error and report a nice message as well B) allow inspection of individual migrations that have been involved in a squash. Normally the workflow is to remove the individual migrations some time after committing the squash, but until that is done it could be useful to see their sql.

# Get the database we're operating from         connection = connections[options['database']]         # Load up an executor to get all the migration data        executor = MigrationExecutor(connection)        # Load up an loader to get all the migration data, but don't replace        # migrations.        loader = MigrationLoader(connection, replace_migrations=False)          # Resolve command-line arguments into a migration         app_label, migration_name = options['app_label'], options['migration_name']


Question: sqlmigrate doesn't allow inspecting migrations that have been squashed Description This project for another ticket can be used to reproduce:  When running sqlmigrate to pick up migration 0001 in this project, it complains that two migrations have that prefix: $ python sqlmigrate testapp 0001 CommandError: More than one migration matches '0001' in app 'testapp'. Please be more specific. But when trying to be more specific, it's not possible to load it: $ python sqlmigrate testapp 0001initial Traceback (most recent call last): File "", line 21, in <modulemain() File "", line 17, in main executefromcommandline() File "", line 401, in executefromcommandline utility.execute() File "", line 395, in execute self.fetchcommand(subcommand).runfromargv() File "", line 328, in runfromargv self.execute(args, cmdoptions) File "", line 30, in execute return super().execute(args, options) File "", line 369, in execute output = (args, *options) File "", line 64, in handle plan = [([targets[0]], options['backwards'])] KeyError: ('testapp', '0001initial') It would be nice to: A) catch this error and report a nice message as well B) allow inspection of individual migrations that have been involved in a squash. Normally the workflow is to remove the individual migrations some time after committing the squash, but until that is done it could be useful to see their sql.

apps.get_app_config(app_label)         except LookupError as err:             raise CommandError(str(err))        if app_label not in executor.loader.migrated_apps:        if app_label not in loader.migrated_apps:             raise CommandError("App '%s' does not have migrations" % app_label)         try:            migration = executor.loader.get_migration_by_prefix(app_label, migration_name)            migration = loader.get_migration_by_prefix(app_label, migration_name)         except AmbiguityError:             raise CommandError("More than one migration matches '%s' in app '%s'. Please be more specific." % (                 migration_name, app_label))


Question: sqlmigrate doesn't allow inspecting migrations that have been squashed Description This project for another ticket can be used to reproduce:  When running sqlmigrate to pick up migration 0001 in this project, it complains that two migrations have that prefix: $ python sqlmigrate testapp 0001 CommandError: More than one migration matches '0001' in app 'testapp'. Please be more specific. But when trying to be more specific, it's not possible to load it: $ python sqlmigrate testapp 0001initial Traceback (most recent call last): File "", line 21, in <modulemain() File "", line 17, in main executefromcommandline() File "", line 401, in executefromcommandline utility.execute() File "", line 395, in execute self.fetchcommand(subcommand).runfromargv() File "", line 328, in runfromargv self.execute(args, cmdoptions) File "", line 30, in execute return super().execute(args, options) File "", line 369, in execute output = (args, *options) File "", line 64, in handle plan = [([targets[0]], options['backwards'])] KeyError: ('testapp', '0001initial') It would be nice to: A) catch this error and report a nice message as well B) allow inspection of individual migrations that have been involved in a squash. Normally the workflow is to remove the individual migrations some time after committing the squash, but until that is done it could be useful to see their sql.

# Make a plan that represents just the requested migrations and show SQL         # for it        plan = [(executor.loader.graph.nodes[target], options['backwards'])]        sql_statements = executor.collect_sql(plan)        plan = [(loader.graph.nodes[target], options['backwards'])]        sql_statements = loader.collect_sql(plan)         if not sql_statements and options['verbosity'] >= 1:             self.stderr.write('No operations found.')         return '\n'.join(sql_statements)diff --git a/django/db/migrations/executor.py b/django/db/migrations/executor.py-- a/django/db/migrations/executor.py++ b/django/db/migrations/executor.py


Question: sqlmigrate doesn't allow inspecting migrations that have been squashed Description This project for another ticket can be used to reproduce:  When running sqlmigrate to pick up migration 0001 in this project, it complains that two migrations have that prefix: $ python sqlmigrate testapp 0001 CommandError: More than one migration matches '0001' in app 'testapp'. Please be more specific. But when trying to be more specific, it's not possible to load it: $ python sqlmigrate testapp 0001initial Traceback (most recent call last): File "", line 21, in <modulemain() File "", line 17, in main executefromcommandline() File "", line 401, in executefromcommandline utility.execute() File "", line 395, in execute self.fetchcommand(subcommand).runfromargv() File "", line 328, in runfromargv self.execute(args, cmdoptions) File "", line 30, in execute return super().execute(args, options) File "", line 369, in execute output = (args, *options) File "", line 64, in handle plan = [([targets[0]], options['backwards'])] KeyError: ('testapp', '0001initial') It would be nice to: A) catch this error and report a nice message as well B) allow inspection of individual migrations that have been involved in a squash. Normally the workflow is to remove the individual migrations some time after committing the squash, but until that is done it could be useful to see their sql.

return state     def collect_sql(self, plan):        """        Take a migration plan and return a list of collected SQL statements        that represent the best-efforts version of that plan.        """        statements = []        state = None        for migration, backwards in plan:            with self.connection.schema_editor(collect_sql=True, atomic=migration.atomic) as schema_editor:                if state is None:                    state = self.loader.project_state((migration.app_label, migration.name), at_end=False)                if not backwards:                    state = migration.apply(state, schema_editor, collect_sql=True)                else:                    state = migration.unapply(state, schema_editor, collect_sql=True)            statements.extend(schema_editor.collected_sql)        return statements     def apply_migration(self, state, migration, fake=False, fake_initial=False):         """Run a migration forwards."""         migration_recorded = Falsediff --git a/django/db/migrations/loader.py b/django/db/migrations/loader.py-- a/django/db/migrations/loader.py++ b/django/db/migrations/loader.py


Question: sqlmigrate doesn't allow inspecting migrations that have been squashed Description This project for another ticket can be used to reproduce:  When running sqlmigrate to pick up migration 0001 in this project, it complains that two migrations have that prefix: $ python sqlmigrate testapp 0001 CommandError: More than one migration matches '0001' in app 'testapp'. Please be more specific. But when trying to be more specific, it's not possible to load it: $ python sqlmigrate testapp 0001initial Traceback (most recent call last): File "", line 21, in <modulemain() File "", line 17, in main executefromcommandline() File "", line 401, in executefromcommandline utility.execute() File "", line 395, in execute self.fetchcommand(subcommand).runfromargv() File "", line 328, in runfromargv self.execute(args, cmdoptions) File "", line 30, in execute return super().execute(args, options) File "", line 369, in execute output = (args, *options) File "", line 64, in handle plan = [([targets[0]], options['backwards'])] KeyError: ('testapp', '0001initial') It would be nice to: A) catch this error and report a nice message as well B) allow inspection of individual migrations that have been involved in a squash. Normally the workflow is to remove the individual migrations some time after committing the squash, but until that is done it could be useful to see their sql.

in memory.     """     def __init__(self, connection, load=True, ignore_no_migrations=False):    def __init__(        self, connection, load=True, ignore_no_migrations=False,        replace_migrations=True,    ):         self.connection = connection         self.disk_migrations = None         self.applied_migrations = None         self.ignore_no_migrations = ignore_no_migrations        self.replace_migrations = replace_migrations         if load:             self.build_graph()


Question: sqlmigrate doesn't allow inspecting migrations that have been squashed Description This project for another ticket can be used to reproduce:  When running sqlmigrate to pick up migration 0001 in this project, it complains that two migrations have that prefix: $ python sqlmigrate testapp 0001 CommandError: More than one migration matches '0001' in app 'testapp'. Please be more specific. But when trying to be more specific, it's not possible to load it: $ python sqlmigrate testapp 0001initial Traceback (most recent call last): File "", line 21, in <modulemain() File "", line 17, in main executefromcommandline() File "", line 401, in executefromcommandline utility.execute() File "", line 395, in execute self.fetchcommand(subcommand).runfromargv() File "", line 328, in runfromargv self.execute(args, cmdoptions) File "", line 30, in execute return super().execute(args, options) File "", line 369, in execute output = (args, *options) File "", line 64, in handle plan = [([targets[0]], options['backwards'])] KeyError: ('testapp', '0001initial') It would be nice to: A) catch this error and report a nice message as well B) allow inspection of individual migrations that have been involved in a squash. Normally the workflow is to remove the individual migrations some time after committing the squash, but until that is done it could be useful to see their sql.

# Add external dependencies now that the internal ones have been resolved.         for key, migration in self.disk_migrations.items():             self.add_external_dependencies(key, migration)        # Carry out replacements where possible.        for key, migration in self.replacements.items():            # Get applied status of each of this migration's replacement targets.            applied_statuses = [(target in self.applied_migrations) for target in migration.replaces]            # Ensure the replacing migration is only marked as applied if all of            # its replacement targets are.            if all(applied_statuses):                self.applied_migrations[key] = migration            else:                self.applied_migrations.pop(key, None)            # A replacing migration can be used if either all or none of its            # replacement targets have been applied.            if all(applied_statuses) or (not any(applied_statuses)):                self.graph.remove_replaced_nodes(key, migration.replaces)            else:                # This replacing migration cannot be used because it is partially applied.                # Remove it from the graph and remap dependencies to it (#25945).                self.graph.remove_replacement_node(key, migration.replaces)        # Carry out replacements where possible and if enabled.        if self.replace_migrations:            for key, migration in self.replacements.items():                # Get applied status of each of this migration's replacement                # targets.                applied_statuses = [(target in self.applied_migrations) for target in migration.replaces]                # The replacing migration is only marked as applied if all of                # its replacement targets are.                if all(applied_statuses):                    self.applied_migrations[key] = migration                else:                    self.applied_migrations.pop(key, None)                # A replacing migration can be used if either all or none of                # its replacement targets have been applied.                if all(applied_statuses) or (not any(applied_statuses)):                    self.graph.remove_replaced_nodes(key, migration.replaces)                else:                    # This replacing migration cannot be used because it is                    # partially applied. Remove it from the graph and remap                    # dependencies to it (#25945).                    self.graph.remove_replacement_node(key, migration.replaces)         # Ensure the graph is consistent.         try:             self.graph.validate_consistency()


Question: sqlmigrate doesn't allow inspecting migrations that have been squashed Description This project for another ticket can be used to reproduce:  When running sqlmigrate to pick up migration 0001 in this project, it complains that two migrations have that prefix: $ python sqlmigrate testapp 0001 CommandError: More than one migration matches '0001' in app 'testapp'. Please be more specific. But when trying to be more specific, it's not possible to load it: $ python sqlmigrate testapp 0001initial Traceback (most recent call last): File "", line 21, in <modulemain() File "", line 17, in main executefromcommandline() File "", line 401, in executefromcommandline utility.execute() File "", line 395, in execute self.fetchcommand(subcommand).runfromargv() File "", line 328, in runfromargv self.execute(args, cmdoptions) File "", line 30, in execute return super().execute(args, options) File "", line 369, in execute output = (args, *options) File "", line 64, in handle plan = [([targets[0]], options['backwards'])] KeyError: ('testapp', '0001initial') It would be nice to: A) catch this error and report a nice message as well B) allow inspection of individual migrations that have been involved in a squash. Normally the workflow is to remove the individual migrations some time after committing the squash, but until that is done it could be useful to see their sql.

See graph.make_state() for the meaning of "nodes" and "at_end".         """         return self.graph.make_state(nodes=nodes, at_end=at_end, real_apps=list(self.unmigrated_apps))<ins>    def collect_sql(self, plan):        """        Take a migration plan and return a list of collected SQL statements        that represent the best-efforts version of that plan.        """        statements = []        state = None        for migration, backwards in plan:            with self.connection.schema_editor(collect_sql=True, atomic=migration.atomic) as schema_editor:                if state is None:                    state = self.project_state((migration.app_label, migration.name), at_end=False)                if not backwards:                    state = migration.apply(state, schema_editor, collect_sql=True)                else:                    state = migration.unapply(state, schema_editor, collect_sql=True)            statements.extend(schema_editor.collected_sql)        return statements</ins>


Question: Subquery annotations are omitted in group by query section if multiple annotation are declared Description (last modified by Johannes Maron) Sadly there is more regression in Django 3.0.2 even after . Background: It's the same query as . I tried upgrading to Django 3.0.2 and now I get duplicate results. Even tho they query should be distinct. Where on 2.2 the queryset yields 490 results, it's 519 on 3.0. A quick diff on the queries still reveals a different grouped by section: This is the new query on 3.0.2: SELECT DISTINCT "campsoffer"."id", "campsoffer"."title", "campsoffer"."slug", "campsoffer"."isactive", "campsoffer"."modified", "campsoffer"."created", "campsoffer"."providerid", "campsoffer"."activitytype", "campsoffer"."description", "campsoffer"."highlights", "campsoffer"."importantinformation", "campsoffer"."minage", "campsoffer"."maxage", "campsoffer"."food", "campsoffer"."video", "campsoffer"."accommodation", "campsoffer"."accommodationtype", "campsoffer"."roomtype", "campsoffer"."roomsizemin", "campsoffer"."roomsizemax", "campsoffer"."externalurl", "campsoffer"."applicationform", "campsoffer"."caseload", "campsoffer"."fieldtrips", MIN(T4."retailprice") AS "minretailprice", (SELECT U0."id" FROM "campsservicepackage" U0 INNER JOIN "campsregion" U2 ON (U0."regionid" = U2."id") WHERE (U0."companyid" = 1 AND U0."option" = "campsoffer"."activitytype" AND STContains(U2."locations", T4."position")) LIMIT 1) AS "inpackage", "campsprovider"."id", "campsprovider"."title", "campsprovider"."slug", "campsprovider"."isactive", "campsprovider"."modified", "campsprovider"."created", "campsprovider"."logo", "campsprovider"."description", "campsprovider"."video", "campsprovider"."externalurl", "campsprovider"."terms", "campsprovider"."cancellationpolicy", "campsprovider"."privacypolicy", "campsprovider"."applicationform" FROM "campsoffer" LEFT OUTER JOIN "campsbookingoption" ON ("campsoffer"."id" = "campsbookingoption"."offerid") INNER JOIN "campsprovider" ON ("campsoffer"."providerid" = "campsprovider"."id") INNER JOIN "campsbookingoption" T4 ON ("campsoffer"."id" = T4."offerid") WHERE ("campsoffer"."isactive" = True AND "campsprovider"."isactive" = True AND T4."end" = STATEMENTTIMESTAMP() AND T4."isactive" = True AND "campsoffer"."maxage" = 5 AND "campsoffer"."minage" <= 13 AND (SELECT U0."id" FROM "campsservicepackage" U0 INNER JOIN "campsregion" U2 ON (U0."regionid" = U2."id") WHERE (U0."companyid" = 1 AND U0."option" = "campsoffer"."activitytype" AND STContains(U2."locations", T4."position")) LIMIT 1) IS NOT NULL) GROUP BY "campsoffer"."id", T4."position", "campsprovider"."id" ORDER BY "campsoffer"."created" ASC And what it was (and should be) on 2.2.9: SELECT DISTINCT "campsoffer"."id", "campsoffer"."title", "campsoffer"."slug", "campsoffer"."isactive", "campsoffer"."modified", "campsoffer"."created", "campsoffer"."providerid", "campsoffer"."activitytype", "campsoffer"."description", "campsoffer"."highlights", "campsoffer"."importantinformation", "campsoffer"."minage", "campsoffer"."maxage", "campsoffer"."food", "campsoffer"."video", "campsoffer"."accommodation", "campsoffer"."accommodationtype", "campsoffer"."roomtype", "campsoffer"."roomsizemin", "campsoffer"."roomsizemax", "campsoffer"."externalurl", "campsoffer"."applicationform", "campsoffer"."caseload", "campsoffer"."fieldtrips", MIN(T4."retailprice") AS "minretailprice", (SELECT U0."id" FROM "campsservicepackage" U0 INNER JOIN "campsregion" U2 ON (U0."regionid" = U2."id") WHERE (U0."companyid" = 1 AND U0."option" = ("campsoffer"."activitytype") AND STContains(U2."locations", (T4."position"))) LIMIT 1) AS "inpackage", "campsprovider"."id", "campsprovider"."title", "campsprovider"."slug", "campsprovider"."isactive", "campsprovider"."modified", "campsprovider"."created", "campsprovider"."logo", "campsprovider"."description", "campsprovider"."video", "campsprovider"."externalurl", "campsprovider"."terms", "campsprovider"."cancellationpolicy", "campsprovider"."privacypolicy", "campsprovider"."applicationform" FROM "campsoffer" LEFT OUTER JOIN "campsbookingoption" ON ("campsoffer"."id" = "campsbookingoption"."offerid") INNER JOIN "campsprovider" ON ("campsoffer"."providerid" = "campsprovider"."id") INNER JOIN "campsbookingoption" T4 ON ("campsoffer"."id" = T4."offerid") WHERE ("campsoffer"."isactive" = True AND "campsprovider"."isactive" = True AND T4."end" = (STATEMENTTIMESTAMP()) AND T4."isactive" = True AND (SELECT U0."id" FROM "campsservicepackage" U0 INNER JOIN "campsregion" U2 ON (U0."regionid" = U2."id") WHERE (U0."companyid" = 1 AND U0."option" = ("campsoffer"."activitytype") AND STContains(U2."locations", (T4."position"))) LIMIT 1) IS NOT NULL) GROUP BY "campsoffer"."id", (SELECT U0."id" FROM "campsservicepackage" U0 INNER JOIN "campsregion" U2 ON (U0."regionid" = U2."id") WHERE (U0."companyid" = 1 AND U0."option" = ("campsoffer"."activitytype") AND STContains(U2."locations", (T4."position"))) LIMIT 1), "campsprovider"."id" ORDER BY "camps_offer"."created" ASC

@cached_property     def mysql_is_mariadb(self):         return 'mariadb' in self.mysql_server_info.lower()    @cached_property    def sql_mode(self):        with self.cursor() as cursor:            cursor.execute('SELECT @@sql_mode')            sql_mode = cursor.fetchone()        return set(sql_mode[0].split(',') if sql_mode else ())diff --git a/django/db/backends/mysql/validation.py b/django/db/backends/mysql/validation.py-- a/django/db/backends/mysql/validation.py++ b/django/db/backends/mysql/validation.py


Question: Subquery annotations are omitted in group by query section if multiple annotation are declared Description (last modified by Johannes Maron) Sadly there is more regression in Django 3.0.2 even after . Background: It's the same query as . I tried upgrading to Django 3.0.2 and now I get duplicate results. Even tho they query should be distinct. Where on 2.2 the queryset yields 490 results, it's 519 on 3.0. A quick diff on the queries still reveals a different grouped by section: This is the new query on 3.0.2: SELECT DISTINCT "campsoffer"."id", "campsoffer"."title", "campsoffer"."slug", "campsoffer"."isactive", "campsoffer"."modified", "campsoffer"."created", "campsoffer"."providerid", "campsoffer"."activitytype", "campsoffer"."description", "campsoffer"."highlights", "campsoffer"."importantinformation", "campsoffer"."minage", "campsoffer"."maxage", "campsoffer"."food", "campsoffer"."video", "campsoffer"."accommodation", "campsoffer"."accommodationtype", "campsoffer"."roomtype", "campsoffer"."roomsizemin", "campsoffer"."roomsizemax", "campsoffer"."externalurl", "campsoffer"."applicationform", "campsoffer"."caseload", "campsoffer"."fieldtrips", MIN(T4."retailprice") AS "minretailprice", (SELECT U0."id" FROM "campsservicepackage" U0 INNER JOIN "campsregion" U2 ON (U0."regionid" = U2."id") WHERE (U0."companyid" = 1 AND U0."option" = "campsoffer"."activitytype" AND STContains(U2."locations", T4."position")) LIMIT 1) AS "inpackage", "campsprovider"."id", "campsprovider"."title", "campsprovider"."slug", "campsprovider"."isactive", "campsprovider"."modified", "campsprovider"."created", "campsprovider"."logo", "campsprovider"."description", "campsprovider"."video", "campsprovider"."externalurl", "campsprovider"."terms", "campsprovider"."cancellationpolicy", "campsprovider"."privacypolicy", "campsprovider"."applicationform" FROM "campsoffer" LEFT OUTER JOIN "campsbookingoption" ON ("campsoffer"."id" = "campsbookingoption"."offerid") INNER JOIN "campsprovider" ON ("campsoffer"."providerid" = "campsprovider"."id") INNER JOIN "campsbookingoption" T4 ON ("campsoffer"."id" = T4."offerid") WHERE ("campsoffer"."isactive" = True AND "campsprovider"."isactive" = True AND T4."end" = STATEMENTTIMESTAMP() AND T4."isactive" = True AND "campsoffer"."maxage" = 5 AND "campsoffer"."minage" <= 13 AND (SELECT U0."id" FROM "campsservicepackage" U0 INNER JOIN "campsregion" U2 ON (U0."regionid" = U2."id") WHERE (U0."companyid" = 1 AND U0."option" = "campsoffer"."activitytype" AND STContains(U2."locations", T4."position")) LIMIT 1) IS NOT NULL) GROUP BY "campsoffer"."id", T4."position", "campsprovider"."id" ORDER BY "campsoffer"."created" ASC And what it was (and should be) on 2.2.9: SELECT DISTINCT "campsoffer"."id", "campsoffer"."title", "campsoffer"."slug", "campsoffer"."isactive", "campsoffer"."modified", "campsoffer"."created", "campsoffer"."providerid", "campsoffer"."activitytype", "campsoffer"."description", "campsoffer"."highlights", "campsoffer"."importantinformation", "campsoffer"."minage", "campsoffer"."maxage", "campsoffer"."food", "campsoffer"."video", "campsoffer"."accommodation", "campsoffer"."accommodationtype", "campsoffer"."roomtype", "campsoffer"."roomsizemin", "campsoffer"."roomsizemax", "campsoffer"."externalurl", "campsoffer"."applicationform", "campsoffer"."caseload", "campsoffer"."fieldtrips", MIN(T4."retailprice") AS "minretailprice", (SELECT U0."id" FROM "campsservicepackage" U0 INNER JOIN "campsregion" U2 ON (U0."regionid" = U2."id") WHERE (U0."companyid" = 1 AND U0."option" = ("campsoffer"."activitytype") AND STContains(U2."locations", (T4."position"))) LIMIT 1) AS "inpackage", "campsprovider"."id", "campsprovider"."title", "campsprovider"."slug", "campsprovider"."isactive", "campsprovider"."modified", "campsprovider"."created", "campsprovider"."logo", "campsprovider"."description", "campsprovider"."video", "campsprovider"."externalurl", "campsprovider"."terms", "campsprovider"."cancellationpolicy", "campsprovider"."privacypolicy", "campsprovider"."applicationform" FROM "campsoffer" LEFT OUTER JOIN "campsbookingoption" ON ("campsoffer"."id" = "campsbookingoption"."offerid") INNER JOIN "campsprovider" ON ("campsoffer"."providerid" = "campsprovider"."id") INNER JOIN "campsbookingoption" T4 ON ("campsoffer"."id" = T4."offerid") WHERE ("campsoffer"."isactive" = True AND "campsprovider"."isactive" = True AND T4."end" = (STATEMENTTIMESTAMP()) AND T4."isactive" = True AND (SELECT U0."id" FROM "campsservicepackage" U0 INNER JOIN "campsregion" U2 ON (U0."regionid" = U2."id") WHERE (U0."companyid" = 1 AND U0."option" = ("campsoffer"."activitytype") AND STContains(U2."locations", (T4."position"))) LIMIT 1) IS NOT NULL) GROUP BY "campsoffer"."id", (SELECT U0."id" FROM "campsservicepackage" U0 INNER JOIN "campsregion" U2 ON (U0."regionid" = U2."id") WHERE (U0."companyid" = 1 AND U0."option" = ("campsoffer"."activitytype") AND STContains(U2."locations", (T4."position"))) LIMIT 1), "campsprovider"."id" ORDER BY "camps_offer"."created" ASC

return issues      def _check_sql_mode(self, **kwargs):        with self.connection.cursor() as cursor:            cursor.execute("SELECT @@sql_mode")            sql_mode = cursor.fetchone()        modes = set(sql_mode[0].split(',') if sql_mode else ())        if not (modes & {'STRICT_TRANS_TABLES', 'STRICT_ALL_TABLES'}):        if not (self.connection.sql_mode & {'STRICT_TRANS_TABLES', 'STRICT_ALL_TABLES'}):             return [checks.Warning(                 "MySQL Strict Mode is not set for database connection '%s'" % self.connection.alias,                 hint="MySQL's Strict Mode fixes many data integrity problems in MySQL, "diff --git a/django/db/models/expressions.py b/django/db/models/expressions.py-- a/django/db/models/expressions.py++ b/django/db/models/expressions.py


Question: Subquery annotations are omitted in group by query section if multiple annotation are declared Description (last modified by Johannes Maron) Sadly there is more regression in Django 3.0.2 even after . Background: It's the same query as . I tried upgrading to Django 3.0.2 and now I get duplicate results. Even tho they query should be distinct. Where on 2.2 the queryset yields 490 results, it's 519 on 3.0. A quick diff on the queries still reveals a different grouped by section: This is the new query on 3.0.2: SELECT DISTINCT "campsoffer"."id", "campsoffer"."title", "campsoffer"."slug", "campsoffer"."isactive", "campsoffer"."modified", "campsoffer"."created", "campsoffer"."providerid", "campsoffer"."activitytype", "campsoffer"."description", "campsoffer"."highlights", "campsoffer"."importantinformation", "campsoffer"."minage", "campsoffer"."maxage", "campsoffer"."food", "campsoffer"."video", "campsoffer"."accommodation", "campsoffer"."accommodationtype", "campsoffer"."roomtype", "campsoffer"."roomsizemin", "campsoffer"."roomsizemax", "campsoffer"."externalurl", "campsoffer"."applicationform", "campsoffer"."caseload", "campsoffer"."fieldtrips", MIN(T4."retailprice") AS "minretailprice", (SELECT U0."id" FROM "campsservicepackage" U0 INNER JOIN "campsregion" U2 ON (U0."regionid" = U2."id") WHERE (U0."companyid" = 1 AND U0."option" = "campsoffer"."activitytype" AND STContains(U2."locations", T4."position")) LIMIT 1) AS "inpackage", "campsprovider"."id", "campsprovider"."title", "campsprovider"."slug", "campsprovider"."isactive", "campsprovider"."modified", "campsprovider"."created", "campsprovider"."logo", "campsprovider"."description", "campsprovider"."video", "campsprovider"."externalurl", "campsprovider"."terms", "campsprovider"."cancellationpolicy", "campsprovider"."privacypolicy", "campsprovider"."applicationform" FROM "campsoffer" LEFT OUTER JOIN "campsbookingoption" ON ("campsoffer"."id" = "campsbookingoption"."offerid") INNER JOIN "campsprovider" ON ("campsoffer"."providerid" = "campsprovider"."id") INNER JOIN "campsbookingoption" T4 ON ("campsoffer"."id" = T4."offerid") WHERE ("campsoffer"."isactive" = True AND "campsprovider"."isactive" = True AND T4."end" = STATEMENTTIMESTAMP() AND T4."isactive" = True AND "campsoffer"."maxage" = 5 AND "campsoffer"."minage" <= 13 AND (SELECT U0."id" FROM "campsservicepackage" U0 INNER JOIN "campsregion" U2 ON (U0."regionid" = U2."id") WHERE (U0."companyid" = 1 AND U0."option" = "campsoffer"."activitytype" AND STContains(U2."locations", T4."position")) LIMIT 1) IS NOT NULL) GROUP BY "campsoffer"."id", T4."position", "campsprovider"."id" ORDER BY "campsoffer"."created" ASC And what it was (and should be) on 2.2.9: SELECT DISTINCT "campsoffer"."id", "campsoffer"."title", "campsoffer"."slug", "campsoffer"."isactive", "campsoffer"."modified", "campsoffer"."created", "campsoffer"."providerid", "campsoffer"."activitytype", "campsoffer"."description", "campsoffer"."highlights", "campsoffer"."importantinformation", "campsoffer"."minage", "campsoffer"."maxage", "campsoffer"."food", "campsoffer"."video", "campsoffer"."accommodation", "campsoffer"."accommodationtype", "campsoffer"."roomtype", "campsoffer"."roomsizemin", "campsoffer"."roomsizemax", "campsoffer"."externalurl", "campsoffer"."applicationform", "campsoffer"."caseload", "campsoffer"."fieldtrips", MIN(T4."retailprice") AS "minretailprice", (SELECT U0."id" FROM "campsservicepackage" U0 INNER JOIN "campsregion" U2 ON (U0."regionid" = U2."id") WHERE (U0."companyid" = 1 AND U0."option" = ("campsoffer"."activitytype") AND STContains(U2."locations", (T4."position"))) LIMIT 1) AS "inpackage", "campsprovider"."id", "campsprovider"."title", "campsprovider"."slug", "campsprovider"."isactive", "campsprovider"."modified", "campsprovider"."created", "campsprovider"."logo", "campsprovider"."description", "campsprovider"."video", "campsprovider"."externalurl", "campsprovider"."terms", "campsprovider"."cancellationpolicy", "campsprovider"."privacypolicy", "campsprovider"."applicationform" FROM "campsoffer" LEFT OUTER JOIN "campsbookingoption" ON ("campsoffer"."id" = "campsbookingoption"."offerid") INNER JOIN "campsprovider" ON ("campsoffer"."providerid" = "campsprovider"."id") INNER JOIN "campsbookingoption" T4 ON ("campsoffer"."id" = T4."offerid") WHERE ("campsoffer"."isactive" = True AND "campsprovider"."isactive" = True AND T4."end" = (STATEMENTTIMESTAMP()) AND T4."isactive" = True AND (SELECT U0."id" FROM "campsservicepackage" U0 INNER JOIN "campsregion" U2 ON (U0."regionid" = U2."id") WHERE (U0."companyid" = 1 AND U0."option" = ("campsoffer"."activitytype") AND STContains(U2."locations", (T4."position"))) LIMIT 1) IS NOT NULL) GROUP BY "campsoffer"."id", (SELECT U0."id" FROM "campsservicepackage" U0 INNER JOIN "campsregion" U2 ON (U0."regionid" = U2."id") WHERE (U0."companyid" = 1 AND U0."option" = ("campsoffer"."activitytype") AND STContains(U2."locations", (T4."position"))) LIMIT 1), "campsprovider"."id" ORDER BY "camps_offer"."created" ASC

from django.core.exceptions import EmptyResultSet, FieldError from django.db import NotSupportedError, connection from django.db.models import fields<ins>from django.db.models.constants import LOOKUP_SEP</ins> from django.db.models.query_utils import Q from django.utils.deconstruct import deconstructible from django.utils.functional import cached_property


Question: Subquery annotations are omitted in group by query section if multiple annotation are declared Description (last modified by Johannes Maron) Sadly there is more regression in Django 3.0.2 even after . Background: It's the same query as . I tried upgrading to Django 3.0.2 and now I get duplicate results. Even tho they query should be distinct. Where on 2.2 the queryset yields 490 results, it's 519 on 3.0. A quick diff on the queries still reveals a different grouped by section: This is the new query on 3.0.2: SELECT DISTINCT "campsoffer"."id", "campsoffer"."title", "campsoffer"."slug", "campsoffer"."isactive", "campsoffer"."modified", "campsoffer"."created", "campsoffer"."providerid", "campsoffer"."activitytype", "campsoffer"."description", "campsoffer"."highlights", "campsoffer"."importantinformation", "campsoffer"."minage", "campsoffer"."maxage", "campsoffer"."food", "campsoffer"."video", "campsoffer"."accommodation", "campsoffer"."accommodationtype", "campsoffer"."roomtype", "campsoffer"."roomsizemin", "campsoffer"."roomsizemax", "campsoffer"."externalurl", "campsoffer"."applicationform", "campsoffer"."caseload", "campsoffer"."fieldtrips", MIN(T4."retailprice") AS "minretailprice", (SELECT U0."id" FROM "campsservicepackage" U0 INNER JOIN "campsregion" U2 ON (U0."regionid" = U2."id") WHERE (U0."companyid" = 1 AND U0."option" = "campsoffer"."activitytype" AND STContains(U2."locations", T4."position")) LIMIT 1) AS "inpackage", "campsprovider"."id", "campsprovider"."title", "campsprovider"."slug", "campsprovider"."isactive", "campsprovider"."modified", "campsprovider"."created", "campsprovider"."logo", "campsprovider"."description", "campsprovider"."video", "campsprovider"."externalurl", "campsprovider"."terms", "campsprovider"."cancellationpolicy", "campsprovider"."privacypolicy", "campsprovider"."applicationform" FROM "campsoffer" LEFT OUTER JOIN "campsbookingoption" ON ("campsoffer"."id" = "campsbookingoption"."offerid") INNER JOIN "campsprovider" ON ("campsoffer"."providerid" = "campsprovider"."id") INNER JOIN "campsbookingoption" T4 ON ("campsoffer"."id" = T4."offerid") WHERE ("campsoffer"."isactive" = True AND "campsprovider"."isactive" = True AND T4."end" = STATEMENTTIMESTAMP() AND T4."isactive" = True AND "campsoffer"."maxage" = 5 AND "campsoffer"."minage" <= 13 AND (SELECT U0."id" FROM "campsservicepackage" U0 INNER JOIN "campsregion" U2 ON (U0."regionid" = U2."id") WHERE (U0."companyid" = 1 AND U0."option" = "campsoffer"."activitytype" AND STContains(U2."locations", T4."position")) LIMIT 1) IS NOT NULL) GROUP BY "campsoffer"."id", T4."position", "campsprovider"."id" ORDER BY "campsoffer"."created" ASC And what it was (and should be) on 2.2.9: SELECT DISTINCT "campsoffer"."id", "campsoffer"."title", "campsoffer"."slug", "campsoffer"."isactive", "campsoffer"."modified", "campsoffer"."created", "campsoffer"."providerid", "campsoffer"."activitytype", "campsoffer"."description", "campsoffer"."highlights", "campsoffer"."importantinformation", "campsoffer"."minage", "campsoffer"."maxage", "campsoffer"."food", "campsoffer"."video", "campsoffer"."accommodation", "campsoffer"."accommodationtype", "campsoffer"."roomtype", "campsoffer"."roomsizemin", "campsoffer"."roomsizemax", "campsoffer"."externalurl", "campsoffer"."applicationform", "campsoffer"."caseload", "campsoffer"."fieldtrips", MIN(T4."retailprice") AS "minretailprice", (SELECT U0."id" FROM "campsservicepackage" U0 INNER JOIN "campsregion" U2 ON (U0."regionid" = U2."id") WHERE (U0."companyid" = 1 AND U0."option" = ("campsoffer"."activitytype") AND STContains(U2."locations", (T4."position"))) LIMIT 1) AS "inpackage", "campsprovider"."id", "campsprovider"."title", "campsprovider"."slug", "campsprovider"."isactive", "campsprovider"."modified", "campsprovider"."created", "campsprovider"."logo", "campsprovider"."description", "campsprovider"."video", "campsprovider"."externalurl", "campsprovider"."terms", "campsprovider"."cancellationpolicy", "campsprovider"."privacypolicy", "campsprovider"."applicationform" FROM "campsoffer" LEFT OUTER JOIN "campsbookingoption" ON ("campsoffer"."id" = "campsbookingoption"."offerid") INNER JOIN "campsprovider" ON ("campsoffer"."providerid" = "campsprovider"."id") INNER JOIN "campsbookingoption" T4 ON ("campsoffer"."id" = T4."offerid") WHERE ("campsoffer"."isactive" = True AND "campsprovider"."isactive" = True AND T4."end" = (STATEMENTTIMESTAMP()) AND T4."isactive" = True AND (SELECT U0."id" FROM "campsservicepackage" U0 INNER JOIN "campsregion" U2 ON (U0."regionid" = U2."id") WHERE (U0."companyid" = 1 AND U0."option" = ("campsoffer"."activitytype") AND STContains(U2."locations", (T4."position"))) LIMIT 1) IS NOT NULL) GROUP BY "campsoffer"."id", (SELECT U0."id" FROM "campsservicepackage" U0 INNER JOIN "campsregion" U2 ON (U0."regionid" = U2."id") WHERE (U0."companyid" = 1 AND U0."option" = ("campsoffer"."activitytype") AND STContains(U2."locations", (T4."position"))) LIMIT 1), "campsprovider"."id" ORDER BY "camps_offer"."created" ASC

'only be used in a subquery.'         ) <ins>    def resolve_expression(self, *args, **kwargs):        col = super().resolve_expression(*args, **kwargs)        # FIXME: Rename possibly_multivalued to multivalued and fix detection        # for non-multivalued JOINs (e.g. foreign key fields). This should take        # into account only many-to-many and one-to-many relationships.        col.possibly_multivalued = LOOKUP_SEP in self.name        return col</ins>     def relabeled_clone(self, relabels):         return self


Question: Subquery annotations are omitted in group by query section if multiple annotation are declared Description (last modified by Johannes Maron) Sadly there is more regression in Django 3.0.2 even after . Background: It's the same query as . I tried upgrading to Django 3.0.2 and now I get duplicate results. Even tho they query should be distinct. Where on 2.2 the queryset yields 490 results, it's 519 on 3.0. A quick diff on the queries still reveals a different grouped by section: This is the new query on 3.0.2: SELECT DISTINCT "campsoffer"."id", "campsoffer"."title", "campsoffer"."slug", "campsoffer"."isactive", "campsoffer"."modified", "campsoffer"."created", "campsoffer"."providerid", "campsoffer"."activitytype", "campsoffer"."description", "campsoffer"."highlights", "campsoffer"."importantinformation", "campsoffer"."minage", "campsoffer"."maxage", "campsoffer"."food", "campsoffer"."video", "campsoffer"."accommodation", "campsoffer"."accommodationtype", "campsoffer"."roomtype", "campsoffer"."roomsizemin", "campsoffer"."roomsizemax", "campsoffer"."externalurl", "campsoffer"."applicationform", "campsoffer"."caseload", "campsoffer"."fieldtrips", MIN(T4."retailprice") AS "minretailprice", (SELECT U0."id" FROM "campsservicepackage" U0 INNER JOIN "campsregion" U2 ON (U0."regionid" = U2."id") WHERE (U0."companyid" = 1 AND U0."option" = "campsoffer"."activitytype" AND STContains(U2."locations", T4."position")) LIMIT 1) AS "inpackage", "campsprovider"."id", "campsprovider"."title", "campsprovider"."slug", "campsprovider"."isactive", "campsprovider"."modified", "campsprovider"."created", "campsprovider"."logo", "campsprovider"."description", "campsprovider"."video", "campsprovider"."externalurl", "campsprovider"."terms", "campsprovider"."cancellationpolicy", "campsprovider"."privacypolicy", "campsprovider"."applicationform" FROM "campsoffer" LEFT OUTER JOIN "campsbookingoption" ON ("campsoffer"."id" = "campsbookingoption"."offerid") INNER JOIN "campsprovider" ON ("campsoffer"."providerid" = "campsprovider"."id") INNER JOIN "campsbookingoption" T4 ON ("campsoffer"."id" = T4."offerid") WHERE ("campsoffer"."isactive" = True AND "campsprovider"."isactive" = True AND T4."end" = STATEMENTTIMESTAMP() AND T4."isactive" = True AND "campsoffer"."maxage" = 5 AND "campsoffer"."minage" <= 13 AND (SELECT U0."id" FROM "campsservicepackage" U0 INNER JOIN "campsregion" U2 ON (U0."regionid" = U2."id") WHERE (U0."companyid" = 1 AND U0."option" = "campsoffer"."activitytype" AND STContains(U2."locations", T4."position")) LIMIT 1) IS NOT NULL) GROUP BY "campsoffer"."id", T4."position", "campsprovider"."id" ORDER BY "campsoffer"."created" ASC And what it was (and should be) on 2.2.9: SELECT DISTINCT "campsoffer"."id", "campsoffer"."title", "campsoffer"."slug", "campsoffer"."isactive", "campsoffer"."modified", "campsoffer"."created", "campsoffer"."providerid", "campsoffer"."activitytype", "campsoffer"."description", "campsoffer"."highlights", "campsoffer"."importantinformation", "campsoffer"."minage", "campsoffer"."maxage", "campsoffer"."food", "campsoffer"."video", "campsoffer"."accommodation", "campsoffer"."accommodationtype", "campsoffer"."roomtype", "campsoffer"."roomsizemin", "campsoffer"."roomsizemax", "campsoffer"."externalurl", "campsoffer"."applicationform", "campsoffer"."caseload", "campsoffer"."fieldtrips", MIN(T4."retailprice") AS "minretailprice", (SELECT U0."id" FROM "campsservicepackage" U0 INNER JOIN "campsregion" U2 ON (U0."regionid" = U2."id") WHERE (U0."companyid" = 1 AND U0."option" = ("campsoffer"."activitytype") AND STContains(U2."locations", (T4."position"))) LIMIT 1) AS "inpackage", "campsprovider"."id", "campsprovider"."title", "campsprovider"."slug", "campsprovider"."isactive", "campsprovider"."modified", "campsprovider"."created", "campsprovider"."logo", "campsprovider"."description", "campsprovider"."video", "campsprovider"."externalurl", "campsprovider"."terms", "campsprovider"."cancellationpolicy", "campsprovider"."privacypolicy", "campsprovider"."applicationform" FROM "campsoffer" LEFT OUTER JOIN "campsbookingoption" ON ("campsoffer"."id" = "campsbookingoption"."offerid") INNER JOIN "campsprovider" ON ("campsoffer"."providerid" = "campsprovider"."id") INNER JOIN "campsbookingoption" T4 ON ("campsoffer"."id" = T4."offerid") WHERE ("campsoffer"."isactive" = True AND "campsprovider"."isactive" = True AND T4."end" = (STATEMENTTIMESTAMP()) AND T4."isactive" = True AND (SELECT U0."id" FROM "campsservicepackage" U0 INNER JOIN "campsregion" U2 ON (U0."regionid" = U2."id") WHERE (U0."companyid" = 1 AND U0."option" = ("campsoffer"."activitytype") AND STContains(U2."locations", (T4."position"))) LIMIT 1) IS NOT NULL) GROUP BY "campsoffer"."id", (SELECT U0."id" FROM "campsservicepackage" U0 INNER JOIN "campsregion" U2 ON (U0."regionid" = U2."id") WHERE (U0."companyid" = 1 AND U0."option" = ("campsoffer"."activitytype") AND STContains(U2."locations", (T4."position"))) LIMIT 1), "campsprovider"."id" ORDER BY "camps_offer"."created" ASC

class Col(Expression):      contains_column_references = True<ins>    possibly_multivalued = False</ins>      def __init__(self, alias, target, output_field=None):         if output_field is None:


Question: Subquery annotations are omitted in group by query section if multiple annotation are declared Description (last modified by Johannes Maron) Sadly there is more regression in Django 3.0.2 even after . Background: It's the same query as . I tried upgrading to Django 3.0.2 and now I get duplicate results. Even tho they query should be distinct. Where on 2.2 the queryset yields 490 results, it's 519 on 3.0. A quick diff on the queries still reveals a different grouped by section: This is the new query on 3.0.2: SELECT DISTINCT "campsoffer"."id", "campsoffer"."title", "campsoffer"."slug", "campsoffer"."isactive", "campsoffer"."modified", "campsoffer"."created", "campsoffer"."providerid", "campsoffer"."activitytype", "campsoffer"."description", "campsoffer"."highlights", "campsoffer"."importantinformation", "campsoffer"."minage", "campsoffer"."maxage", "campsoffer"."food", "campsoffer"."video", "campsoffer"."accommodation", "campsoffer"."accommodationtype", "campsoffer"."roomtype", "campsoffer"."roomsizemin", "campsoffer"."roomsizemax", "campsoffer"."externalurl", "campsoffer"."applicationform", "campsoffer"."caseload", "campsoffer"."fieldtrips", MIN(T4."retailprice") AS "minretailprice", (SELECT U0."id" FROM "campsservicepackage" U0 INNER JOIN "campsregion" U2 ON (U0."regionid" = U2."id") WHERE (U0."companyid" = 1 AND U0."option" = "campsoffer"."activitytype" AND STContains(U2."locations", T4."position")) LIMIT 1) AS "inpackage", "campsprovider"."id", "campsprovider"."title", "campsprovider"."slug", "campsprovider"."isactive", "campsprovider"."modified", "campsprovider"."created", "campsprovider"."logo", "campsprovider"."description", "campsprovider"."video", "campsprovider"."externalurl", "campsprovider"."terms", "campsprovider"."cancellationpolicy", "campsprovider"."privacypolicy", "campsprovider"."applicationform" FROM "campsoffer" LEFT OUTER JOIN "campsbookingoption" ON ("campsoffer"."id" = "campsbookingoption"."offerid") INNER JOIN "campsprovider" ON ("campsoffer"."providerid" = "campsprovider"."id") INNER JOIN "campsbookingoption" T4 ON ("campsoffer"."id" = T4."offerid") WHERE ("campsoffer"."isactive" = True AND "campsprovider"."isactive" = True AND T4."end" = STATEMENTTIMESTAMP() AND T4."isactive" = True AND "campsoffer"."maxage" = 5 AND "campsoffer"."minage" <= 13 AND (SELECT U0."id" FROM "campsservicepackage" U0 INNER JOIN "campsregion" U2 ON (U0."regionid" = U2."id") WHERE (U0."companyid" = 1 AND U0."option" = "campsoffer"."activitytype" AND STContains(U2."locations", T4."position")) LIMIT 1) IS NOT NULL) GROUP BY "campsoffer"."id", T4."position", "campsprovider"."id" ORDER BY "campsoffer"."created" ASC And what it was (and should be) on 2.2.9: SELECT DISTINCT "campsoffer"."id", "campsoffer"."title", "campsoffer"."slug", "campsoffer"."isactive", "campsoffer"."modified", "campsoffer"."created", "campsoffer"."providerid", "campsoffer"."activitytype", "campsoffer"."description", "campsoffer"."highlights", "campsoffer"."importantinformation", "campsoffer"."minage", "campsoffer"."maxage", "campsoffer"."food", "campsoffer"."video", "campsoffer"."accommodation", "campsoffer"."accommodationtype", "campsoffer"."roomtype", "campsoffer"."roomsizemin", "campsoffer"."roomsizemax", "campsoffer"."externalurl", "campsoffer"."applicationform", "campsoffer"."caseload", "campsoffer"."fieldtrips", MIN(T4."retailprice") AS "minretailprice", (SELECT U0."id" FROM "campsservicepackage" U0 INNER JOIN "campsregion" U2 ON (U0."regionid" = U2."id") WHERE (U0."companyid" = 1 AND U0."option" = ("campsoffer"."activitytype") AND STContains(U2."locations", (T4."position"))) LIMIT 1) AS "inpackage", "campsprovider"."id", "campsprovider"."title", "campsprovider"."slug", "campsprovider"."isactive", "campsprovider"."modified", "campsprovider"."created", "campsprovider"."logo", "campsprovider"."description", "campsprovider"."video", "campsprovider"."externalurl", "campsprovider"."terms", "campsprovider"."cancellationpolicy", "campsprovider"."privacypolicy", "campsprovider"."applicationform" FROM "campsoffer" LEFT OUTER JOIN "campsbookingoption" ON ("campsoffer"."id" = "campsbookingoption"."offerid") INNER JOIN "campsprovider" ON ("campsoffer"."providerid" = "campsprovider"."id") INNER JOIN "campsbookingoption" T4 ON ("campsoffer"."id" = T4."offerid") WHERE ("campsoffer"."isactive" = True AND "campsprovider"."isactive" = True AND T4."end" = (STATEMENTTIMESTAMP()) AND T4."isactive" = True AND (SELECT U0."id" FROM "campsservicepackage" U0 INNER JOIN "campsregion" U2 ON (U0."regionid" = U2."id") WHERE (U0."companyid" = 1 AND U0."option" = ("campsoffer"."activitytype") AND STContains(U2."locations", (T4."position"))) LIMIT 1) IS NOT NULL) GROUP BY "campsoffer"."id", (SELECT U0."id" FROM "campsservicepackage" U0 INNER JOIN "campsregion" U2 ON (U0."regionid" = U2."id") WHERE (U0."companyid" = 1 AND U0."option" = ("campsoffer"."activitytype") AND STContains(U2."locations", (T4."position"))) LIMIT 1), "campsprovider"."id" ORDER BY "camps_offer"."created" ASC

def get_group_by_cols(self, alias=None):         if alias:             return [Ref(alias, self)]        return self.query.get_external_cols()        external_cols = self.query.get_external_cols()        if any(col.possibly_multivalued for col in external_cols):            return [self]        return external_cols   class Exists(Subquery):


Question: forms.ModelMultipleChoiceField should use "invalidlist" as error message key Description The MultipleChoiceField uses "invalidlist", but ModelMultipleChoiceField uses "list" as the key for the similar error message.

Helper functions for creating Form classes from Django models and database field objects. """import warnings from itertools import chain  from django.core.exceptions import (