Question: def formset_factory(form, formset=BaseFormSet, extra=1, can_order=False,                     can_delete=False, max_num=None, validate_max=False,                    min_num=None, validate_min=False):                    min_num=None, validate_min=False, absolute_max=None):     """Return a FormSet for the given form class."""     if min_num is None:         min_num = DEFAULT_MIN_NUM     if max_num is None:         max_num = DEFAULT_MAX_NUM    # hard limit on forms instantiated, to prevent memory-exhaustion attacks    # limit is simply max_num + DEFAULT_MAX_NUM (which is 2*DEFAULT_MAX_NUM    # if max_num is None in the first place)    absolute_max = max_num + DEFAULT_MAX_NUM    # absolute_max is a hard limit on forms instantiated, to prevent    # memory-exhaustion attacks. Default to max_num + DEFAULT_MAX_NUM    # (which is 2 * DEFAULT_MAX_NUM if max_num is None in the first place).    if absolute_max is None:        absolute_max = max_num + DEFAULT_MAX_NUM    if max_num > absolute_max:        raise ValueError(            "'absolute_max' must be greater or equal to 'max_num'."        )     attrs = {         'form': form,         'extra': extra,diff --git a/django/forms/models.py b/django/forms/models.py-- a/django/forms/models.py++ b/django/forms/models.py

Add an absolutemax parameter to formsetfactory Description The documentation at  seems to indicate (if I understood it correctly) that the purpose of the maxnum parameter is to prevent that someone sends a manipulated, excessively large value in the hidden form field that states the number of (extra) forms that are submitted, whereas it is not (directly) related to the number of forms that are actually POSTed, or initialized via parameter initials. However, following the example at that page, with MyInitials being a list of e.g. 1500 initial values and request.POST containing more than 1500 formsets: ArticleFormSet = formsetfactory(ArticleForm, extra=0) formset1 = ArticleFormSet(initial=MyInitials) formset2 = ArticleFormSet(request.POST) Now, accessing [1000] throws an IndexError exception. The maxnum is at its default value of 1000, but in the above context, it is not expected that formset1 or formset2 is reduced to maxnum forms -- I'd have expected each to have the full number of forms as initialized. Related thread at django-users:


Question: can_order=False, max_num=None, fields=None, exclude=None,                          widgets=None, validate_max=False, localized_fields=None,                          labels=None, help_texts=None, error_messages=None,                         min_num=None, validate_min=False, field_classes=None):                         min_num=None, validate_min=False, field_classes=None,                         absolute_max=None):     """Return a FormSet class for the given Django model class."""     meta = getattr(form, 'Meta', None)     if (getattr(meta, 'fields', fields) is None and


Question: error_messages=error_messages, field_classes=field_classes)     FormSet = formset_factory(form, formset, extra=extra, min_num=min_num, max_num=max_num,                               can_order=can_order, can_delete=can_delete,                              validate_min=validate_min, validate_max=validate_max)                              validate_min=validate_min, validate_max=validate_max,                              absolute_max=absolute_max)     FormSet.model = model     return FormSet


Question: can_delete=True, max_num=None, formfield_callback=None,                           widgets=None, validate_max=False, localized_fields=None,                           labels=None, help_texts=None, error_messages=None,                          min_num=None, validate_min=False, field_classes=None):                          min_num=None, validate_min=False, field_classes=None,                          absolute_max=None):     """     Return an ``InlineFormSet`` for the given kwargs.


Question: 'help_texts': help_texts,         'error_messages': error_messages,         'field_classes': field_classes,<ins>        'absolute_max': absolute_max,</ins>     }     FormSet = modelformset_factory(model, **kwargs)     FormSet.fk = fk


Question: from django.conf import settings from django.utils import timezone<ins>from django.utils.deprecation import RemovedInDjango40Warning</ins> from django.utils.html import conditional_escape, format_html from django.utils.lorem_ipsum import paragraphs, words from django.utils.safestring import mark_safe

Remove ifequal from the template language. Description No modern project uses ifequal. No one recommends it. I argue it is taking up valuable bytes in the project. Let's remove it.


Question: class IfEqualNode(Node):<ins>    # RemovedInDjango40Warning.</ins>     child_nodelists = ('nodelist_true', 'nodelist_false')      def __init__(self, var1, var2, nodelist_true, nodelist_false, negate):


Question: def do_ifequal(parser, token, negate):<ins>    # RemovedInDjango40Warning.</ins>     bits = list(token.split_contents())     if len(bits) != 3:         raise TemplateSyntaxError("%r takes two arguments" % bits[0])


Question: ...         {% endifnotequal %}     """<ins>    warnings.warn(        'The {% ifequal %} template tag is deprecated in favor of {% if %}.',        RemovedInDjango40Warning,    )</ins>     return do_ifequal(parser, token, False)


Question: Output the contents of the block if the two arguments are not equal.     See ifequal.     """<ins>    warnings.warn(        'The {% ifnotequal %} template tag is deprecated in favor of '        '{% if %}.',        RemovedInDjango40Warning,    )</ins>     return do_ifequal(parser, token, True)


Question: import warnings from django.urls import include, re_pathfrom django.utils.deprecation import RemovedInDjango40Warning from django.views import defaults  __all__ = ['handler400', 'handler403', 'handler404', 'handler500', 'include', 'url']

Deprecate (). Description The docs for  say: This function is an alias to (). It’s likely to be deprecated in a future release. It looks like the change was made in this commit back in 2016 (Django 2.0). Given some years have passed, is it now the time to deprecate this function?


Question: def url(regex, view, kwargs=None, name=None):    warnings.warn(        'django.conf.urls.url() is deprecated in favor of '        'django.urls.re_path().',        RemovedInDjango40Warning,    )     return re_path(regex, view, kwargs, name)diff --git a/django/views/i18n.py b/django/views/i18n.py-- a/django/views/i18n.py++ b/django/views/i18n.py


Question: Return the selected language catalog as a JavaScript library.      Receive the list of packages to check for translations in the `packages`    kwarg either from the extra dictionary passed to the url() function or as a    plus-sign delimited string from the request. Default is 'django.conf'.    kwarg either from the extra dictionary passed to the path() function or as    a plus-sign delimited string from the request. Default is 'django.conf'.      You can override the gettext domain for this view, but usually you don't     want to do that as JavaScript messages go to the djangojs domain. This


Question: id='models.W038',                     )                 )<ins>            fields = (                field                for constraint in cls._meta.constraints if isinstance(constraint, UniqueConstraint)                for field in constraint.fields            )            errors.extend(cls._check_local_fields(fields, 'constraints'))</ins>         return errors

Add check for fields of UniqueConstraints. Description (last modified by Marnanel Thurman) When a model gains a UniqueConstraint, makemigrations doesn't check that the fields named therein actually exist. This is in contrast to the older uniquetogether syntax, which raises models.E012 if the fields don't exist. In the attached demonstration, you'll need to uncomment "withuniquetogether" in in order to show that uniquetogether raises E012.


Question: else:                         _cls = None                 except (FieldDoesNotExist, AttributeError):                    if fld is None or fld.get_transform(part) is None:                    if fld is None or (                        fld.get_transform(part) is None and fld.get_lookup(part) is None                    ):                         errors.append(                             checks.Error(                                 "'ordering' refers to the nonexistent field, "

models.E015 is raised when ordering uses lookups that are not transforms. Description check SystemCheckError: System check identified some issues: ERRORS: app.Stock: (models.E015) 'ordering' refers to the nonexistent field, related field, or lookup 'supplyproductparentisnull'. However this ordering works fine: list(Stock.objects.orderby('supplyproductparentisnull').valueslist('pk', flat=True)[:5]) [1292, 1293, 1300, 1295, 1294] list(Stock.objects.orderby('-supplyproductparentisnull').valueslist('pk', flat=True)[:5]) [108, 109, 110, 23, 107] I believe it was fine until was implemented. is a foreign key to Supply, Supply.product is a foreign key to Product, is a ForeignKey('self', models.CASCADE, null=True)


Question: ignore_patterns = ['CVS', '.*', '*~']      def ready(self):        checks.register(check_finders, 'staticfiles')        checks.register(check_finders, checks.Tags.staticfiles)diff --git a/django/contrib/staticfiles/management/commands/collectstatic.py b/django/contrib/staticfiles/management/commands/collectstatic.py-- a/django/contrib/staticfiles/management/commands/collectstatic.py++ b/django/contrib/staticfiles/management/commands/collectstatic.py

collectstatic doesn't run staticfiles checks. Description Piecing this together from a user who is having trouble with staticfiles (who doesn't, at least once) on IRC. the system checks framework has a checkfinders which is invoked if django.contrib.staticfiles is in your INSTALLEDAPPS which defers checks to each individually configured finder as () - this accounts for running the following line: if not isinstance(settings.STATICFILESDIRS, (list, tuple)): which is intended to catch the problem scenario of STATICFILESDIRS = () being edited to STATICFILESDIRS = ("mypath") with no trailing comma to make it a tuple, rather than an iterable string. However, the collectstatic management command has requiressystemchecks = False so it appears possible to edit the value and directly run python collectstatic without the type being checked or the error raised. Naively, I'm assuming that something like the following needs to occur: for finder in getfinders(): () # ignoring NotImplementedError # then somehow surface any errors back to stderr as if I've not delved deeply into the "surface errors" part.


Question: from django.apps import apps from django.contrib.staticfiles.finders import get_finders from django.contrib.staticfiles.storage import staticfiles_storage<ins>from django.core.checks import Tags</ins> from django.core.files.storage import FileSystemStorage from django.core.management.base import BaseCommand, CommandError from django.core.management.color import no_style


Question: return True      def add_arguments(self, parser):<ins>        parser.add_argument(            '--skip-checks', action='store_true',            help='Skip system checks.',        )</ins>         parser.add_argument(             '--noinput', '--no-input', action='store_false', dest='interactive',             help="Do NOT prompt the user for input of any kind.",


Question: def handle(self, **options):         self.set_options(**options)        if not options['skip_checks']:            self.check(tags=[Tags.staticfiles])          message = ['\n']         if self.dry_run:diff --git a/django/core/checks/registry.py b/django/core/checks/registry.py-- a/django/core/checks/registry.py++ b/django/core/checks/registry.py


Question: models = 'models'     security = 'security'     signals = 'signals'<ins>    staticfiles = 'staticfiles'</ins>     templates = 'templates'     translation = 'translation'     urls = 'urls'


Question: """     Check that the admin's dependencies are correctly installed.     """<ins>    from django.contrib.admin.sites import all_sites</ins>     if not apps.is_installed('django.contrib.admin'):         return []     errors = []

Document admin's requirement on django.template.contextprocessors.request context processor. Description Since commit , the admin templates use the implied request variable normally by django.template.contextprocessors.request. As Django templates silence errors, this went unnoticed during testing, and won't immediately break the templates, but certain expected rendering features won't work. Django should document this change: In the release notes (provide a deprecation period where it is a warning only) In the admin docs In the system check framework as a warning (but eventually an error)


Question: "the admin application.",                 id='admin.E404',             ))<ins>        sidebar_enabled = any(site.enable_nav_sidebar for site in all_sites)        if (sidebar_enabled and 'django.template.context_processors.request'                not in django_templates_instance.context_processors):            errors.append(checks.Warning(                "'django.template.context_processors.request' must be enabled "                "in DjangoTemplates (TEMPLATES) in order to use the admin "                "navigation sidebar.",                id='admin.W411',            ))</ins>      if not _contains_subclass('django.contrib.auth.middleware.AuthenticationMiddleware', settings.MIDDLEWARE):         errors.append(checks.Error(


Question: """         Return a new QuerySet instance that will select only distinct results.         """<ins>        self._not_support_combined_queries('distinct')</ins>         assert not self.query.is_sliced, \             "Cannot create distinct fields once a slice has been taken."         obj = self._chain()

Union queryset should raise on distinct(). Description (last modified by Sielc Technologies) After using .annotate() on 2 different querysets and then .union() .distinct() will not affect the queryset def setUp(self) -None: user = self.getorcreateadminuser() (user, name="Sam1") (user, name="Sam2 acid") (user, name="Sam3") (user, name="Sam4 acid") (user, name="Dub") (user, name="Dub") (user, name="Dub") = user def testunionannotateddiffdistinct(self): qs = () qs1 = (name='Dub').annotate(rank=Value(0, IntegerField())) qs2 = (name='Sam1').annotate(rank=Value(1, IntegerField())) qs = (qs2) qs = qs.orderby('name').distinct('name') # THIS DISTINCT DOESN'T WORK self.assertEqual((), 2) expected to get wrapped union SELECT DISTINCT ON () * FROM (SELECT ... UNION SELECT ...) AS siebox_sample


Question: else:                 # Reuse aliases of expressions already selected in subquery.                 for col_alias, selected_annotation in self.annotation_select.items():                    if selected_annotation == expr:                    if selected_annotation is expr:                         new_expr = Ref(col_alias, expr)                         break                 else:

Alias used in aggregate filtering is incorrect. Description With the following queryset: IndicatorValue.objects .values("freight") .annotate( loadingtime=Min("datetime", filter=Q(type=IndicatorValue.TYPEFREIGHTCREATED)) - Max("datetime", filter=Q(type=IndicatorValue.TYPEFREIGHTCOMPLETED)), hastoploading=Exists(OrderItemResult.objects.orderby().filter(orderlineorderfreight=OuterRef("freight"), loadingarmloadingtype=LoadingArm.LOADINGTYPETOP, ).values('pk')), hasbottomloading=Exists(OrderItemResult.objects.orderby().filter(orderlineorderfreight=OuterRef("freight"), loadingarmloadingtype=LoadingArm.LOADINGTYPEBOTTOM, ).values('pk')) ) .aggregate( topmin=Min("loadingtime", filter=Q(hastoploading=True, hasbottomloading=False)) ) I get the following SQL generated for the aggregate (notice that both alias used are the same in the SQL, whereas they are not in the queryset): MIN("loadingtime") FILTER (WHERE ("hastoploading" = false AND "hastoploading" = true)) The full SQL generated is: SELECT MIN("loadingtime") FILTER (WHERE ("hastoploading" = false AND "hastoploading" = true)) FROM (SELECT "indicatorsindicatorvalue"."freightid" AS Col1, (MIN("indicatorsindicatorvalue"."datetime") FILTER (WHERE "indicatorsindicatorvalue"."type" = \'freightcreated\') - MAX("indicatorsindicatorvalue"."datetime") FILTER (WHERE "indicatorsindicatorvalue"."type" = \'freightcompleted\')) AS "loadingtime", EXISTS(SELECT U0."id" FROM "ordersorderitemresult" U0 INNER JOIN "loadingterminalloadingarm" U1 ON (U0."loadingarmid" = U1."id") INNER JOIN "ordersorderitem" U2 ON (U0."orderlineid" = U2."id") INNER JOIN "ordersorder" U3 ON (U2."orderid" = U3."id") WHERE (U1."loadingtype" = \'TOP\' AND U3."freightid" = "indicatorsindicatorvalue"."freightid")) AS "hastoploading", EXISTS(SELECT U0.


Question: else:                 # Reuse aliases of expressions already selected in subquery.                 for col_alias, selected_annotation in self.annotation_select.items():                    if selected_annotation == expr:                    if selected_annotation is expr:                         new_expr = Ref(col_alias, expr)                         break                 else:

"id" FROM "ordersorderitemresult" U0 INNER JOIN "loadingterminalloadingarm" U1 ON (U0."loadingarmid" = U1."id") INNER JOIN "ordersorderitem" U2 ON (U0."orderlineid" = U2."id") INNER JOIN "ordersorder" U3 ON (U2."orderid" = U3."id") WHERE (U1."loadingtype" = \'BOTTOM\' AND U3."freightid" = "indicatorsindicatorvalue"."freightid")) AS "hasbottomloading" FROM "indicatorsindicatorvalue" WHERE "indicatorsindicatorvalue"."deleted" IS NULL GROUP BY "indicatorsindicatorvalue"."freightid", "hastoploading", "hasbottomloading") subquery It works fine with Django 2.2 (which does not use alias there if I'm not mistaken).


Question: from urllib.parse import urlparse from urllib.request import url2pathname <ins>from asgiref.sync import sync_to_async</ins> from django.conf import settings from django.contrib.staticfiles import utils from django.contrib.staticfiles.views import serve

Add getresponseasync for ASGIStaticFilesHandler Description It looks like the StaticFilesHandlerMixin is missing the the async response function. Without this, when trying to use the ASGIStaticFilesHandler, this is the traceback: Exception inside application: 'NoneType' object is not callable Traceback (most recent call last): File ".../lib/python3.7/site-", line 30, in asgi await (scope, receive, send) File "", line 86, in call return await super().call(scope, receive, send) File "", line 161, in call response = await self.getresponseasync(request) File "", line 148, in getresponseasync response = await self.middlewarechain(request) TypeError: 'NoneType' object is not callable


Question: except Http404 as e:             return response_for_exception(request, e) <ins>    async def get_response_async(self, request):        try:            return await sync_to_async(self.serve)(request)        except Http404 as e:            return await sync_to_async(response_for_exception)(request, e)</ins>  class StaticFilesHandler(StaticFilesHandlerMixin, WSGIHandler):     """


Question: # Library management from .library import Library                                            # NOQA isort:skip # Import the .autoreload module to trigger the registrations of signals.from . import autoreload                                                # NOQA isort:skip  __all__ += ('Template', 'Context', 'RequestContext')diff --git a/django/template/autoreload.py b/django/template/autoreload.pynew file mode 100644-- /dev/null++ b/django/template/autoreload.py

Implement autoreload behaviour for cached template loader Description It would be nice to be able get the speed benefit of the cached template loader during development, but without the downside of having to restart the server every time you change a template. It turns out it's possible with just a few changes. Because it's not really possible to configure the cached template loader I did have to base this patch on the fix for . Enabling autoreloading for the cached template loader would work like this: TEMPLATES = [{ 'BACKEND': '', 'DIRS': [(BASEDIR, 'templates')], 'APPDIRS': True 'OPTIONS': { 'cache_templates': True, 'autoreload': DEBUG } }]


Question: from django.dispatch import receiverfrom django.template import enginesfrom django.template.backends.django import DjangoTemplatesfrom django.utils.autoreload import (    autoreload_started, file_changed, is_django_path,)def get_template_directories():    # Iterate through each template backend and find    # any template_loader that has a 'get_dirs' method.    # Collect the directories, filtering out Django templates.    items = set()    for backend in engines.all():        if not isinstance(backend, DjangoTemplates):            continue        items.update(backend.engine.dirs)        for loader in backend.engine.template_loaders:            if not hasattr(loader, 'get_dirs'):                continue            items.update(                directory                for directory in loader.get_dirs()                if not is_django_path(directory)            )    return itemsdef reset_loaders():    for backend in engines.all():        if not isinstance(backend, DjangoTemplates):            continue        for loader in backend.engine.template_loaders:            loader.reset()@receiver(autoreload_started, dispatch_uid='template_loaders_watch_changes')def watch_for_template_changes(sender, **kwargs):    for directory in get_template_directories():        sender.watch_dir(directory, '**/*')@receiver(file_changed, dispatch_uid='template_loaders_file_changed')def template_changed(sender, file_path, **kwargs):    for template_dir in get_template_directories():        if template_dir in file_path.parents:            reset_loaders()            return Truediff --git a/django/template/loaders/cached.py b/django/template/loaders/cached.py-- a/django/template/loaders/cached.py++ b/django/template/loaders/cached.py


Question: self.loaders = engine.get_template_loaders(loaders)         super().__init__(engine)     def get_dirs(self):        for loader in self.loaders:            if hasattr(loader, "get_dirs"):                yield from loader.get_dirs()     def get_contents(self, origin):         return origin.loader.get_contents(origin) diff --git a/django/utils/autoreload.py b/django/utils/autoreload.py-- a/django/utils/autoreload.py++ b/django/utils/autoreload.py


Question: from types import ModuleType from zipimport import zipimporter <ins>import django</ins> from django.apps import apps from django.core.signals import request_finished from django.dispatch import Signal


Question: pywatchman = None  <ins>def is_django_module(module):    """Return True if the given module is nested under Django."""    return module.__name__.startswith('django.')def is_django_path(path):    """Return True if the given file path is nested under Django."""    return Path(django.__file__).parent in Path(path).parents</ins> def check_errors(fn):     @functools.wraps(fn)     def wrapper(*args, **kwargs):


Question: def _subscribe(self, directory, name, expression):         root, rel_path = self._watch_root(directory)        # Only receive notifications of files changing, filtering out other types        # like special files: https://facebook.github.io/watchman/docs/type        only_files_expression = [            'allof',            ['anyof', ['type', 'f'], ['type', 'l']],            expression        ]         query = {            'expression': expression,            'expression': only_files_expression,             'fields': ['name'],             'since': self._get_clock(root),             'dedup_results': True,diff --git a/django/utils/translation/reloader.py b/django/utils/translation/reloader.py-- a/django/utils/translation/reloader.py++ b/django/utils/translation/reloader.py


Question: from asgiref.local import Local  from django.apps import appsdef _is_django_module(module):    """Return True if the given module is nested under Django."""    return module.__name__.startswith('django.')from django.utils.autoreload import is_django_module   def watch_for_translation_changes(sender, **kwargs):


Question: directories.extend(             Path(config.path) / 'locale'             for config in apps.get_app_configs()            if not _is_django_module(config.module)            if not is_django_module(config.module)         )         directories.extend(Path(p) for p in settings.LOCALE_PATHS)         for path in directories:


Question: from importlib import import_module  from django.conf import settingsfrom django.core.management.base import BaseCommandfrom django.core.management.base import BaseCommand, CommandError   class Command(BaseCommand):

Raise CommandError when clearsessions is called on a backend not implementing clearexpired() Description Formal ticket for the issue raised by François Freitag in PR. clearsessions writes to stderr directly when a backend has not implemented clearexpired(). François notes that the recommended behavior is to raise CommandError.


Question: try:             engine.SessionStore.clear_expired()         except NotImplementedError:            self.stderr.write(            raise CommandError(                 "Session engine '%s' doesn't support clearing expired "                 "sessions." % settings.SESSION_ENGINE             )


Question: def __init__(self, queryset, output_field=None, **extra):         self.query = queryset.query         self.extra = extra        # Prevent the QuerySet from being evaluated.        self.queryset = queryset._chain(_result_cache=[], prefetch_done=True)         super().__init__(output_field)      def __getstate__(self):         state = super().__getstate__()        state.pop('_constructor_args', None)        args, kwargs = state['_constructor_args']        if args:            args = (self.queryset, *args[1:])        else:            kwargs['queryset'] = self.queryset        state['_constructor_args'] = args, kwargs         return state      def get_source_expressions(self):

() doesn't work properly for resolved subqueries. Description () doesn't work properly for resolved subqueries, basically all resolved subqueries are now equal. Regression in .


Question: conditional = False      def __init__(self, condition=None, then=None, **lookups):        if lookups and condition is None:            condition, lookups = Q(**lookups), None        if lookups:            if condition is None:                condition, lookups = Q(**lookups), None            elif getattr(condition, 'conditional', False):                condition, lookups = Q(condition, **lookups), None         if condition is None or not getattr(condition, 'conditional', False) or lookups:             raise TypeError(                 'When() supports a Q object, a boolean expression, or lookups '

Cannot mix Exists expression with keyword arguments to When Description (last modified by Ryan Heard) I don't seem to be able to provide an Exists expression to When alongside keyword arguments like you can with filter. For instance, consider: class State(models.Model): pass class County(models.Model): name = CharField(maxlength=50) state = ForeignKey(State, relatedname='counties') I can execute the following query just fine: ( Exists((counties=OuterRef('pk'), name="Texas")), name="Dallas", ) But a similar query using When does not work: County.objects.annotate( status=Case( When(Exists((counties=OuterRef('pk'), name="Texas")), name="Dallas", then=Value("DALLAS COUNTY")), default=Value("ELSEWHERE"), )) TypeError: When() supports a Q object, a boolean expression, or lookups as a condition. Instead the arguments must be wrapped in a Q object: County.objects.annotate( status=Case( When(Q(Exists((counties=OuterRef('pk'), name="Texas")), name="Dallas"), then=Value("DALLAS COUNTY")), default=Value("ELSEWHERE"), )) This is inconvenient and inconsistent with how filter works, as shown. When's init method can be modified to allow similar input as filter. Code is in a branch in my repo, but as this is my first time contributing to Django, I want to make sure I open a ticket and get feedback first. Also I wasn't sure how to classify this. I wasn't sure if it was a bug, as I wasn't sure if it was designed this way.


Question: from django.db.models.functions import Coalesce from django.db.models.manager import Manager from django.db.models.options import Optionsfrom django.db.models.query import Qfrom django.db.models.query import F, Q from django.db.models.signals import (     class_prepared, post_init, post_save, pre_init, pre_save, )

Check that and UniqueConstraint.condition don't span joins. Description (last modified by Mariusz Felisiak) Similar to but for check and condition. Not everyone is familiar with the fact database level constraint cannot span across tables and might be tempted to do class Person(models.Model): age = models.PositiveSmallIntegerField() parent = models.ForeignKey(self) class Meta: constraints = { CheckConstraint( name='ageltparent', check=Q(agelt=parentage) ), } Which we'll happily create migrations for but we'll then crash because we prevent JOINs when resolving check.


Question: return errors <ins>    @classmethod    def _get_expr_references(cls, expr):        if isinstance(expr, Q):            for child in expr.children:                if isinstance(child, tuple):                    lookup, value = child                    yield tuple(lookup.split(LOOKUP_SEP))                    yield from cls._get_expr_references(value)                else:                    yield from cls._get_expr_references(child)        elif isinstance(expr, F):            yield tuple(expr.name.split(LOOKUP_SEP))        elif hasattr(expr, 'get_source_expressions'):            for src_expr in expr.get_source_expressions():                yield from cls._get_expr_references(src_expr)</ins>     @classmethod     def _check_constraints(cls, databases):         errors = []


Question: id='models.W039',                     )                 )            fields = chain.from_iterable(            fields = set(chain.from_iterable(                 (*constraint.fields, *constraint.include)                 for constraint in cls._meta.constraints if isinstance(constraint, UniqueConstraint)            )            ))            references = set()            for constraint in cls._meta.constraints:                if isinstance(constraint, UniqueConstraint):                    if (                        connection.features.supports_partial_indexes or                        'supports_partial_indexes' not in cls._meta.required_db_features                    ) and isinstance(constraint.condition, Q):                        references.update(cls._get_expr_references(constraint.condition))                elif isinstance(constraint, CheckConstraint):                    if (                        connection.features.supports_table_check_constraints or                        'supports_table_check_constraints' not in cls._meta.required_db_features                    ) and isinstance(constraint.check, Q):                        references.update(cls._get_expr_references(constraint.check))            for field_name, *lookups in references:                # pk is an alias that won't be found by opts.get_field.                if field_name != 'pk':                    fields.add(field_name)                if not lookups:                    # If it has no lookups it cannot result in a JOIN.                    continue                try:                    if field_name == 'pk':                        field = cls._meta.pk                    else:                        field = cls._meta.get_field(field_name)                    if not field.is_relation or field.many_to_many or field.one_to_many:                        continue                except FieldDoesNotExist:                    continue                # JOIN must happen at the first lookup.                first_lookup = lookups[0]                if (                    field.get_transform(first_lookup) is None and                    field.get_lookup(first_lookup) is None                ):                    errors.append(                        checks.Error(                            "'constraints' refers to the joined field '%s'."                            % LOOKUP_SEP.join([field_name] + lookups),                            obj=cls,                            id='models.E041',                        )                    )             errors.extend(cls._check_local_fields(fields, 'constraints'))         return errors


Question: elif isinstance(value, bool):  # Make sure booleans don't get treated as numbers         return str(value)     elif isinstance(value, (decimal.Decimal, float, int)):<ins>        if use_l10n is False:            return str(value)</ins>         return number_format(value, use_l10n=use_l10n)     elif isinstance(value, datetime.datetime):         return date_format(value, 'DATETIME_FORMAT', use_l10n=use_l10n)

Geodjango js template should use for float values to avoid DECIMALSEPARATOR ruin the js syntax Description should use |safe on float values to avoid DECIMALSEPARATOR (and probably other settings in this category) ruin the js syntax by adding unexpected characters instead of dot.


Question: from django.core.exceptions import EmptyResultSet, FieldError from django.db import DatabaseError, NotSupportedError from django.db.models.constants import LOOKUP_SEPfrom django.db.models.expressions import OrderBy, Random, RawSQL, Ref, Valuefrom django.db.models.expressions import F, OrderBy, Random, RawSQL, Ref, Value from django.db.models.functions import Cast from django.db.models.query_utils import Q, select_related_descend from django.db.models.sql.constants import (

orderby() with expressions crashes on union() querysets. Description (last modified by Mariusz Felisiak) I have read the recent tickets about unions and orderby (, , ) , and my bug is slightly different, so I hope it's not a duplicate. Let's consider two similar models: class EntityA(models.Model): namea = models.CharField(maxlength=128, null=True) dta = models.DateTimeField(null=True) class EntityB(models.Model): nameb = models.CharField(maxlength=128, null=True) dtb = models.DateTimeField(null=True) (namea="a") (namea="qwerty", ()) (nameb="random", ()) (nameb="b") qsa = (name=F("namea"), dt=F("dta")) qsb = (name=F("nameb"), dt=F("dtb")) queryset = (qsb) I can use a simple ORDER BY clause: queryset.orderby("-dt") And everything will work, no problem here. What I actually want is the same query, but with a NULLS LAST Usually the query becomes: queryset.orderby(F("dt").desc(nulls_last=True)) but that raises a DatabaseError: ORDER BY term does not match any column in the result set. I know unions can handle only a few clauses, but ORDER BY is one of them, so I'm unsure whether this is the expected behaviour or not. If it's expected, then the raised exception could be more explicit.


Question: resolved = expr.resolve_expression(self.query, allow_joins=True, reuse=None)             if self.query.combinator:                 src = resolved.get_source_expressions()[0]                expr_src = expr.get_source_expressions()[0]                 # Relabel order by columns to raw numbers if this is a combined                 # query; necessary since the columns can't be referenced by the                 # fully qualified name and the simple column names may collide.                 for idx, (sel_expr, _, col_alias) in enumerate(self.select):                     if is_ref and col_alias == src.refs:                         src = src.source                    elif col_alias:                    elif col_alias and not (                        isinstance(expr_src, F) and col_alias == expr_src.name                    ):                         continue                     if src == sel_expr:                         resolved.set_source_expressions([RawSQL('%d' % (idx + 1), ())])


Question: class SQLDeleteCompiler(SQLCompiler):     @cached_property     def single_alias(self):<ins>        # Ensure base table is in aliases.        self.query.get_initial_alias()</ins>         return sum(self.query.alias_refcount[t] > 0 for t in self.query.alias_map) == 1      def _as_sql(self, query):

().delete() subquery usage performance regression Description Lock tests are failing with Django-MySQL on Django 3.1:  . The tests run ().delete(). Django 3.0 generates this SQL: DELETE FROM Django 3.1 generates this SQL: DELETE FROM WHERE . IN (SELECT . FROM ) The subquery is a blocker for using LOCK TABLES along with delete() - as per the mysql docs: You cannot refer to a locked table multiple times in a single query using the same name. Use aliases instead, and obtain a separate lock for the table and each alias: mysqlLOCK TABLE t WRITE, t AS t1 READ; mysqlINSERT INTO t SELECT FROM t; ERROR 1100: Table 't' was not locked with LOCK TABLES mysqlINSERT INTO t SELECT FROM t AS t1; Since there's no alias on the subquery, there's no way to lock it. Additionally this change is a performance regression. I benchmarked with MariaDB 10.3 and filling an InnoDB a table of 100k rows via the [sequence storage engine ]. Using the old DELETE FROM, deletion takes 0.2 seconds: root [19]create table t(c int primary key); Query OK, 0 rows affected (0.079 sec) root [16]insert into t select from seq1to100000; Query OK, rows affected (0.252 sec) Records: Duplicates: 0 Warnings: 0 root [17]delete from t; Query OK, rows affected (0.200 sec) Using DELETE FROM WHERE id IN (...), deletion takes 7.5 seconds: root [18]drop table t; Query OK, 0 rows affected (0.008 sec) root [19]create table t(c int primary key); Query OK, 0 rows affected (0.079 sec) root [20]insert into t select from seq1to_100000; Query OK, rows affected (0.594 sec) Records: Duplicates: 0 Warnings: 0 root [21]delete from t where c in (select c from t); Query OK, rows affected (7.543 sec) root [22]drop table t; Query OK, 0 rows affected (0.013 sec) Yes in theory the optimizer should be able to find this, and it may even be fixed in later MySQL/MariaDB versions.


Question: errors.append(checks.Error(             "'django.contrib.sessions.middleware.SessionMiddleware' must "             "be in MIDDLEWARE in order to use the admin application.",<ins>            hint=(                "Insert "                "'django.contrib.sessions.middleware.SessionMiddleware' "                "before "                "'django.contrib.auth.middleware.AuthenticationMiddleware'."            ),</ins>             id='admin.E410',         ))     return errors

Add hint to the E410 about AuthenticationMiddleware. Description Given an empty MIDDLEWARE in your project, you try and run the admin. The system checks framework kicks in, giving you: (admin.E408) '' must be in MIDDLEWARE in order to use the admin application. (admin.E409) 'django.contrib.messages.middleware.MessageMiddleware' must be in MIDDLEWARE in order to use the admin application. (admin.E410) 'django.contrib.sessions.middleware.SessionMiddleware' must be in MIDDLEWARE in order to use the admin application. You add each of those middlewares to the stack, and run the application: File "", line 23, in processrequest ) % ("CLASSES" if settings.MIDDLEWARE is None else "") AssertionError: The Django authentication middleware requires session middleware to be installed. Edit your MIDDLEWARE setting to insert 'django.contrib.sessions.middleware.SessionMiddleware' before '' Ignoring the fact that it should be a system check, and it's an unfortunate side-effect of spanning 2 separate contrib apps that it isn't, if the order of the errors output was updated, we'd not see this exception in this specific scenario.


Question: @keep_lazy_text def slugify(value, allow_unicode=False):     """    Convert to ASCII if 'allow_unicode' is False. Convert spaces to hyphens.    Remove characters that aren't alphanumerics, underscores, or hyphens.    Convert to lowercase. Also strip leading and trailing whitespace.    Convert to ASCII if 'allow_unicode' is False. Convert spaces or repeated    dashes to single dashes. Remove characters that aren't alphanumerics,    underscores, or hyphens. Convert to lowercase. Also strip leading and    trailing whitespace, dashes, and underscores.     """     value = str(value)     if allow_unicode:         value = unicodedata.normalize('NFKC', value)     else:         value = unicodedata.normalize('NFKD', value).encode('ascii', 'ignore').decode('ascii')    value = re.sub(r'[^\w\s-]', '', value.lower()).strip()    return re.sub(r'[-\s]+', '-', value)    value = re.sub(r'[^\w\s-]', '', value.lower())    return re.sub(r'[-\s]+', '-', value).strip('-_')   def camel_case_to_spaces(value):

Make () strip dashes and underscores Description (last modified by Elinaldo do Nascimento Monteiro) Bug generation slug Example: from import text text.slugify("This is a test ---") output: this-is-a-test- Improvement after correction from import text text.slugify("_This is a test ---") output: this-is-a-test PR


Question: def get_source_expressions(self):         return [self.expression] <ins>    def get_group_by_cols(self, alias=None):        return self.expression.get_group_by_cols(alias=alias)</ins>     def as_sql(self, compiler, connection):         return self.expression.as_sql(compiler, connection)

Constant expressions of an ExpressionWrapper object are incorrectly placed at the GROUP BY clause Description I have a function that expects an arbitrary Query expression and constructs a query on a Postgres db def execQuery(expr): expr = ExpressionWrapper(expr, outputfield=IntegerField()) return Model.objects.annotate(exprres=expr).values('exprres', 'columna').annotate(sum=Sum('columnb')) However, when the given expr is a constant expression (e.g., Value(3)), Django generates an SQL query that contains this constant expression in its GROUP BY clause. SELECT "model"."columna", 3 AS "exprres", SUM("model"."columnb") AS "sum" FROM "model" GROUP BY "model"."columna", 3 This leads to an exception because in Postgres, the query above is invalid: aggregate functions are not allowed in GROUP BY LINE 1: SELECT "model"."columna", 3 AS "exprres", SUM("model"."col... Note that when the given query expression is not wrapped by the ExpressionWrapper object, Django correctly identifies and omits the constant from the GROUP BY clause. For example, the query below runs correctly. def execQuery(expr): return Model.objects.annotate(exprres=Value(3, outputfield=IntegerField())).values('exprres', 'columna').annotate(sum=Sum('columnb')) SELECT "model"."columna", 3 AS "exprres", SUM("model"."columnb") AS "sum" FROM "model" GROUP BY "model"."columna"


Question: if ord(char) < 33 or ord(char) == 127:             yield (                 'Cache key contains characters that will cause errors if '                'used with memcached: %r' % key, CacheKeyWarning                'used with memcached: %r' % key             )             break

Memcached key validation raises InvalidCacheKey with clunky message. Description On Django 2.2.13 the code for memcachekeywarnings in has a bad format string that results in raising an exception rather than just producing a warning. This can be reproduced with a memcached key with a space in it, e.g. "foo bar". This code was present before the 2.2.13 release, but becomes more exposed with that release, since it begins validating cache keys. I think it's as simple as removing the , CacheKeyWarning.


Question: return self.context.create_decimal_from_float(value)         try:             return decimal.Decimal(value)        except decimal.InvalidOperation:        except (decimal.InvalidOperation, TypeError, ValueError):             raise exceptions.ValidationError(                 self.error_messages['invalid'],                 code='invalid',

DecimalField.topython() raises TypeError on dict values. Description A call to DecimalField.topython() with a dictionary as the value parameter produces TypeError instead of ValidationError. This is a problem, for example, when you try to save a model object, and a decimal field got set to a dictionary by mistake. The TypeError exception that comes back makes it hard to track the problem to the field if the object has a lot of fields. I am proposing a patch to fix it:


Question: def check_filterable(self, expression):         """Raise an error if expression cannot be used in a WHERE clause."""        if not getattr(expression, 'filterable', True):        if (            hasattr(expression, 'resolve_expression') and            not getattr(expression, 'filterable', True)        ):             raise NotSupportedError(                 expression.__class__.__name__ + ' is disallowed in the filter '                 'clause.'

Description (last modified by Nicolas Baccelli) I'm migrating my app to django 3.0.7 and I hit a strange behavior using a model class with a field labeled filterable class ProductMetaDataType(models.Model): label = models.CharField(maxlength=255, unique=True, blank=False, null=False) filterable = models.BooleanField(default=False, verbosename=("filterable")) class Meta: applabel = "adminpricing" verbosename = ("product meta data type") verbosenameplural = ("product meta data types") def str(self): return class ProductMetaData(models.Model): id = models.BigAutoField(primarykey=True) product = models.ForeignKey( Produit, null=False, blank=False, ondelete=models.CASCADE ) value = models.TextField(null=False, blank=False) marketplace = models.ForeignKey( Plateforme, null=False, blank=False, ondelete=models.CASCADE ) datecreated = models.DateTimeField(null=True, ) metadatatype = models.ForeignKey( ProductMetaDataType, null=False, blank=False, ondelete=models.CASCADE ) class Meta: applabel = "adminpricing" verbosename = ("product meta data") verbosenameplural = ("product meta datas") Error happened when filtering ProductMetaData with a metadatatype : (value="Dark Vador", metadatatype=self.brandmetadata) Error traceback : Traceback (most recent call last): File "", line 481, in testcheckpolicywarningbyfields for p in ( File "/usr/local/lib/python3.8/site-", line 82, in managermethod return getattr(self.getqueryset(), name)(args, kwargs) File "/usr/local/lib/python3.8/site-", line 904, in filter return self.filterorexclude(False, args, kwargs) File "/usr/local/lib/python3.8/site-", line 923, in filterorexclude (Q(args, *kwargs)) File "/usr/local/lib/python3.8/site-", line 1351, in addq clause, = (qobject, self.usedaliases) File "/usr/local/lib/python3.8/site-", line 1378, in addq childclause, neededinner = self.buildfilter( File "/usr/local/lib/python3.8/site-", line 1264, in buildfilter self.checkfilterable(value) File "/usr/local/lib/python3.8/site-", line 1131, in checkfilterable raise NotSupportedError( ProductMetaDataType is disallowed in the filter clause.


Question: )          if self.rhs_is_direct_value():            # Remove None from the list as NULL is never equal to anything.             try:                 rhs = OrderedSet(self.rhs)                rhs.discard(None)             except TypeError:  # Unhashable items in self.rhs                rhs = self.rhs                rhs = [r for r in self.rhs if r is not None]              if not rhs:                 raise EmptyResultSet

Avoid passing NULL to the IN lookup Description Currently prefetchrelated on a FK passes the NULL through to the database for e.g. authorid IN (NULL, 2). Passing NULL is always unnecessary, since it's not allowed in FK's. There's a small risk from passing NULL that it could lead to incorrect with complex prefetch querysets using PK refs because of NULL's weirdness in SQL. For example with these models: from import models class Author(models.Model): pass class Book(models.Model): author = models.ForeignKey(Author, null=True, ondelete=models.DONOTHING) Prefetching authors on Books, when at least one Book has author=None, uses IN (..., NULL, ...) in the query: In [1]: from import Author, Book In [2]: a1 = () In [3]: (author=a1) Out[3]: <Book: Book object (3)In [4]: (author=None) Out[4]: <Book: Book object (4)In [5]: Book.objects.prefetchrelated('author') Out[5]: <QuerySet [<Book: Book object (3), <Book: Book object (4)]In [6]: from import connection In [7]: connection.queries Out[7]: [{'sql': 'INSERT INTO "coreauthor" ("id") VALUES (NULL)', 'time': '0.001'}, {'sql': 'INSERT INTO "corebook" ("authorid") VALUES (2)', 'time': '0.001'}, {'sql': 'INSERT INTO "corebook" ("authorid") VALUES (NULL)', 'time': '0.001'}, {'sql': 'SELECT "corebook"."id", "corebook"."authorid" FROM "corebook" LIMIT 21', 'time': '0.000'}, {'sql': 'SELECT "coreauthor"."id" FROM "coreauthor" WHERE "coreauthor"."id" IN (NULL, 2)', 'time': '0.000'}] Maybe this could generally be extended to use of _in with non-nullable fields?


Question: except OperationalError as error:                     warnings.warn(                         "Got an error checking a consistent migration history "                        "performed for database connection '%s': %s."                        "performed for database connection '%s': %s"                         % (alias, error),                         RuntimeWarning,                     )

Improve makemigrations warning message when calling without an active database connection. Description I was looking at the gis install instructions and I came across an error when running makemigrations. (Error is I had not entered password correctly). However, the error message that is generated is a bit odd, it has a full stop on a new line and shows ( I was also able to get the same error message on a clean project, with a postgres database (not gis) and an incorrect password. I'm not sure if this is a 'bug' but it doesn't look quite right? (gis) PS C:\Users\smith\gis\geodjangopython .\ makemigrations c:\users\smith\pycharmprojects\django2\django\core\management\commands\ RuntimeWarning: Got an error checking a consistent migration history performed for database connection 'default': fe_sendauth: no password supplied . ( No changes detected (gis) PS C:\Users\smith\gis\geodjango


Question: # If we get to this point and the field is a relation to another model,         # append the default ordering for that model unless it is the pk         # shortcut or the attribute name of the field that is specified.        if field.is_relation and opts.ordering and getattr(field, 'attname', None) != name and name != 'pk':        if (            field.is_relation and            opts.ordering and            getattr(field, 'attname', None) != pieces[-1] and            name != 'pk'        ):             # Firstly, avoid infinite loops.             already_seen = already_seen or set()             join_tuple = tuple(getattr(self.query.alias_map[j], 'join_cols', None) for j in joins)

Self referencing foreign key doesn't correctly order by a relation "id" field. Description Initially discovered on 2.2.10 but verified still happens on 3.0.6. Given the following models: class OneModel(models.Model): class Meta: ordering = ("-id",) id = models.BigAutoField(primarykey=True) root = models.ForeignKey("OneModel", ondelete=models.CASCADE, null=True) oneval = models.BigIntegerField(null=True) class TwoModel(models.Model): id = models.BigAutoField(primarykey=True) record = models.ForeignKey(OneModel, ondelete=models.CASCADE) twoval = models.BigIntegerField(null=True) The following queryset gives unexpected results and appears to be an incorrect SQL query: qs = (recordonevalin=[1,2,3]) qs = qs.orderby("recordrootid") print() SELECT "oriontwomodel"."id", "oriontwomodel"."recordid", "oriontwomodel"."twoval" FROM "oriontwomodel" INNER JOIN "oriononemodel" ON ("oriontwomodel"."recordid" = "oriononemodel"."id") LEFT OUTER JOIN "oriononemodel" T3 ON ("oriononemodel"."rootid" = T3."id") WHERE "oriononemodel"."oneval" IN (1, 2, 3) ORDER BY T3."id" DESC The query has an unexpected DESCENDING sort. That appears to come from the default sort order on the OneModel class, but I would expect the orderby() to take prececence. The the query has two JOINS, which is unnecessary. It appears that, since is a foreign key to itself, that is causing it to do the unnecessary extra join. In fact, testing a model where root is a foreign key to a third model doesn't show the problem behavior. Note also that the queryset with orderby("recordroot") gives the exact same SQL. This queryset gives correct results and what looks like a pretty optimal SQL: qs = (recordonevalin=[1,2,3]) qs = qs.orderby("recordrootid") print() SELECT "oriontwomodel"."id", "oriontwomodel"."recordid", "oriontwomodel".


Question: # If we get to this point and the field is a relation to another model,         # append the default ordering for that model unless it is the pk         # shortcut or the attribute name of the field that is specified.        if field.is_relation and opts.ordering and getattr(field, 'attname', None) != name and name != 'pk':        if (            field.is_relation and            opts.ordering and            getattr(field, 'attname', None) != pieces[-1] and            name != 'pk'        ):             # Firstly, avoid infinite loops.             already_seen = already_seen or set()             join_tuple = tuple(getattr(self.query.alias_map[j], 'join_cols', None) for j in joins)

"twoval" FROM "oriontwomodel" INNER JOIN "oriononemodel" ON ("oriontwomodel"."recordid" = "oriononemodel"."id") WHERE "oriononemodel"."oneval" IN (1, 2, 3) ORDER BY "oriononemodel"."rootid" ASC So is this a potential bug or a misunderstanding on my part? Another queryset that works around the issue and gives a reasonable SQL query and expected results: qs = (recordonevalin=[1,2,3]) qs = qs.annotate(rootid=F("recordrootid")) qs = qs.orderby("rootid") print() SELECT "oriontwomodel"."id", "oriontwomodel"."recordid", "oriontwomodel"."twoval" FROM "oriontwomodel" INNER JOIN "oriononemodel" ON ("oriontwomodel"."recordid" = "oriononemodel"."id") WHERE "oriononemodel"."oneval" IN (1, 2, 3) ORDER BY "oriononemodel"."zeroid" ASC ASCENDING sort, and a single INNER JOIN, as I'd expect. That actually works for my use because I need that output column anyway. One final oddity; with the original queryset but the inverted sort orderby(): qs = (recordonevalin=[1,2,3]) qs = qs.orderby("-recordrootid") print() SELECT "oriontwomodel"."id", "oriontwomodel"."recordid", "oriontwomodel"."twoval" FROM "oriontwomodel" INNER JOIN "oriononemodel" ON ("oriontwomodel"."recordid" = "oriononemodel"."id") LEFT OUTER JOIN "oriononemodel" T3 ON ("oriononemodel"."rootid" = T3."id") WHERE "oriononemodel"."oneval" IN (1, 2, 3) ORDER BY T3."id" ASC One gets the query with the two JOINs but an ASCENDING sort order. I was not under the impression that sort orders are somehow relative to the class level sort order, eg: does specifing orderby("-record_rootid") invert the class sort order?


Question: # If we get to this point and the field is a relation to another model,         # append the default ordering for that model unless it is the pk         # shortcut or the attribute name of the field that is specified.        if field.is_relation and opts.ordering and getattr(field, 'attname', None) != name and name != 'pk':        if (            field.is_relation and            opts.ordering and            getattr(field, 'attname', None) != pieces[-1] and            name != 'pk'        ):             # Firstly, avoid infinite loops.             already_seen = already_seen or set()             join_tuple = tuple(getattr(self.query.alias_map[j], 'join_cols', None) for j in joins)

Testing that on a simple case doesn't show that behavior at all. Thanks for any assistance and clarification.


Question: library = 'argon2'      time_cost = 2    memory_cost = 512    parallelism = 2    memory_cost = 102400    parallelism = 8      def encode(self, password, salt):         argon2 = self._load_library()        params = self.params()         data = argon2.low_level.hash_secret(             password.encode(),             salt.encode(),            time_cost=self.time_cost,            memory_cost=self.memory_cost,            parallelism=self.parallelism,            hash_len=argon2.DEFAULT_HASH_LENGTH,            type=argon2.low_level.Type.I,            time_cost=params.time_cost,            memory_cost=params.memory_cost,            parallelism=params.parallelism,            hash_len=params.hash_len,            type=params.type,         )         return self.algorithm + data.decode('ascii')

Argon2id should be supported and become the default variety for Argon2PasswordHasher Description (last modified by Si Feng) There were three important changes in the upstream argon2-cffi library since Django 1.10 was released with Argon2 support: (Nov 10, 2016) argon2id support was :  (Apr 9, 2018) argon2id became its default type:  (Aug 18, 2018) its hasher's default memory cost changed from 512 to , and parallelism from 2 to 8, per RFC draft recommendations:  When Django 1.10 was released, only argon2d and argon2i were available, hence the hard-coded argon2i variety in Argon2PasswordHasher. Though Django does not use its hasher, the memorycost = 512 and parallelism = 2 in Argon2PasswordHasher were simply copied from argon2-cffi's hasher back then. Now we should sync Django with upstream updates.


Question: algorithm, rest = encoded.split('$', 1)         assert algorithm == self.algorithm         try:            return argon2.low_level.verify_secret(                ('$' + rest).encode('ascii'),                password.encode(),                type=argon2.low_level.Type.I,            )            return argon2.PasswordHasher().verify('$' + rest, password)         except argon2.exceptions.VerificationError:             return False


Question: }      def must_update(self, encoded):        (algorithm, variety, version, time_cost, memory_cost, parallelism,            salt, data) = self._decode(encoded)        algorithm, rest = encoded.split('$', 1)         assert algorithm == self.algorithm         argon2 = self._load_library()        return (            argon2.low_level.ARGON2_VERSION != version or            self.time_cost != time_cost or            self.memory_cost != memory_cost or            self.parallelism != parallelism        )        current_params = argon2.extract_parameters('$' + rest)        new_params = self.params()        # Set salt_len to the salt_len of the current parameters because salt        # is explicitly passed to argon2.        new_params.salt_len = current_params.salt_len        return current_params != new_params      def harden_runtime(self, password, encoded):         # The runtime for Argon2 is too complicated to implement a sensible         # hardening algorithm.         pass     def params(self):        argon2 = self._load_library()        # salt_len is a noop, because we provide our own salt.        return argon2.Parameters(            type=argon2.low_level.Type.ID,            version=argon2.low_level.ARGON2_VERSION,            salt_len=argon2.DEFAULT_RANDOM_SALT_LENGTH,            hash_len=argon2.DEFAULT_HASH_LENGTH,            time_cost=self.time_cost,            memory_cost=self.memory_cost,            parallelism=self.parallelism,        )     def _decode(self, encoded):         """         Split an encoded hash and return: (


Question: parallelism, salt, data,         ).         """        bits = encoded.split('$')        if len(bits) == 5:            # Argon2 < 1.3            algorithm, variety, raw_params, salt, data = bits            version = 0x10        else:            assert len(bits) == 6            algorithm, variety, raw_version, raw_params, salt, data = bits            assert raw_version.startswith('v=')            version = int(raw_version[len('v='):])        params = dict(bit.split('=', 1) for bit in raw_params.split(','))        assert len(params) == 3 and all(x in params for x in ('t', 'm', 'p'))        time_cost = int(params['t'])        memory_cost = int(params['m'])        parallelism = int(params['p'])        argon2 = self._load_library()        algorithm, rest = encoded.split('$', 1)        params = argon2.extract_parameters('$' + rest)        variety, *_, salt, data = rest.split('$')         return (            algorithm, variety, version, time_cost, memory_cost, parallelism,            salt, data,            algorithm, variety, params.version, params.time_cost,            params.memory_cost, params.parallelism, salt, data,         )


Question: on reloading.     """     import django.__main__    django_main_path = Path(django.__main__.__file__)    py_script = Path(sys.argv[0])      args = [sys.executable] + ['-W%s' % o for o in sys.warnoptions]    if sys.argv[0] == django.__main__.__file__:    if py_script == django_main_path:         # The server was started with `python -m django runserver`.         args += ['-m', 'django']         args += sys.argv[1:]    elif not py_script.exists():        # sys.argv[0] may not exist for several reasons on Windows.        # It may exist with a .exe extension or have a -script.py suffix.        exe_entrypoint = py_script.with_suffix('.exe')        if exe_entrypoint.exists():            # Should be executed directly, ignoring sys.executable.            return [exe_entrypoint, *sys.argv[1:]]        script_entrypoint = py_script.with_name('%s-script.py' % py_script.name)        if script_entrypoint.exists():            # Should be executed as usual.            return [*args, script_entrypoint, *sys.argv[1:]]        raise RuntimeError('Script %s does not exist.' % py_script)     else:         args += sys.argv     return args

django-admin runserver mostly does not work on Windows Description I frequently encounter the problem that running "django-admin runserver" on Windows fails: <some-venv\scripts\ can't open file '<some-venv\test\Scripts\django-admin': [Errno 2] No such file or directory The command works if run with --noreload. There are probably other conditions that must be met for it to fail, or work. I think it makes a difference, for example, if Django was installed with pip or setuptools. The .exe "console scripts " are different between the two; one type has a bit of readable Python at the end, the other does not. What I do know is that the problem is either caused or enabled by the generated console scripts (the .exe wrappers in venv/Scripts) stripping the .exe extension from their own [0] when passing that to Python to run. If I sabotage this by changing the regex in the file itself (where present), or by adding the extension again in before running the command line, it works fine. To be honest, I have no idea whether this is a bug in Django, or pip, or setuptools, or distutils, or something else entirely. I have spent the last two days trying to figure out what exactly goes wrong, and where, with little success. I'm reporting it here because Django is the only place I could find where it is possible to kludge the code into working. I'll be happy to take my business elsewhere if someone can point the way.


Question: import concurrent.futures import glob import os<ins>from pathlib import Path</ins>  from django.core.management.base import BaseCommand, CommandError from django.core.management.utils import (

compilemessages needlessly runs msgfmt on unchanged .po files Description I have a project where running django-admin compilemessages takes 1.75 seconds. Running it again, when all the .mo files already exists and are up-to-date, also takes 1.75 seconds. I propose that is changed so that it only invokes msgfmt when it would do anything useful. This can be implemented by checking the mtime of the .po file and the corresponding .mo file. (If statting the .mo file fails, treat that as if the mtime was 0.) Only submit the command to the executor if the mtime of the .po file is greater than that of the .mo file. In effect: don't do anything if the .mo file is newer than the .po file. There is one issue with this: the way the code currently uses the iswritable function. Since it modifies the mtime of the .mo file, you would have to perform the stat of the .mo file before you check if it is writable. (Or, you could just remove the iswritable function and its use. That feature is, in my opinion, of dubious value, and it doesn't appear to be documented.) After I made the changes above, the runtime in the common case where nothing needs to be done was reduced from 1.75 seconds to 0.2 seconds. (Unfortunately, I doubt that I will be able to get a Corporate Contributor License Agreement signed, so I can unfortunately not contribute my change.) 75 seconds may not be much, but when a CI system does it repeatedly, it adds up.


Question: def has_bom(fn):    with open(fn, 'rb') as f:    with fn.open('rb') as f:         sample = f.read(4)     return sample.startswith((codecs.BOM_UTF8, codecs.BOM_UTF16_LE, codecs.BOM_UTF16_BE))


Question: with concurrent.futures.ThreadPoolExecutor() as executor:             futures = []             for i, (dirpath, f) in enumerate(locations):                po_path = Path(dirpath) / f                mo_path = po_path.with_suffix('.mo')                try:                    if mo_path.stat().st_mtime >= po_path.stat().st_mtime:                        if self.verbosity > 0:                            self.stdout.write(                                'File “%s” is already compiled and up to date.'                                % po_path                            )                        continue                except FileNotFoundError:                    pass                 if self.verbosity > 0:                     self.stdout.write('processing file %s in %s' % (f, dirpath))                po_path = os.path.join(dirpath, f)                 if has_bom(po_path):                     self.stderr.write(                         'The %s file has a BOM (Byte Order Mark). Django only '


Question: )                     self.has_errors = True                     continue                base_path = os.path.splitext(po_path)[0]                  # Check writability on first location                if i == 0 and not is_writable(base_path + '.mo'):                if i == 0 and not is_writable(mo_path):                     self.stderr.write(                         'The po files under %s are in a seemingly not writable location. '                         'mo files will not be updated/created.' % dirpath


Question: self.has_errors = True                     return                 args = [self.program] + self.program_options + [                    '-o', base_path + '.mo', base_path + '.po'                ]                args = [self.program, *self.program_options, '-o', mo_path, po_path]                 futures.append(executor.submit(popen_wrapper, args))              for future in concurrent.futures.as_completed(futures):


Question: cursor.execute(                     connection.ops.cache_key_culling_sql() % table,                     [cull_num])                cursor.execute("DELETE FROM %s "                               "WHERE cache_key < %%s" % table,                               [cursor.fetchone()[0]])                last_cache_key = cursor.fetchone()                if last_cache_key:                    cursor.execute(                        'DELETE FROM %s WHERE cache_key < %%s' % table,                        [last_cache_key[0]],                    )      def clear(self):         db = router.db_for_write(self.cache_model_class)

sometimes fails with 'NoneType' object is not subscriptable Description (last modified by Guillermo Bonvehí) I'm sporadically getting some cache errors using database backend. The error is: 'NoneType' object is not subscriptable And the backtrace: /usr/local/lib/python3.7/site-→ getresponse /usr/local/lib/python3.7/site-→ render /usr/local/lib/python3.7/site-→ callback /usr/local/lib/python3.7/site-→ processresponse /usr/local/lib/python3.7/site-→ learncachekey /usr/local/lib/python3.7/site-→ set /usr/local/lib/python3.7/site-→ baseset /usr/local/lib/python3.7/site-→ cull This is using Django 2.2.11 but I see the same code is in master.  cursor.execute( () % table, [cullnum]) cursor.execute("DELETE FROM %s " "WHERE cachekey < %%s" % table, [cursor.fetchone()[0]]) From what I can understand, the cursor after running () command is not returning any data, so cursor.fetchone()[0] afterwards fails. I guess a simple check to see if it contains data would be enough, may apply for an easy picking. Edit: Wording


Question: def add_fields(self, form, index):         """A hook for adding extra fields on to each form instance."""        initial_form_count = self.initial_form_count()         if self.can_order:             # Only pre-fill the ordering field for initial forms.            if index is not None and index < self.initial_form_count():            if index is not None and index < initial_form_count:                 form.fields[ORDERING_FIELD_NAME] = IntegerField(                     label=_('Order'),                     initial=index + 1,

formsets with candelete=True shouldn't add delete field to extra forms Description Current behavior of formsets with candelete=True is to add a delete field to every form. This behavior differs from that expected, however (why would one want a delete option on an "add" form?), as well as that of the builtin admin. I've included a patch on , but haven't bothered with patching tests yet.


Question: required=False,                     widget=self.get_ordering_widget(),                 )        if self.can_delete:        if self.can_delete and (self.can_delete_extra or index < initial_form_count):             form.fields[DELETION_FIELD_NAME] = BooleanField(label=_('Delete'), required=False)      def add_prefix(self, index):


Question: def formset_factory(form, formset=BaseFormSet, extra=1, can_order=False,                     can_delete=False, max_num=None, validate_max=False,                    min_num=None, validate_min=False, absolute_max=None):                    min_num=None, validate_min=False, absolute_max=None,                    can_delete_extra=True):     """Return a FormSet for the given form class."""     if min_num is None:         min_num = DEFAULT_MIN_NUM


Question: 'extra': extra,         'can_order': can_order,         'can_delete': can_delete,        'can_delete_extra': can_delete_extra,         'min_num': min_num,         'max_num': max_num,         'absolute_max': absolute_max,diff --git a/django/forms/models.py b/django/forms/models.py-- a/django/forms/models.py++ b/django/forms/models.py


Question: widgets=None, validate_max=False, localized_fields=None,                          labels=None, help_texts=None, error_messages=None,                          min_num=None, validate_min=False, field_classes=None,                         absolute_max=None):                         absolute_max=None, can_delete_extra=True):     """Return a FormSet class for the given Django model class."""     meta = getattr(form, 'Meta', None)     if (getattr(meta, 'fields', fields) is None and


Question: FormSet = formset_factory(form, formset, extra=extra, min_num=min_num, max_num=max_num,                               can_order=can_order, can_delete=can_delete,                               validate_min=validate_min, validate_max=validate_max,                              absolute_max=absolute_max)                              absolute_max=absolute_max, can_delete_extra=can_delete_extra)     FormSet.model = model     return FormSet


Question: widgets=None, validate_max=False, localized_fields=None,                           labels=None, help_texts=None, error_messages=None,                           min_num=None, validate_min=False, field_classes=None,                          absolute_max=None):                          absolute_max=None, can_delete_extra=True):     """     Return an ``InlineFormSet`` for the given kwargs.


Question: 'error_messages': error_messages,         'field_classes': field_classes,         'absolute_max': absolute_max,<ins>        'can_delete_extra': can_delete_extra,</ins>     }     FormSet = modelformset_factory(model, **kwargs)     FormSet.fk = fk


Question: return          using = router.db_for_read(self.remote_field.model, instance=model_instance)        qs = self.remote_field.model._default_manager.using(using).filter(        qs = self.remote_field.model._base_manager.using(using).filter(             **{self.remote_field.field_name: value}         )         qs = qs.complex_filter(self.get_limit_choices_to())

ForeignKey.validate() should validate using the base manager. Description ForeignKey.validate() should validate using the base manager instead of the default manager. Consider the models: class ArticleManager(models.Manage): def getqueryset(self): qs = super().getqueryset() return (archived=False) class Article(models.Model): title = models.CharField(maxlength=100) archived = models.BooleanField(default=False) # Don't include archived articles by default. objects = ArticleManager() class FavoriteAricles(models.Model): article = models.ForeignKey(Article, ondelete=models.CASCADE) In the example, now consider a form that allows users to pick a favorite article including archived articles. class FavoriteAriclesForm(forms.ModelForm): class Meta: model = FavoriteArticle fields = 'all' def init(self, args, kwargs): super().init(args, kwargs) # Use the base manager instead of the default manager to allow archived articles. ['article'].queryset = () The above form will never validate as True when a user selects an archived article. This is because the ForeignKey validation always uses defaultmanager instead of basemanager. The user facing error message is "article instance with id 123 does not exist." (quite confusing to typical users). The code for this validation is here:  The FavoriteAriclesForm is specifically designed to use a different manager, but the ForeignKey validation makes this difficult. In this example scenario, it is not acceptable to change the model's default manager as the default should avoid archived articles in other typical scenarios. Suggested solution: the ForeignKey validation should use the basemanager instead which does not include the default filters.


Question: The first day according to the week format is 0 and the last day is 6.         """         week_format = self.get_week_format()        if week_format == '%W':                 # week starts on Monday        if week_format in {'%W', '%V'}:         # week starts on Monday             return date.weekday()         elif week_format == '%U':               # week starts on Sunday             return (date.weekday() + 1) % 7

Support "%V" format in WeekArchiveView. Description (Docs for WeekArchiveView are misleading about %W) - closed 4 years ago mentioned support for %V week format. Since python 3.6, %G, %u and %V ISO 8601 formatters were to strptime. WeekArchiveView should add %V to the list of accepted week formatters. This would require as well the special case to change the year format to %G, or simply ValueError in datefrom_string should mention the message passed from datetime.datetime.strptime: ISO week directive '%V' is incompatible with the year directive '%Y'. Use the ISO year '%G'.


Question: date_field = self.get_date_field()         week_format = self.get_week_format()        week_choices = {'%W': '1', '%U': '0'}        week_choices = {'%W': '1', '%U': '0', '%V': '1'}         try:             week_start = week_choices[week_format]         except KeyError:


Question: week_format,                 ', '.join(sorted(week_choices)),             ))        date = _date_from_string(year, self.get_year_format(),                                 week_start, '%w',                                 week, week_format)        year_format = self.get_year_format()        if week_format == '%V' and year_format != '%G':            raise ValueError(                "ISO week directive '%s' is incompatible with the year "                "directive '%s'. Use the ISO year '%%G' instead." % (                    week_format, year_format,                )            )        date = _date_from_string(year, year_format, week_start, '%w', week, week_format)         since = self._make_date_lookup_arg(date)         until = self._make_date_lookup_arg(self._get_next_week(date))         lookup_kwargs = {


Question: if self.remote_field.parent_link:             kwargs['parent_link'] = self.remote_field.parent_link         if isinstance(self.remote_field.model, str):            kwargs['to'] = self.remote_field.model.lower()            if '.' in self.remote_field.model:                app_label, model_name = self.remote_field.model.split('.')                kwargs['to'] = '%s.%s' % (app_label, model_name.lower())            else:                kwargs['to'] = self.remote_field.model.lower()         else:             kwargs['to'] = self.remote_field.model._meta.label_lower         # If swappable is True, then see if we're actually pointing to the target

makemigrations crashes for ForeignKey with mixed-case app name. Description When i run "python3 migrate" on Django 3.1b1 shows me that error (Please, note that the code works well in 3.0) ValueError: The field DJRegLogin.Content.category was declared with a lazy reference to 'djreglogin.category', but app 'djreglogin' isn't installed. (Conflict Part) class Category(models.Model): title = models.CharField(maxlength=100, dbindex=True) slug = models.SlugField(maxlength=100, dbindex=True) class Meta: verbosename = 'Category' verbosenameplural = 'Categories' def str(self): return def getabsoluteurl(self): return reverse('viewblogcategory', None, kwargs={'slug': }) class Content(models.Model): title = models.CharField(maxlength=100, unique=True) slug = models.SlugField(maxlength=100, unique=True) body = RichTextField(configname='default') posted = models.DateTimeField(dbindex=True, autonowadd=True) sites = models.ManyToManyField(Site) ip = models.GenericIPAddressField(editable=False) category = models.ForeignKey(Category, ondelete=models.CASCADE) user = models.ForeignKey(User, ondelete=models.CASCADE, null=False, blank=False, editable=False) status = models.CharField(maxlength=10, choices=STATUSCHOICES, default='draft') def str(self): return def getabsoluteurl(self): return reverse('viewblogpost', None, kwargs={'slug': }) (Related to issue part) INSTALLEDAPPS = [ 'DJRegLogin', '', '', 'django.contrib.contenttypes', 'django.contrib.sessions', 'django.contrib.messages', 'django.contrib.staticfiles', 'socialdjango', 'ckeditor', '', 'django.contrib.flatpages', 'django.contrib.sitemaps', ] from import AppConfig class DJRegLoginConfig(AppConfig): name = 'DJRegLogin' verbosename = "Contents"


Question: raise ValueError(                 'UniqueConstraint with conditions cannot be deferred.'             )<ins>        if include and deferrable:            raise ValueError(                'UniqueConstraint with include fields cannot be deferred.'            )        if opclasses and deferrable:            raise ValueError(                'UniqueConstraint with opclasses cannot be deferred.'            )</ins>         if not isinstance(deferrable, (type(None), Deferrable)):             raise ValueError(                 'UniqueConstraint.deferrable must be a Deferrable instance.'

Add errors when UniqueConstraint.deferrable is combined with index specific parameters Description UniqueConstraint should throw an error when deferrable is combined with parameters which require the usage of an explicit unique index through CREATE UNIQUE INDEX. This was missed when UniqueConstraint.include and UniqueConstraint.opclasses was . We should also add documentation about this incompatibility.


Question: require_outer = lookup_type == 'isnull' and condition.rhs is True and not current_negated         if current_negated and (lookup_type != 'isnull' or condition.rhs is False) and condition.rhs is not None:             require_outer = True            if (lookup_type != 'isnull' and (                    self.is_nullable(targets[0]) or                    self.alias_map[join_list[-1]].join_type == LOUTER)):            if lookup_type != 'isnull':                 # The condition added here will be SQL like this:                 # NOT (col IS NOT NULL), where the first NOT is added in                 # upper layers of code. The reason for addition is that if col

SQL generated for negated F() expressions is incorrect Description Consider the following model definition. from import models class Rectangle(models.Model): length = models.IntegerField(null=True) width = models.IntegerField(null=True) We make the following queries: Q1) Rectangles that are squares. Q2a) Rectangles that are not squares (length != width). Q2b) Rectangles that are not squares (width != length). Queries Q2a and Q2b semantically mean the same. However, the ORM generates different SQL for these two queries. import django () from import F from import Rectangle print '--- Q1: Get the rectangles that are squares' print (length=F('width')).values('pk').query print '--- Q2a: Get the rectangles that are not squares' print Rectangle.objects.exclude(length=F('width')).values('pk').query print '--- Q2b: Get the rectangles that are not squares' print Rectangle.objects.exclude(width=F('length')).values('pk').query The generated SQL is --- Q1: Get the rectangles that are squares SELECT "testapprectangle"."id" FROM "testapprectangle" WHERE "testapprectangle"."length" = ("testapprectangle"."width") --- Q2a: Get the rectangles that are not squares SELECT "testapprectangle"."id" FROM "testapprectangle" WHERE NOT ("testapprectangle"."length" = ("testapprectangle"."width") AND "testapprectangle"."length" IS NOT NULL) --- Q2b: Get the rectangles that are not squares SELECT "testapprectangle"."id" FROM "testapprectangle" WHERE NOT ("testapprectangle"."width" = ("testapprectangle"."length") AND "testapprectangle"."width" IS NOT NULL) Note the asymmetry between Q2a and Q2b. These queries will return different results. Reddit user /u/charettes set up this useful SQL fiddle with the above mentioned schema and test values: !12//4 Here's my reddit post on this issue:


Question: # (col IS NULL OR col != someval)                 #   <=>                 # NOT (col IS NOT NULL AND col = someval).                lookup_class = targets[0].get_lookup('isnull')                col = self._get_col(targets[0], join_info.targets[0], alias)                clause.add(lookup_class(col, False), AND)                if (                    self.is_nullable(targets[0]) or                    self.alias_map[join_list[-1]].join_type == LOUTER                ):                    lookup_class = targets[0].get_lookup('isnull')                    col = self._get_col(targets[0], join_info.targets[0], alias)                    clause.add(lookup_class(col, False), AND)                # If someval is a nullable column, someval IS NOT NULL is                # added.                if isinstance(value, Col) and self.is_nullable(value.target):                    lookup_class = value.target.get_lookup('isnull')                    clause.add(lookup_class(value, False), AND)         return clause, used_joins if not require_outer else ()      def add_filter(self, filter_clause):


Question: """         raise NotImplementedError('subclasses of BaseDatabaseOperations may require a date_extract_sql() method')     def date_interval_sql(self, timedelta):        """        Implement the date interval functionality for expressions.        """        raise NotImplementedError('subclasses of BaseDatabaseOperations may require a date_interval_sql() method')     def date_trunc_sql(self, lookup_type, field_name):         """         Given a lookup_type of 'year', 'month', or 'day', return the SQL thatdiff --git a/django/db/backends/mysql/operations.py b/django/db/backends/mysql/operations.py-- a/django/db/backends/mysql/operations.py++ b/django/db/backends/mysql/operations.py

durations-only expressions doesn't work on SQLite and MySQL Description class Experiment(models.Model): estimatedtime = models.DurationField() list(Experiment.objects.annotate(duration=F('estimatedtime') + datime.timedelta(1))) Traceback (most recent call last): File "", line 1218, in testdurationexpressions list(Experiment.objects.annotate(duration=F('estimatedtime') + delta)) File "", line 269, in iter self.fetchall() File "", line 1172, in fetchall self.resultcache = list(self.iterableclass(self)) File "", line 63, in iter for row in compiler.resultsiter(results): File "", line 998, in applyconverters value = converter(value, expression, connection) File "", line 571, in convertdurationfield_value value = str(decimal.Decimal(value) / decimal.Decimal()) decimal.InvalidOperation: [<class 'decimal.ConversionSyntax']


Question: from django.conf import settings from django.db.backends.base.operations import BaseDatabaseOperations from django.utils import timezone<del>from django.utils.duration import duration_microseconds</del> from django.utils.encoding import force_str


Question: else:             return "TIME(%s)" % (field_name)     def date_interval_sql(self, timedelta):        return 'INTERVAL %s MICROSECOND' % duration_microseconds(timedelta)     def fetch_returned_insert_rows(self, cursor):         """         Given a cursor object that has just performed an INSERT...RETURNINGdiff --git a/django/db/backends/sqlite3/operations.py b/django/db/backends/sqlite3/operations.py-- a/django/db/backends/sqlite3/operations.py++ b/django/db/backends/sqlite3/operations.py


Question: from django.db.models.expressions import Col from django.utils import timezone from django.utils.dateparse import parse_date, parse_datetime, parse_time<del>from django.utils.duration import duration_microseconds</del> from django.utils.functional import cached_property


Question: """         return "django_date_extract('%s', %s)" % (lookup_type.lower(), field_name)     def date_interval_sql(self, timedelta):        return str(duration_microseconds(timedelta))     def format_for_duration_arithmetic(self, sql):         """Do nothing since formatting is handled in the custom function."""         return sqldiff --git a/django/db/models/expressions.py b/django/db/models/expressions.py-- a/django/db/models/expressions.py++ b/django/db/models/expressions.py


Question: def _combine(self, other, connector, reversed):         if not hasattr(other, 'resolve_expression'):             # everything must be resolvable to an expression            if isinstance(other, datetime.timedelta):                other = DurationValue(other, output_field=fields.DurationField())            else:                other = Value(other)            output_field = (                fields.DurationField()                if isinstance(other, datetime.timedelta) else                None            )            other = Value(other, output_field=output_field)          if reversed:             return CombinedExpression(other, connector, self)


Question: def as_sql(self, compiler, connection):         try:            lhs_output = self.lhs.output_field            lhs_type = self.lhs.output_field.get_internal_type()         except FieldError:            lhs_output = None            lhs_type = None         try:            rhs_output = self.rhs.output_field            rhs_type = self.rhs.output_field.get_internal_type()         except FieldError:            rhs_output = None        if (not connection.features.has_native_duration_field and                ((lhs_output and lhs_output.get_internal_type() == 'DurationField') or                 (rhs_output and rhs_output.get_internal_type() == 'DurationField'))):            rhs_type = None        if (            not connection.features.has_native_duration_field and            'DurationField' in {lhs_type, rhs_type} and            lhs_type != rhs_type        ):             return DurationExpression(self.lhs, self.connector, self.rhs).as_sql(compiler, connection)        if (lhs_output and rhs_output and self.connector == self.SUB and            lhs_output.get_internal_type() in {'DateField', 'DateTimeField', 'TimeField'} and                lhs_output.get_internal_type() == rhs_output.get_internal_type()):        datetime_fields = {'DateField', 'DateTimeField', 'TimeField'}        if self.connector == self.SUB and lhs_type in datetime_fields and lhs_type == rhs_type:             return TemporalSubtraction(self.lhs, self.rhs).as_sql(compiler, connection)         expressions = []         expression_params = []


Question: class DurationExpression(CombinedExpression):     def compile(self, side, compiler, connection):        if not isinstance(side, DurationValue):            try:                output = side.output_field            except FieldError:                pass            else:                if output.get_internal_type() == 'DurationField':                    sql, params = compiler.compile(side)                    return connection.ops.format_for_duration_arithmetic(sql), params        try:            output = side.output_field        except FieldError:            pass        else:            if output.get_internal_type() == 'DurationField':                sql, params = compiler.compile(side)                return connection.ops.format_for_duration_arithmetic(sql), params         return compiler.compile(side)      def as_sql(self, compiler, connection):


Question: return []  <del>class DurationValue(Value):    def as_sql(self, compiler, connection):        connection.ops.check_expression_support(self)        if connection.features.has_native_duration_field:            return super().as_sql(compiler, connection)        return connection.ops.date_interval_sql(self.value), []</del> class RawSQL(Expression):     def __init__(self, sql, params, output_field=None):         if output_field is None:


Question: self.lhs, self.rhs = exprs      def as_sql(self, compiler, connection):<del>        try:            lhs_type = self.lhs.output_field.get_internal_type()        except FieldError:            lhs_type = None        try:            rhs_type = self.rhs.output_field.get_internal_type()        except FieldError:            rhs_type = None        if (            not connection.features.has_native_duration_field and            'DurationField' in {lhs_type, rhs_type} and            lhs_type != rhs_type        ):            return DurationExpression(self.lhs, self.connector, self.rhs).as_sql(compiler, connection)        datetime_fields = {'DateField', 'DateTimeField', 'TimeField'}        if self.connector == self.SUB and lhs_type in datetime_fields and lhs_type == rhs_type:            return TemporalSubtraction(self.lhs, self.rhs).as_sql(compiler, connection)</del>         expressions = []         expression_params = []         sql, params = compiler.compile(self.lhs)

make temporal subtraction work without ExpressionWrapper Description class Experiment(models.Model): start = models.DateTimeField() end = models.DateTimeField() Experiment.objects.annotate( delta=F('end') - F('start') + Value(datetime.timedelta(), outputfield=DurationField()) ) This gives: Expression contains mixed types: DateTimeField, DurationField. You must set outputfield.


Question: return expression_wrapper % sql, expression_params      def resolve_expression(self, query=None, allow_joins=True, reuse=None, summarize=False, for_save=False):        lhs = self.lhs.resolve_expression(query, allow_joins, reuse, summarize, for_save)        rhs = self.rhs.resolve_expression(query, allow_joins, reuse, summarize, for_save)        if not isinstance(self, (DurationExpression, TemporalSubtraction)):            try:                lhs_type = lhs.output_field.get_internal_type()            except (AttributeError, FieldError):                lhs_type = None            try:                rhs_type = rhs.output_field.get_internal_type()            except (AttributeError, FieldError):                rhs_type = None            if 'DurationField' in {lhs_type, rhs_type} and lhs_type != rhs_type:                return DurationExpression(self.lhs, self.connector, self.rhs).resolve_expression(                    query, allow_joins, reuse, summarize, for_save,                )            datetime_fields = {'DateField', 'DateTimeField', 'TimeField'}            if self.connector == self.SUB and lhs_type in datetime_fields and lhs_type == rhs_type:                return TemporalSubtraction(self.lhs, self.rhs).resolve_expression(                    query, allow_joins, reuse, summarize, for_save,                )         c = self.copy()         c.is_summary = summarize        c.lhs = c.lhs.resolve_expression(query, allow_joins, reuse, summarize, for_save)        c.rhs = c.rhs.resolve_expression(query, allow_joins, reuse, summarize, for_save)        c.lhs = lhs        c.rhs = rhs         return c


Question: return compiler.compile(side)      def as_sql(self, compiler, connection):<ins>        if connection.features.has_native_duration_field:            return super().as_sql(compiler, connection)</ins>         connection.ops.check_expression_support(self)         expressions = []         expression_params = []


Question: )  def timesince(d, now=None, reversed=False, time_strings=None):def timesince(d, now=None, reversed=False, time_strings=None, depth=2):     """     Take two datetime objects and return the time between d and now as a nicely     formatted string, e.g. "10 minutes". If d occurs after now, return     "0 minutes".      Units used are years, months, weeks, days, hours, and minutes.    Seconds and microseconds are ignored.  Up to two adjacent units will be    Seconds and microseconds are ignored. Up to `depth` adjacent units will be     displayed.  For example, "2 weeks, 3 days" and "1 year, 3 months" are     possible outputs, but "2 weeks, 3 hours" and "1 year, 5 days" are not.      `time_strings` is an optional dict of strings to replace the default     TIME_STRINGS dict.     `depth` is an optional integer to control the number of adjacent time    units returned.     Adapted from     https://web.archive.org/web/20060617175230/http://blog.natbat.co.uk/archive/2003/Jun/14/time_since     """     if time_strings is None:         time_strings = TIME_STRINGS    if depth <= 0:        raise ValueError('depth must be greater than 0.')     # Convert datetime.date to datetime.datetime for comparison.     if not isinstance(d, datetime.datetime):         d = datetime.datetime(d.year, d.month, d.day)

timesince 'depth' parameter Description (last modified by Toby Such) Discussed here. The timesince function is a bit limiting as it is right now. A depth parameter should be to configure how many values are shown. The depth parameter should be defaulted to 2 as this is how the current implementation behaves. The existing rule of values having to be adjacent to one another should still remain. The logic for calculating the time since, before formatting should also be pulled out of the function and placed in its own so that custom implementations can be created. For example: With a depth of one it should always display as "1 week" or "3 years" etc. With a depth of two: "1 week, 3 days" or "3 years, 7 months" With a depth of three: "1 week, 3 days, 5 hours" or "3 years, 7 months, 2 weeks"


Question: count = since // seconds         if count != 0:             break    result = avoid_wrapping(time_strings[name] % count)    if i + 1 < len(TIMESINCE_CHUNKS):        # Now get the second item        seconds2, name2 = TIMESINCE_CHUNKS[i + 1]        count2 = (since - (seconds * count)) // seconds2        if count2 != 0:            result += gettext(', ') + avoid_wrapping(time_strings[name2] % count2)    return result    else:        return avoid_wrapping(time_strings['minute'] % 0)    result = []    current_depth = 0    while i < len(TIMESINCE_CHUNKS) and current_depth < depth:        seconds, name = TIMESINCE_CHUNKS[i]        count = since // seconds        if count == 0:            break        result.append(avoid_wrapping(time_strings[name] % count))        since -= seconds * count        current_depth += 1        i += 1    return gettext(', ').join(result)  def timeuntil(d, now=None, time_strings=None):def timeuntil(d, now=None, time_strings=None, depth=2):     """     Like timesince, but return a string measuring the time until the given time.     """    return timesince(d, now, reversed=True, time_strings=time_strings)    return timesince(d, now, reversed=True, time_strings=time_strings, depth=depth)


Question: obj.annotation_select_mask = None         else:             obj.annotation_select_mask = self.annotation_select_mask.copy()<ins>        obj.combined_queries = tuple(query.clone() for query in self.combined_queries)</ins>         # _annotation_select_cache cannot be copied, as doing so breaks the         # (necessary) state in which both annotations and         # _annotation_select_cache point to the same underlying objects.

() on combined queries returns all results. Description I came across this issue on Stack Overflow. I'm not 100% sure it's a bug, but it does seem strange. With this code (excuse the bizarre example filtering): class Publication(models.Model): pass class Article(models.Model): publications = models.ManyToManyField(to=Publication, blank=True, null=True) class ArticleForm(forms.ModelForm): publications = forms.ModelMultipleChoiceField( (idlt=2) | (idgt=5), required=False, ) class Meta: model = Article fields = ["publications"] class ArticleAdmin(admin.ModelAdmin): form = ArticleForm This works well. However, changing the ModelMultipleChoiceField queryset to use union() breaks things. publications = forms.ModelMultipleChoiceField( (idlt=2).union( (idgt=5) ), required=False, ) The form correctly shows only the matching objects. However, if you submit this form while empty (i.e. you didn't select any publications), ALL objects matching the queryset will be . Using the OR query, NO objects are , as I'd expect.


Question: def set_empty(self):         self.where.add(NothingNode(), AND)<ins>        for query in self.combined_queries:            query.set_empty()</ins>      def is_empty(self):         return any(isinstance(c, NothingNode) for c in self.where.children)