Question: Migration autodetector crashes when renaming a model and field. Description Migration autodetector crashes when renaming a model and field in a single step: $ python makemigrations Did you rename the testone.MyModel model to MyModel2? [y/N] y Traceback (most recent call last): File "", line 22, in <modulemain() File "", line 18, in main executefromcommandline() File "", line 419, in executefromcommandline utility.execute() File "", line 413, in execute self.fetchcommand(subcommand).runfromargv() File "", line 354, in runfromargv self.execute(args, cmdoptions) File "", line 398, in execute output = (args, options) File "", line 89, in wrapped res = handlefunc(args, *kwargs) File "", line 172, in handle changes = autodetector.changes( File "", line 43, in changes changes = self.detectchanges(convertapps, graph) File "", line 182, in detectchanges self.generaterenamedfields() File "", line 823, in generaterenamedfields newmodelstate = [applabel, oldmodelname] KeyError: ('testone', 'mymodel') Reported by HoskeOwl. Regression in .

for app_label, model_name, field_name in sorted(self.new_field_keys - self.old_field_keys):             old_model_name = self.renamed_models.get((app_label, model_name), model_name)             old_model_state = self.from_state.models[app_label, old_model_name]            new_model_state = self.to_state.models[app_label, old_model_name]            new_model_state = self.to_state.models[app_label, model_name]             field = new_model_state.get_field(field_name)             # Scan to see if this is actually a rename!             field_dec = self.deep_deconstruct(field)


Question: filter on exists-subquery with empty queryset removes whole WHERE block Description (last modified by Tobias Bengfort) qs = (~models.Exists(()), name='test') qs <QuerySet []print() EmptyResultSet With django-debug-toolbar I can still see the query, but there WHERE block is missing completely. This seems to be very similar to .

def as_sql(self, compiler, connection, template=None, **extra_context):         query = self.query.exists(using=connection.alias)        sql, params = super().as_sql(            compiler,            connection,            template=template,            query=query,            **extra_context,        )        try:            sql, params = super().as_sql(                compiler,                connection,                template=template,                query=query,                **extra_context,            )        except EmptyResultSet:            if self.negated:                return '', ()            raise         if self.negated:             sql = 'NOT {}'.format(sql)         return sql, params


Question: Dev Server fails to restart after adding BASEDIR to TEMPLATES[0]['DIRS'] in settings Description Repro steps: $ pip install -U django $ django-admin startproject <nameOpen , copy the BASEDIR variable from line 16 and paste it into the empty DIRS list on line 57 $ runserver Back in your IDE, save a file and watch the dev server NOT restart. Back in , remove BASE_DIR from the templates DIRS list. Manually CTRL-C your dev server (as it won't restart on its own when you save), restart the dev server. Now return to your file, re-save it, and notice the development server once again detects changes and restarts. This bug prevents the dev server from restarting no matter where you make changes - it is not just scoped to edits to

@receiver(file_changed, dispatch_uid='template_loaders_file_changed') def template_changed(sender, file_path, **kwargs):<ins>    if file_path.suffix == '.py':        return</ins>     for template_dir in get_template_directories():         if template_dir in file_path.parents:             reset_loaders()


Question: SimpleLazyObject doesn't implement radd Description Technically, there's a whole bunch of magic methods it doesn't implement, compared to a complete proxy implementation, like that of wrapt.ObjectProxy, but radd being missing is the one that's biting me at the moment. As far as I can tell, the implementation can't just be radd = newmethodproxy() because that doesn't exist, which is rubbish. radd = newmethodproxy(operator.attrgetter("radd")) also won't work because types may not have that attr, and attrgetter doesn't supress the exception (correctly) The minimal implementation I've found that works for me is: def radd(self, other): if self.wrapped is empty: () return other + self._wrapped

return result         return copy.deepcopy(self._wrapped, memo) <ins>    __add__ = new_method_proxy(operator.add)    @new_method_proxy    def __radd__(self, other):        return other + self</ins>  def partition(predicate, values):     """


Question: Deprecate passing unsaved objects to related filters. Description Consider this filter: (relatedobj=bar) Where 'bar' is an unsaved object instance. In Django 1.11, this would always return an empty QuerySet (since no Foo object is related to unsaved 'bar'). In Django 2.0 through 2.2, this is equivalent to doing (which can return a non-empty QuerySet): (relatedobj=None) I found a somewhat related issue that touches on this subject: My questions: What is the intended behaviour? In the aforementioned issue Simon Charette suggests that unsaved objects should be prevented from being used in related filters. I agree with that. Is this documented anywhere? I couldn't find anything. At the very least this should be documented somewhere.

<ins>import warnings</ins> from django.db.models.lookups import (     Exact,     GreaterThan,


Question: Deprecate passing unsaved objects to related filters. Description Consider this filter: (relatedobj=bar) Where 'bar' is an unsaved object instance. In Django 1.11, this would always return an empty QuerySet (since no Foo object is related to unsaved 'bar'). In Django 2.0 through 2.2, this is equivalent to doing (which can return a non-empty QuerySet): (relatedobj=None) I found a somewhat related issue that touches on this subject: My questions: What is the intended behaviour? In the aforementioned issue Simon Charette suggests that unsaved objects should be prevented from being used in related filters. I agree with that. Is this documented anywhere? I couldn't find anything. At the very least this should be documented somewhere.

LessThan,     LessThanOrEqual, )<ins>from django.utils.deprecation import RemovedInDjango50Warning</ins>   class MultiColSource:


Question: Deprecate passing unsaved objects to related filters. Description Consider this filter: (relatedobj=bar) Where 'bar' is an unsaved object instance. In Django 1.11, this would always return an empty QuerySet (since no Foo object is related to unsaved 'bar'). In Django 2.0 through 2.2, this is equivalent to doing (which can return a non-empty QuerySet): (relatedobj=None) I found a somewhat related issue that touches on this subject: My questions: What is the intended behaviour? In the aforementioned issue Simon Charette suggests that unsaved objects should be prevented from being used in related filters. I agree with that. Is this documented anywhere? I couldn't find anything. At the very least this should be documented somewhere.

from django.db.models import Model      if isinstance(value, Model):<ins>        if value.pk is None:            # When the deprecation ends, replace with:            # raise ValueError(            #     "Model instances passed to related filters must be saved."            # )            warnings.warn(                "Passing unsaved model instances to related filters is deprecated.",                RemovedInDjango50Warning,            )</ins>         value_list = []         sources = lhs.output_field.path_infos[-1].target_fields         for source in sources:


Question: "Synchronous middleware ... adapted" is not logged for synchronous middleware Description This is a bug in log messages that contradicts the guide in a warning in the docs; the feature works. The guide at , emphasis mine: Warning You will only get the benefits of a fully-asynchronous request stack if you have no synchronous middleware loaded into your site. If there is a piece of synchronous middleware, then Django must use a thread per request to safely emulate a synchronous environment for it. Middleware can be built to support both sync and async contexts. Some of Django’s middleware is built like this, but not all. To see what middleware Django has to adapt, you can turn on debug logging for the django.request logger and look for log messages about “Synchronous middleware … adapted”. The test for "Synchronous middleware ... adapted" is instead testing an async middleware over a sync method:  'middlewareexceptions.middleware.asyncpaymentmiddleware', ]) def testasyncmiddleware(self): with self.assertLogs('django.request', 'DEBUG') as cm: response = ('/middlewareexceptions/view/') self.assertEqual(response.statuscode, 402) self.assertEqual( cm.records[0].getMessage(), "Synchronous middleware " "middlewareexceptions.middleware.asyncpaymentmiddleware " "adapted.", ) About the existing implementation: BaseHandler.loadmiddleware passes the middleware name as name to BaseHandler.adaptmethodmode:  adaptedhandler = self.adaptmethodmode( middlewareisasync, handler, handlerisasync, debug=settings.DEBUG, name='middleware %s' % middlewarepath, ) BaseHandler.adaptmethodmode adapts the method and treats name as the method name rather than the middleware name; when the middleware name is used, it implies a method has been adapted for the middleware, not that the middleware was adapted: if debug and not name: name = name or 'method %s()' % method.qualname if isasync: if not methodisasync: if debug: ('Synchronous %s adapted.', name) return synctoasync(method, threadsensitive=True) elif methodisasync: if debug: ('Asynchronous %s adapted.', name) return asynctosync(method) Proposed fix: Handle middleware name and method name separately within BaseHandler.adaptmethodmode: def adaptmethodmode( self, isasync, method, methodisasync=None, debug=False, name=None, ): """ Adapt a method to be in the correct "mode": If isasync is False: Synchronous methods are left alone Asynchronous methods are wrapped with asynctosync If isasync is True: Synchronous methods are wrapped with synctoasync() Asynchronous methods are left alone """ methodname = None if methodisasync is None: methodisasync = asyncio.iscoroutinefunction(method) if debug and not name: name = name or 'method %s()' % method.qualname methodname = 'method %s()' % method.qualname if isasync: if not methodisasync: if debug: ('Synchronous %s adapted.', name) if name: ('Asynchronous %s adapted.', name) else: ('Synchronous %s adapted.', methodname) return synctoasync(method, threadsensitive=True) elif methodisasync: if debug: ('Asynchronous %s adapted.', name) if name: ('Synchronous %s adapted.', name) else: ('Asynchronous %s adapted.', methodname) return asynctosync(method)

if is_async:             if not method_is_async:                 if debug:                    logger.debug("Synchronous %s adapted.", name)                    logger.debug("Synchronous handler adapted for %s.", name)                 return sync_to_async(method, thread_sensitive=True)         elif method_is_async:             if debug:                logger.debug("Asynchronous %s adapted.", name)                logger.debug("Asynchronous handler adapted for %s.", name)             return async_to_sync(method)         return method


Question: QuerySet.bulkupdate() should perform atomic writes against routed dbforwrite. Description Discovered in but the summary is that bulkupdate doesn't set self.forwrite = True prior to accessing so the latter is actually dbforread and that includes the block creation.

)         # PK is used twice in the resulting update query, once in the filter         # and once in the WHEN. Each field will also have one CAST.<ins>        self._for_write = True</ins>         connection = connections[self.db]         max_batch_size = connection.ops.bulk_batch_size(["pk", "pk"] + fields, objs)         batch_size = min(batch_size, max_batch_size) if batch_size else max_batch_size


Question: QuerySet.bulkupdate() should perform atomic writes against routed dbforwrite. Description Discovered in but the summary is that bulkupdate doesn't set self.forwrite = True prior to accessing so the latter is actually dbforread and that includes the block creation.

update_kwargs[field.attname] = case_statement             updates.append(([obj.pk for obj in batch_objs], update_kwargs))         rows_updated = 0        queryset = self.using(self.db)         with transaction.atomic(using=self.db, savepoint=False):             for pks, update_kwargs in updates:                rows_updated += self.filter(pk__in=pks).update(**update_kwargs)                rows_updated += queryset.filter(pk__in=pks).update(**update_kwargs)         return rows_updated      bulk_update.alters_data = True


Question: Allow parallel test runner to work with Windows/macOS process start method. Description (last modified by Brandon Navra) Python 3.8 on MacOS has changed the default start method for the multiprocessing module from fork to spawn:  When running tests with the --parallel flag, this causes the worker processes to fail with Apps aren't loaded yet. as they no longer have a copy of the parent memory state. It can also cause the workers to fail to find the cloned dbs ( {{ FATAL: database "xxx1" does not exist}} ) as the db test prefix is missing. I have attached a patch which changes (the worker initialiser for ParallelTestSuite) to run () and set the db name to one with the test prefix.

import multiprocessing import os import shutilimport sqlite3 import sys from pathlib import Path from django.db import NotSupportedError from django.db.backends.base.creation import BaseDatabaseCreation


Question: Allow parallel test runner to work with Windows/macOS process start method. Description (last modified by Brandon Navra) Python 3.8 on MacOS has changed the default start method for the multiprocessing module from fork to spawn:  When running tests with the --parallel flag, this causes the worker processes to fail with Apps aren't loaded yet. as they no longer have a copy of the parent memory state. It can also cause the workers to fail to find the cloned dbs ( {{ FATAL: database "xxx1" does not exist}} ) as the db test prefix is missing. I have attached a patch which changes (the worker initialiser for ParallelTestSuite) to run () and set the db name to one with the test prefix.

def get_test_db_clone_settings(self, suffix):         orig_settings_dict = self.connection.settings_dict         source_database_name = orig_settings_dict["NAME"]        if self.is_in_memory_db(source_database_name):        if not self.is_in_memory_db(source_database_name):            root, ext = os.path.splitext(source_database_name)            return {**orig_settings_dict, "NAME": f"{root}_{suffix}{ext}"}        start_method = multiprocessing.get_start_method()        if start_method == "fork":             return orig_settings_dict        else:            root, ext = os.path.splitext(orig_settings_dict["NAME"])            return {**orig_settings_dict, "NAME": "{}_{}{}".format(root, suffix, ext)}        if start_method == "spawn":            return {                **orig_settings_dict,                "NAME": f"{self.connection.alias}_{suffix}.sqlite3",            }        raise NotSupportedError(            f"Cloning with start method {start_method!r} is not supported."        )      def _clone_test_db(self, suffix, verbosity, keepdb=False):         source_database_name = self.connection.settings_dict["NAME"]         target_database_name = self.get_test_db_clone_settings(suffix)["NAME"]        # Forking automatically makes a copy of an in-memory database.         if not self.is_in_memory_db(source_database_name):             # Erase the old test database             if os.access(target_database_name, os.F_OK):


Question: Allow parallel test runner to work with Windows/macOS process start method. Description (last modified by Brandon Navra) Python 3.8 on MacOS has changed the default start method for the multiprocessing module from fork to spawn:  When running tests with the --parallel flag, this causes the worker processes to fail with Apps aren't loaded yet. as they no longer have a copy of the parent memory state. It can also cause the workers to fail to find the cloned dbs ( {{ FATAL: database "xxx1" does not exist}} ) as the db test prefix is missing. I have attached a patch which changes (the worker initialiser for ParallelTestSuite) to run () and set the db name to one with the test prefix.

except Exception as e:                 self.log("Got an error cloning the test database: %s" % e)                 sys.exit(2)<ins>        # Forking automatically makes a copy of an in-memory database.        # Spawn requires migrating to disk which will be re-opened in        # setup_worker_connection.        elif multiprocessing.get_start_method() == "spawn":            ondisk_db = sqlite3.connect(target_database_name, uri=True)            self.connection.connection.backup(ondisk_db)</ins>      def _destroy_test_db(self, test_database_name, verbosity):         if test_database_name and not self.is_in_memory_db(test_database_name):


Question: Allow parallel test runner to work with Windows/macOS process start method. Description (last modified by Brandon Navra) Python 3.8 on MacOS has changed the default start method for the multiprocessing module from fork to spawn:  When running tests with the --parallel flag, this causes the worker processes to fail with Apps aren't loaded yet. as they no longer have a copy of the parent memory state. It can also cause the workers to fail to find the cloned dbs ( {{ FATAL: database "xxx1" does not exist}} ) as the db test prefix is missing. I have attached a patch which changes (the worker initialiser for ParallelTestSuite) to run () and set the db name to one with the test prefix.

else:             sig.append(test_database_name)         return tuple(sig)    def setup_worker_connection(self, _worker_id):        settings_dict = self.get_test_db_clone_settings(_worker_id)        # connection.settings_dict must be updated in place for changes to be        # reflected in django.db.connections. Otherwise new threads would        # connect to the default database instead of the appropriate clone.        start_method = multiprocessing.get_start_method()        if start_method == "fork":            # Update settings_dict in place.            self.connection.settings_dict.update(settings_dict)            self.connection.close()        elif start_method == "spawn":            alias = self.connection.alias            connection_str = (                f"file:memorydb_{alias}_{_worker_id}?mode=memory&cache=shared"            )            source_db = self.connection.Database.connect(                f"file:{alias}_{_worker_id}.sqlite3", uri=True            )            target_db = sqlite3.connect(connection_str, uri=True)            source_db.backup(target_db)            source_db.close()            # Update settings_dict in place.            self.connection.settings_dict.update(settings_dict)            self.connection.settings_dict["NAME"] = connection_str            # Re-open connection to in-memory database before closing copy            # connection.            self.connection.connect()            target_db.close()            if os.environ.get("RUNNING_DJANGOS_TEST_SUITE") == "true":                self.mark_expected_failures_and_skips()diff --git a/django/utils/autoreload.py b/django/utils/autoreload.py-- a/django/utils/autoreload.py++ b/django/utils/autoreload.py


Question: Allow parallel test runner to work with Windows/macOS process start method. Description (last modified by Brandon Navra) Python 3.8 on MacOS has changed the default start method for the multiprocessing module from fork to spawn:  When running tests with the --parallel flag, this causes the worker processes to fail with Apps aren't loaded yet. as they no longer have a copy of the parent memory state. It can also cause the workers to fail to find the cloned dbs ( {{ FATAL: database "xxx1" does not exist}} ) as the db test prefix is missing. I have attached a patch which changes (the worker initialiser for ParallelTestSuite) to run () and set the db name to one with the test prefix.

# cause issues here.         if not isinstance(module, ModuleType):             continue        if module.__name__ == "__main__":        if module.__name__ in ("__main__", "__mp_main__"):             # __main__ (usually manage.py) doesn't always have a __spec__ set.             # Handle this by falling back to using __file__, resolved below.             # See https://docs.python.org/reference/import.html#main-spec


Question: LazyObject defines attribute that don't exist on wrapped object Description LazyObject defines magic methods (getitem, iter) which may be missing from the wrapped object. This leads to the following errors: somevariable = if hasattr(somevariable, "getitem"): foo = somevariable["foo"] # raises TypeError: 'User' object has no attribute 'getitem' if hasattr(somevariable, "iter"): for item in some_variable: # raises TypeError: 'User' object is not iterable

def USE_L10N(self):         stack = traceback.extract_stack()         # Show a warning if the setting is used outside of Django.        # Stack index: -1 this line, -2 the caller.        filename, _, _, _ = stack[-2]        # Stack index: -1 this line, -2 the LazyObject __getattribute__(),        # -3 the caller.        filename, _, _, _ = stack[-3]         if not filename.startswith(os.path.dirname(django.__file__)):             warnings.warn(                 USE_L10N_DEPRECATED_MSG,diff --git a/django/utils/functional.py b/django/utils/functional.py-- a/django/utils/functional.py++ b/django/utils/functional.py


Question: LazyObject defines attribute that don't exist on wrapped object Description LazyObject defines magic methods (getitem, iter) which may be missing from the wrapped object. This leads to the following errors: somevariable = if hasattr(somevariable, "getitem"): foo = somevariable["foo"] # raises TypeError: 'User' object has no attribute 'getitem' if hasattr(somevariable, "iter"): for item in some_variable: # raises TypeError: 'User' object is not iterable

self._setup()         return func(self._wrapped, *args) <ins>    inner._mask_wrapped = False</ins>     return inner


Question: LazyObject defines attribute that don't exist on wrapped object Description LazyObject defines magic methods (getitem, iter) which may be missing from the wrapped object. This leads to the following errors: somevariable = if hasattr(somevariable, "getitem"): foo = somevariable["foo"] # raises TypeError: 'User' object has no attribute 'getitem' if hasattr(somevariable, "iter"): for item in some_variable: # raises TypeError: 'User' object is not iterable

# override __copy__() and __deepcopy__() as well.         self._wrapped = empty <ins>    def __getattribute__(self, name):        value = super().__getattribute__(name)        # If attribute is a proxy method, raise an AttributeError to call        # __getattr__() and use the wrapped object method.        if not getattr(value, "_mask_wrapped", True):            raise AttributeError        return value</ins>     __getattr__ = new_method_proxy(getattr)      def __setattr__(self, name, value):


Question: ManyToManyField to lowercased swappable setting causes generating infinite migrations. Description If I create a custom user model that extends AbstractUser and then try to add a ManyToManyField that references this custom User model, django keeps making the same AlterField migration over and over again unnecessarily. I've attached a Git repository that I used to reproduce this issue with very simple code. You can see the two erroneous migrations after the initial migration. If I use the built in user, there is no issue. It seems to appear once I extend AbstractUser. Git repository:  Here is the accounts app code. from import settings from import AbstractUser from import models class User(AbstractUser): pass class Test(models.Model): members = models.ManyToManyField(settings.AUTHUSERMODEL) Here is one of the erroneous migrations. Notice it depends on 0002altertestmembers which is an erroneous migrations file that looks just like this one. from import settings from import migrations, models class Migration(migrations.Migration): dependencies = [ ('accounts', '0002altertestmembers'), ] operations = [ migrations.AlterField( modelname='test', name='members', field=models.ManyToManyField(to=settings.AUTHUSER_MODEL), ), ]

kwargs["db_table"] = self.db_table         if self.remote_field.db_constraint is not True:             kwargs["db_constraint"] = self.remote_field.db_constraint        # Rel needs more work.        # Lowercase model names as they should be treated as case-insensitive.         if isinstance(self.remote_field.model, str):            kwargs["to"] = self.remote_field.model            if "." in self.remote_field.model:                app_label, model_name = self.remote_field.model.split(".")                kwargs["to"] = "%s.%s" % (app_label, model_name.lower())            else:                kwargs["to"] = self.remote_field.model.lower()         else:            kwargs["to"] = self.remote_field.model._meta.label            kwargs["to"] = self.remote_field.model._meta.label_lower         if getattr(self.remote_field, "through", None) is not None:             if isinstance(self.remote_field.through, str):                 kwargs["through"] = self.remote_field.through


Question: Fallback to a more generic language variant for Select2 translations. Description When using for example the setting LANGUAGECODE="de-ch", the autocompletefield widget will be in English as select2 does not provide the specific regional translation for "de-ch". However the existing translation language "de" would be a much better approximation. I suggest using the language without region from the LANGUAGE_CODE in case an exact match can not be found. This is not a duplicate of .

SELECT2_TRANSLATIONS.update({"zh-hans": "zh-CN", "zh-hant": "zh-TW"})  <ins>def get_select2_language():    lang_code = get_language()    supported_code = SELECT2_TRANSLATIONS.get(lang_code)    if supported_code is None:        # If 'zh-hant-tw' is not supported, try subsequent language codes i.e.        # 'zh-hant' and 'zh'.        i = None        while (i := lang_code.rfind("-", 0, i)) > -1:            if supported_code := SELECT2_TRANSLATIONS.get(lang_code[:i]):                return supported_code    return supported_code</ins> class AutocompleteMixin:     """     Select widget mixin that loads options from AutocompleteJsonView via AJAX.


Question: Fallback to a more generic language variant for Select2 translations. Description When using for example the setting LANGUAGECODE="de-ch", the autocompletefield widget will be in English as select2 does not provide the specific regional translation for "de-ch". However the existing translation language "de" would be a much better approximation. I suggest using the language without region from the LANGUAGE_CODE in case an exact match can not be found. This is not a duplicate of .

self.db = using         self.choices = choices         self.attrs = {} if attrs is None else attrs.copy()        self.i18n_name = SELECT2_TRANSLATIONS.get(get_language())        self.i18n_name = get_select2_language()      def get_url(self):         return reverse(self.url_name % self.admin_site.name)


Question: forces evaluation of lazy objects Description Consider the following example: from import marksafe from import activate, ugettextlazy as s = marksafe(("username")) tpl = Template('{{ s }}') activate('fr') print((Context({'s': s}))) I would expect this to output nom d'utilisateur (which is the french translation of username) but what happens instead is that it outputs username. The reason for this is that marksafe will force the evaluation of the lazy string provided by ugettextlazy when it's called. Unfortunately, the solution to this it trickier than simply wrapping marksafe with , because marksafe can operate both on bytes and text (and allowlazy needs to know the type of object return by the wrapped function). I wrote some tests and a proposed solution on my branch:

from functools import wraps <ins>from django.utils.functional import keep_lazy</ins>  class SafeData:     __slots__ = ()


Question: forces evaluation of lazy objects Description Consider the following example: from import marksafe from import activate, ugettextlazy as s = marksafe(("username")) tpl = Template('{{ s }}') activate('fr') print((Context({'s': s}))) I would expect this to output nom d'utilisateur (which is the french translation of username) but what happens instead is that it outputs username. The reason for this is that marksafe will force the evaluation of the lazy string provided by ugettextlazy when it's called. Unfortunately, the solution to this it trickier than simply wrapping marksafe with , because marksafe can operate both on bytes and text (and allowlazy needs to know the type of object return by the wrapped function). I wrote some tests and a proposed solution on my branch:

return wrapped  <ins>@keep_lazy(SafeString)</ins> def mark_safe(s):     """     Explicitly mark a string as safe for (HTML) output purposes. The returned


Question: ModelAdmin with defined radiofields override emptylabel Description ModelAdmin drops my "emptylabel" and set "defaultemptylabel". For example: class MyModelAdmin(ModelAdmin): radiofields = 'myfield', def formfieldforforeignkey(self, dbfield, args, kwargs): if == 'myfield': kwargs['emptylabel'] = "I WANT TO SET MY OWN EMPTY LABEL" return super().formfieldforforeignkey(dbfield, args, kwargs) You get never the "I WANT TO SET MY OWN EMPTY LABEL" How to fix it: In django\contrib\admin\, row 234: kwargs['emptylabel'] = ('None') if else None Should be changed on: kwargs['emptylabel'] = (('emptylabel') or ('None')) if else None

"class": get_ul_class(self.radio_fields[db_field.name]),                     }                 )                kwargs["empty_label"] = _("None") if db_field.blank else None                kwargs["empty_label"] = (                    kwargs.get("empty_label", _("None")) if db_field.blank else None                )          if "queryset" not in kwargs:             queryset = self.get_field_queryset(db, db_field, request)


Question: Unapplying ContentType migration 0002 does not populate legacy name field on non-default database Description Given a database with the alias other that has all contenttypes migrations applied, roll the state back to initial: $ migrate --database=other contenttypes 0001 All ContentType rows in the other database will be null instead of their intended value.

def add_legacy_name(apps, schema_editor):    alias = schema_editor.connection.alias     ContentType = apps.get_model("contenttypes", "ContentType")    for ct in ContentType.objects.all():    for ct in ContentType.objects.using(alias):         try:             ct.name = apps.get_model(ct.app_label, ct.model)._meta.object_name         except LookupError:


Question: setcookie and setsignedcookie should accept timedelta object for maxage argument Description This already works for getsignedcookie: request.getsignedcookie("mykey", maxage=timedelta(days=3)) This is due to the underlying behaviour of TimestampSigner, which was fixed to do this in . But for setcookie and setsignedcookie it accepts only a number: response = HttpResponse() response.setcookie("mykey", maxage=timedelta(days=3)) TypeError: int() argument must be a string, a bytes-like object or a number, not 'datetime.timedelta'

- a naive ``datetime.datetime`` object in UTC,         - an aware ``datetime.datetime`` object in any time zone.         If it is a ``datetime.datetime`` object then calculate ``max_age``.<ins>        ``max_age`` can be:        - int/float specifying seconds,        - ``datetime.timedelta`` object.</ins>         """         self.cookies[key] = value         if expires is not None:


Question: setcookie and setsignedcookie should accept timedelta object for maxage argument Description This already works for getsignedcookie: request.getsignedcookie("mykey", maxage=timedelta(days=3)) This is due to the underlying behaviour of TimestampSigner, which was fixed to do this in . But for setcookie and setsignedcookie it accepts only a number: response = HttpResponse() response.setcookie("mykey", maxage=timedelta(days=3)) TypeError: int() argument must be a string, a bytes-like object or a number, not 'datetime.timedelta'

delta = delta + datetime.timedelta(seconds=1)                 # Just set max_age - the max_age logic will set expires.                 expires = None                if max_age is not None:                    raise ValueError("'expires' and 'max_age' can't be used together.")                 max_age = max(0, delta.days * 86400 + delta.seconds)             else:                 self.cookies[key]["expires"] = expires         else:             self.cookies[key]["expires"] = ""         if max_age is not None:            if isinstance(max_age, datetime.timedelta):                max_age = max_age.total_seconds()             self.cookies[key]["max-age"] = int(max_age)             # IE requires expires, so set it if hasn't been already.             if not expires:


Question: AppAdmin class for customizing app listing in admin index Description See  for details. This supplements the app directive. As discussed with brosner and jkocherhans in #django-dev: <brosnerit looks reasonable, but haven't spent much time thinking about it <jkocherhansmrts: I think this is clearly backwards incompatible with the current nfa api and has to go in pre 1.0 if it goes in at all <jkocherhansI'm a big -1 on the order attribute and -0 on models (maybe just a different syntax), but the other stuff seems reasonable <mrtsjkocherhans: what's wrong with ordering? <jkocherhansit just feels like the wrong place to specify it <jkocherhansit's a global issue, and an issue any particular app should handle <mrtsmy use case: I have a lot of functionality exposed to somewhat dumb users <mrtsand they have trouble finding the right bits in the admin interface ordering is only used in context of admin index I would like to put the important apps to top and collapse the rest <jkocherhansexactly. what should 3rd party apps put there? therein lies my objection. <mrtswell, I'd say decouple admin from models (as nfa already does) and don't specify any admin options at all -- users are free to customize things with AppAdmin <jkocherhansI guess not if using a AppAdmin class is optional. I was originally thinking it would replace model registration with an admin site. <mrtsjkocherhans: yeah, that's what I kinda meant... it looks more coherent this way jkocherhans: and it may solve some of the issues register() currently has <jkocherhansmrts: I'm gonna have to let it sit for awhile. I'm trying to think of what else an AdminApp class would do besides being a coathanger for a few attributes, nothing is coming to mind. <mrtsjkocherhans: but jezdez has a point -- it would also provide easy bridging for app instances Example syntax follows. class BarModelAdmin(admin.ModelAdmin): description = 'A bar is a bar is a bar' ... class FooAppAdmin(admin.AppAdmin): app = settings.INSTALLED_APPS[0] name = "Foo" # overrides app().name description = "An application that does foo" style = {'classes' : ('collapse',)} order = 1 models = ( # model order in this list determines their display order in app block (BarModel, BarModelAdmin), (BazModel, None), # use default ModelAdmin, don't show description ) (FooAppAdmin) # no need for the tedious for model in [A, B, C, D]: (model)

"models": [model_dict],                 }         if label:            return app_dict.get(label)         return app_dict     def get_app_list(self, request):    def get_app_list(self, request, app_label=None):         """         Return a sorted list of all the installed apps that have been         registered in this site.         """        app_dict = self._build_app_dict(request)        app_dict = self._build_app_dict(request, app_label)          # Sort the apps alphabetically.         app_list = sorted(app_dict.values(), key=lambda x: x["name"].lower())


Question: AppAdmin class for customizing app listing in admin index Description See  for details. This supplements the app directive. As discussed with brosner and jkocherhans in #django-dev: <brosnerit looks reasonable, but haven't spent much time thinking about it <jkocherhansmrts: I think this is clearly backwards incompatible with the current nfa api and has to go in pre 1.0 if it goes in at all <jkocherhansI'm a big -1 on the order attribute and -0 on models (maybe just a different syntax), but the other stuff seems reasonable <mrtsjkocherhans: what's wrong with ordering? <jkocherhansit just feels like the wrong place to specify it <jkocherhansit's a global issue, and an issue any particular app should handle <mrtsmy use case: I have a lot of functionality exposed to somewhat dumb users <mrtsand they have trouble finding the right bits in the admin interface ordering is only used in context of admin index I would like to put the important apps to top and collapse the rest <jkocherhansexactly. what should 3rd party apps put there? therein lies my objection. <mrtswell, I'd say decouple admin from models (as nfa already does) and don't specify any admin options at all -- users are free to customize things with AppAdmin <jkocherhansI guess not if using a AppAdmin class is optional. I was originally thinking it would replace model registration with an admin site. <mrtsjkocherhans: yeah, that's what I kinda meant... it looks more coherent this way jkocherhans: and it may solve some of the issues register() currently has <jkocherhansmrts: I'm gonna have to let it sit for awhile. I'm trying to think of what else an AdminApp class would do besides being a coathanger for a few attributes, nothing is coming to mind. <mrtsjkocherhans: but jezdez has a point -- it would also provide easy bridging for app instances Example syntax follows. class BarModelAdmin(admin.ModelAdmin): description = 'A bar is a bar is a bar' ... class FooAppAdmin(admin.AppAdmin): app = settings.INSTALLED_APPS[0] name = "Foo" # overrides app().name description = "An application that does foo" style = {'classes' : ('collapse',)} order = 1 models = ( # model order in this list determines their display order in app block (BarModel, BarModelAdmin), (BazModel, None), # use default ModelAdmin, don't show description ) (FooAppAdmin) # no need for the tedious for model in [A, B, C, D]: (model)

)      def app_index(self, request, app_label, extra_context=None):        app_dict = self._build_app_dict(request, app_label)        if not app_dict:        app_list = self.get_app_list(request, app_label)        if not app_list:             raise Http404("The requested admin page does not exist.")        # Sort the models alphabetically within each app.        app_dict["models"].sort(key=lambda x: x["name"])         context = {             **self.each_context(request),            "title": _("%(app)s administration") % {"app": app_dict["name"]},            "title": _("%(app)s administration") % {"app": app_list[0]["name"]},             "subtitle": None,            "app_list": [app_dict],            "app_list": app_list,             "app_label": app_label,             **(extra_context or {}),         }


Question: Builtin csrffailure() view uses wrong charset Description When Django detects wrong CSRF token, it shows an error using view That file ends with return HttpResponseForbidden(t.render(c), contenttype="text/html;") When the template (CSRFFAILURETEMPLATENAME) is written using UTF-8, it is rendered incorrectly. I suggest changing that line to return HttpResponseForbidden(t.render(c), contenttype="text/html;"+ f" charset={settings.DEFAULTCHARSET};") or perhaps leaving out the content_type entirely. Currently I'm using a workaround, by adding <meta http-equiv="Content-type" content="text/html; charset=utf-8" /to the template's HEAD, but it seems to me that the suggested fix is a better solution.

else:             # Raise if a developer-specified template doesn't exist.             raise    return HttpResponseForbidden(t.render(c), content_type="text/html")    return HttpResponseForbidden(t.render(c))diff --git a/django/views/debug.py b/django/views/debug.py-- a/django/views/debug.py++ b/django/views/debug.py


Question: Builtin csrffailure() view uses wrong charset Description When Django detects wrong CSRF token, it shows an error using view That file ends with return HttpResponseForbidden(t.render(c), contenttype="text/html;") When the template (CSRFFAILURETEMPLATENAME) is written using UTF-8, it is rendered incorrectly. I suggest changing that line to return HttpResponseForbidden(t.render(c), contenttype="text/html;"+ f" charset={settings.DEFAULTCHARSET};") or perhaps leaving out the content_type entirely. Currently I'm using a workaround, by adding <meta http-equiv="Content-type" content="text/html; charset=utf-8" /to the template's HEAD, but it seems to me that the suggested fix is a better solution.

reporter = get_exception_reporter_class(request)(request, exc_type, exc_value, tb)     if request.accepts("text/html"):         html = reporter.get_traceback_html()        return HttpResponse(html, status=status_code, content_type="text/html")        return HttpResponse(html, status=status_code)     else:         text = reporter.get_traceback_text()         return HttpResponse(


Question: Builtin csrffailure() view uses wrong charset Description When Django detects wrong CSRF token, it shows an error using view That file ends with return HttpResponseForbidden(t.render(c), contenttype="text/html;") When the template (CSRFFAILURETEMPLATENAME) is written using UTF-8, it is rendered incorrectly. I suggest changing that line to return HttpResponseForbidden(t.render(c), contenttype="text/html;"+ f" charset={settings.DEFAULTCHARSET};") or perhaps leaving out the content_type entirely. Currently I'm using a workaround, by adding <meta http-equiv="Content-type" content="text/html; charset=utf-8" /to the template's HEAD, but it seems to me that the suggested fix is a better solution.

"raising_view_name": get_caller(request),         }     )    return HttpResponseNotFound(t.render(c), content_type="text/html")    return HttpResponseNotFound(t.render(c))   def default_urlconf(request):


Question: Builtin csrffailure() view uses wrong charset Description When Django detects wrong CSRF token, it shows an error using view That file ends with return HttpResponseForbidden(t.render(c), contenttype="text/html;") When the template (CSRFFAILURETEMPLATENAME) is written using UTF-8, it is rendered incorrectly. I suggest changing that line to return HttpResponseForbidden(t.render(c), contenttype="text/html;"+ f" charset={settings.DEFAULTCHARSET};") or perhaps leaving out the content_type entirely. Currently I'm using a workaround, by adding <meta http-equiv="Content-type" content="text/html; charset=utf-8" /to the template's HEAD, but it seems to me that the suggested fix is a better solution.

}     )     return HttpResponse(t.render(c), content_type="text/html")    return HttpResponse(t.render(c))diff --git a/django/views/defaults.py b/django/views/defaults.py-- a/django/views/defaults.py++ b/django/views/defaults.py


Question: Builtin csrffailure() view uses wrong charset Description When Django detects wrong CSRF token, it shows an error using view That file ends with return HttpResponseForbidden(t.render(c), contenttype="text/html;") When the template (CSRFFAILURETEMPLATENAME) is written using UTF-8, it is rendered incorrectly. I suggest changing that line to return HttpResponseForbidden(t.render(c), contenttype="text/html;"+ f" charset={settings.DEFAULTCHARSET};") or perhaps leaving out the content_type entirely. Currently I'm using a workaround, by adding <meta http-equiv="Content-type" content="text/html; charset=utf-8" /to the template's HEAD, but it seems to me that the suggested fix is a better solution.

try:         template = loader.get_template(template_name)         body = template.render(context, request)<del>        content_type = None  # Django will use 'text/html'.</del>     except TemplateDoesNotExist:         if template_name != ERROR_404_TEMPLATE_NAME:             # Reraise if it's a missing custom template.


Question: Builtin csrffailure() view uses wrong charset Description When Django detects wrong CSRF token, it shows an error using view That file ends with return HttpResponseForbidden(t.render(c), contenttype="text/html;") When the template (CSRFFAILURETEMPLATENAME) is written using UTF-8, it is rendered incorrectly. I suggest changing that line to return HttpResponseForbidden(t.render(c), contenttype="text/html;"+ f" charset={settings.DEFAULTCHARSET};") or perhaps leaving out the content_type entirely. Currently I'm using a workaround, by adding <meta http-equiv="Content-type" content="text/html; charset=utf-8" /to the template's HEAD, but it seems to me that the suggested fix is a better solution.

},         )         body = template.render(Context(context))        content_type = "text/html"    return HttpResponseNotFound(body, content_type=content_type)    return HttpResponseNotFound(body)   @requires_csrf_token


Question: Builtin csrffailure() view uses wrong charset Description When Django detects wrong CSRF token, it shows an error using view That file ends with return HttpResponseForbidden(t.render(c), contenttype="text/html;") When the template (CSRFFAILURETEMPLATENAME) is written using UTF-8, it is rendered incorrectly. I suggest changing that line to return HttpResponseForbidden(t.render(c), contenttype="text/html;"+ f" charset={settings.DEFAULTCHARSET};") or perhaps leaving out the content_type entirely. Currently I'm using a workaround, by adding <meta http-equiv="Content-type" content="text/html; charset=utf-8" /to the template's HEAD, but it seems to me that the suggested fix is a better solution.

raise         return HttpResponseServerError(             ERROR_PAGE_TEMPLATE % {"title": "Server Error (500)", "details": ""},<del>            content_type="text/html",</del>         )     return HttpResponseServerError(template.render())


Question: Builtin csrffailure() view uses wrong charset Description When Django detects wrong CSRF token, it shows an error using view That file ends with return HttpResponseForbidden(t.render(c), contenttype="text/html;") When the template (CSRFFAILURETEMPLATENAME) is written using UTF-8, it is rendered incorrectly. I suggest changing that line to return HttpResponseForbidden(t.render(c), contenttype="text/html;"+ f" charset={settings.DEFAULTCHARSET};") or perhaps leaving out the content_type entirely. Currently I'm using a workaround, by adding <meta http-equiv="Content-type" content="text/html; charset=utf-8" /to the template's HEAD, but it seems to me that the suggested fix is a better solution.

raise         return HttpResponseBadRequest(             ERROR_PAGE_TEMPLATE % {"title": "Bad Request (400)", "details": ""},<del>            content_type="text/html",</del>         )     # No exception content is passed to the template, to not disclose any     # sensitive information.


Question: Builtin csrffailure() view uses wrong charset Description When Django detects wrong CSRF token, it shows an error using view That file ends with return HttpResponseForbidden(t.render(c), contenttype="text/html;") When the template (CSRFFAILURETEMPLATENAME) is written using UTF-8, it is rendered incorrectly. I suggest changing that line to return HttpResponseForbidden(t.render(c), contenttype="text/html;"+ f" charset={settings.DEFAULTCHARSET};") or perhaps leaving out the content_type entirely. Currently I'm using a workaround, by adding <meta http-equiv="Content-type" content="text/html; charset=utf-8" /to the template's HEAD, but it seems to me that the suggested fix is a better solution.

raise         return HttpResponseForbidden(             ERROR_PAGE_TEMPLATE % {"title": "403 Forbidden", "details": ""},<del>            content_type="text/html",</del>         )     return HttpResponseForbidden(         template.render(request=request, context={"exception": str(exception)})


Question: Allow syncing user attributes on every authentication with RemoteUserBackend. Description If using a custom RemoteUserBackend, it is possible to want to synchronize any changes from the remote system back into the Django user records whenever authentication happens. Currently, if any user attributes change in the remote system there is no easy way to reflect these changes back into the users in the Django system. The goal of this feature is to introduce a new method in the class called synchronizeuser with a method signature equal to that of configureuser (which it complements) that will be called in the authenticate method of said class right after fetching the user from the database (if any), regardless of whether the user was unknown and created or otherwise. Implementors can then override this method and implement data synchronization between the remote user and the Django user that will be applied on every user authentication attempt.

import warnings from django.contrib.auth import get_user_model from django.contrib.auth.models import Permission from django.db.models import Exists, OuterRef, Qfrom django.utils.deprecation import RemovedInDjango50Warningfrom django.utils.inspect import func_supports_parameter  UserModel = get_user_model()


Question: Allow syncing user attributes on every authentication with RemoteUserBackend. Description If using a custom RemoteUserBackend, it is possible to want to synchronize any changes from the remote system back into the Django user records whenever authentication happens. Currently, if any user attributes change in the remote system there is no easy way to reflect these changes back into the users in the Django system. The goal of this feature is to introduce a new method in the class called synchronizeuser with a method signature equal to that of configureuser (which it complements) that will be called in the authenticate method of said class right after fetching the user from the database (if any), regardless of whether the user was unknown and created or otherwise. Implementors can then override this method and implement data synchronization between the remote user and the Django user that will be applied on every user authentication attempt.

"""         if not remote_user:             return<ins>        created = False</ins>         user = None         username = self.clean_username(remote_user)


Question: Allow syncing user attributes on every authentication with RemoteUserBackend. Description If using a custom RemoteUserBackend, it is possible to want to synchronize any changes from the remote system back into the Django user records whenever authentication happens. Currently, if any user attributes change in the remote system there is no easy way to reflect these changes back into the users in the Django system. The goal of this feature is to introduce a new method in the class called synchronizeuser with a method signature equal to that of configureuser (which it complements) that will be called in the authenticate method of said class right after fetching the user from the database (if any), regardless of whether the user was unknown and created or otherwise. Implementors can then override this method and implement data synchronization between the remote user and the Django user that will be applied on every user authentication attempt.

user, created = UserModel._default_manager.get_or_create(                 **{UserModel.USERNAME_FIELD: username}             )            if created:                user = self.configure_user(request, user)         else:             try:                 user = UserModel._default_manager.get_by_natural_key(username)             except UserModel.DoesNotExist:                 pass        # RemovedInDjango50Warning: When the deprecation ends, replace with:        #   user = self.configure_user(request, user, created=created)        if func_supports_parameter(self.configure_user, "created"):            user = self.configure_user(request, user, created=created)        else:            warnings.warn(                f"`created=True` must be added to the signature of "                f"{self.__class__.__qualname__}.configure_user().",                category=RemovedInDjango50Warning,            )            if created:                user = self.configure_user(request, user)         return user if self.user_can_authenticate(user) else None      def clean_username(self, username):


Question: Allow syncing user attributes on every authentication with RemoteUserBackend. Description If using a custom RemoteUserBackend, it is possible to want to synchronize any changes from the remote system back into the Django user records whenever authentication happens. Currently, if any user attributes change in the remote system there is no easy way to reflect these changes back into the users in the Django system. The goal of this feature is to introduce a new method in the class called synchronizeuser with a method signature equal to that of configureuser (which it complements) that will be called in the authenticate method of said class right after fetching the user from the database (if any), regardless of whether the user was unknown and created or otherwise. Implementors can then override this method and implement data synchronization between the remote user and the Django user that will be applied on every user authentication attempt.

"""         return username     def configure_user(self, request, user):    def configure_user(self, request, user, created=True):         """        Configure a user after creation and return the updated user.        Configure a user and return the updated user.          By default, return the user unmodified.         """


Question: Add support for multiple values for the x-forwarded-proto header Description When Django is deployed behind more than one proxy, the proxy behavior is sometimes to list the protocol as a comma-separated list. However, currently, Django expects only one value for the x-forwarded-proto header, instead of parsing it as a list of values and setting the protocol accordingly. x-forwarded-proto is a non-standard header, so there isn't a specification for its use, but different reverse-proxy vendors do use it in different ways, and some append the protocol as a comma-separated value from left-to-right (left being the furthermost proxy and rightmost being the closest). Similar issues have been raised and implemented in other projects, for example: Tornado: Issue:  Implementation:  Ruby: Issue:  Implemenation:  Reactor-Netty:  Implementation:  Most implementation use the leftmost-value or rightmost value. I would expect that provided that you are certain that the initial proxy can be trusted, that the left-most value makes the most sense, since it represent the original value at the entry-point for the HTTP request which is often where TLS is being terminated. Common example of this behavior is when using mulitple AWS proxies such as API Gateway proxying to an elastic load balancer.

)             header_value = self.META.get(header)             if header_value is not None:                return "https" if header_value == secure_value else "http"                header_value, *_ = header_value.split(",", 1)                return "https" if header_value.strip() == secure_value else "http"         return self._get_scheme()      def is_secure(self):


Question: Fix handling empty string for If-Modified-Since header Description Empty string used to be ignored for If-Modified-Since header, but now raises exception since . Fix handling empty string for If-Modified-Since header Description Empty string used to be ignored for If-Modified-Since header, but now raises exception since .

if header is None:             raise ValueError         matches = re.match(r"^([^;]+)(; length=([0-9]+))?$", header, re.IGNORECASE)        if matches is None:            raise ValueError         header_mtime = parse_http_date(matches[1])         header_len = matches[3]         if header_len and int(header_len) != size:             raise ValueError         if int(mtime) > header_mtime:             raise ValueError    except (AttributeError, ValueError, OverflowError):    except (ValueError, OverflowError):         return True     return False


Question: Optimize CreateModel + AlterModelManagers to CreateModel Description During migration optimization, CreateModel + AlterModelOptions is reduced to just CreateModel, with the model options. Similarly, CreateModel + AlterModelManagers can become just CreateModel.

managers=self.managers,                 ),             ]<ins>        elif (            isinstance(operation, AlterModelManagers)            and self.name_lower == operation.name_lower        ):            return [                CreateModel(                    self.name,                    fields=self.fields,                    options=self.options,                    bases=self.bases,                    managers=operation.managers,                ),            ]</ins>         elif (             isinstance(operation, AlterTogetherOptionOperation)             and self.name_lower == operation.name_lower


Question: haskey, haskeys, and hasanykeys JSONField() lookups don't handle numeric keys on SQLite, MySQL, and Oracle. Description (last modified by TheTerrasque) Problem When using models.JSONField() haskey lookup with numerical keys on SQLite database it fails to find the keys. Versions: Django: 4.0.3 Python: 3.9.6 (tags/v3.9.6:, Jun 28 2021, 15:26:21) [MSC v.1929 64 bit (AMD64)] on win32 sqlite3.version: '2.6.0' sqlite3.sqliteversion: '3.35.5' Example: Database DATABASES = { 'default': { 'ENGINE': '', 'NAME': 'db.sqlite3', } } Model class JsonFieldHasKeyTest(models.Model): data = models.JSONField() Test from import TestCase from .models import JsonFieldHasKeyTest class JsonFieldHasKeyTestCase(TestCase): def setUp(self) -None: test = JsonFieldHasKeyTest(data={'foo': 'bar'}) test2 = JsonFieldHasKeyTest(data={'1111': 'bar'}) () () def testjsonfieldhaskey(self): c1 = (datahaskey='foo').count() c2 = (datahaskey='1111').count() self.assertEqual(c1, 1, "Should have found 1 entry with key 'foo'") self.assertEqual(c2, 1, "Should have found 1 entry with key '1111'") Result FAIL: testjsonfieldhaskey () Traceback (most recent call last): File "H:\Files\Projects\Electaco\Webservice\elecserve\markers\", line 16, in testjsonfieldhaskey self.assertEqual(c2, 1, "Should have found 1 entry with key '1111'") AssertionError: 0 != 1 : Should have found 1 entry with key '1111' Additional info This has been tested on SQLite and Postgresql backend, it works on postgresql but fails on sqlite.

class HasKeyLookup(PostgresOperatorLookup):     logical_operator = None <ins>    def compile_json_path_final_key(self, key_transform):        # Compile the final key without interpreting ints as array elements.        return ".%s" % json.dumps(key_transform)</ins>     def as_sql(self, compiler, connection, template=None):         # Process JSON path from the left-hand side.         if isinstance(self.lhs, KeyTransform):


Question: haskey, haskeys, and hasanykeys JSONField() lookups don't handle numeric keys on SQLite, MySQL, and Oracle. Description (last modified by TheTerrasque) Problem When using models.JSONField() haskey lookup with numerical keys on SQLite database it fails to find the keys. Versions: Django: 4.0.3 Python: 3.9.6 (tags/v3.9.6:, Jun 28 2021, 15:26:21) [MSC v.1929 64 bit (AMD64)] on win32 sqlite3.version: '2.6.0' sqlite3.sqliteversion: '3.35.5' Example: Database DATABASES = { 'default': { 'ENGINE': '', 'NAME': 'db.sqlite3', } } Model class JsonFieldHasKeyTest(models.Model): data = models.JSONField() Test from import TestCase from .models import JsonFieldHasKeyTest class JsonFieldHasKeyTestCase(TestCase): def setUp(self) -None: test = JsonFieldHasKeyTest(data={'foo': 'bar'}) test2 = JsonFieldHasKeyTest(data={'1111': 'bar'}) () () def testjsonfieldhaskey(self): c1 = (datahaskey='foo').count() c2 = (datahaskey='1111').count() self.assertEqual(c1, 1, "Should have found 1 entry with key 'foo'") self.assertEqual(c2, 1, "Should have found 1 entry with key '1111'") Result FAIL: testjsonfieldhaskey () Traceback (most recent call last): File "H:\Files\Projects\Electaco\Webservice\elecserve\markers\", line 16, in testjsonfieldhaskey self.assertEqual(c2, 1, "Should have found 1 entry with key '1111'") AssertionError: 0 != 1 : Should have found 1 entry with key '1111' Additional info This has been tested on SQLite and Postgresql backend, it works on postgresql but fails on sqlite.

*_, rhs_key_transforms = key.preprocess_lhs(compiler, connection)             else:                 rhs_key_transforms = [key]            rhs_params.append(                "%s%s"                % (                    lhs_json_path,                    compile_json_path(rhs_key_transforms, include_root=False),                )            )            *rhs_key_transforms, final_key = rhs_key_transforms            rhs_json_path = compile_json_path(rhs_key_transforms, include_root=False)            rhs_json_path += self.compile_json_path_final_key(final_key)            rhs_params.append(lhs_json_path + rhs_json_path)         # Add condition for each key.         if self.logical_operator:             sql = "(%s)" % self.logical_operator.join([sql] * len(rhs_params))


Question: haskey, haskeys, and hasanykeys JSONField() lookups don't handle numeric keys on SQLite, MySQL, and Oracle. Description (last modified by TheTerrasque) Problem When using models.JSONField() haskey lookup with numerical keys on SQLite database it fails to find the keys. Versions: Django: 4.0.3 Python: 3.9.6 (tags/v3.9.6:, Jun 28 2021, 15:26:21) [MSC v.1929 64 bit (AMD64)] on win32 sqlite3.version: '2.6.0' sqlite3.sqliteversion: '3.35.5' Example: Database DATABASES = { 'default': { 'ENGINE': '', 'NAME': 'db.sqlite3', } } Model class JsonFieldHasKeyTest(models.Model): data = models.JSONField() Test from import TestCase from .models import JsonFieldHasKeyTest class JsonFieldHasKeyTestCase(TestCase): def setUp(self) -None: test = JsonFieldHasKeyTest(data={'foo': 'bar'}) test2 = JsonFieldHasKeyTest(data={'1111': 'bar'}) () () def testjsonfieldhaskey(self): c1 = (datahaskey='foo').count() c2 = (datahaskey='1111').count() self.assertEqual(c1, 1, "Should have found 1 entry with key 'foo'") self.assertEqual(c2, 1, "Should have found 1 entry with key '1111'") Result FAIL: testjsonfieldhaskey () Traceback (most recent call last): File "H:\Files\Projects\Electaco\Webservice\elecserve\markers\", line 16, in testjsonfieldhaskey self.assertEqual(c2, 1, "Should have found 1 entry with key '1111'") AssertionError: 0 != 1 : Should have found 1 entry with key '1111' Additional info This has been tested on SQLite and Postgresql backend, it works on postgresql but fails on sqlite.

logical_operator = " OR "  <ins>class HasKeyOrArrayIndex(HasKey):    def compile_json_path_final_key(self, key_transform):        return compile_json_path([key_transform], include_root=False)</ins> class CaseInsensitiveMixin:     """     Mixin to allow case-insensitive comparison of JSON values on MySQL.


Question: haskey, haskeys, and hasanykeys JSONField() lookups don't handle numeric keys on SQLite, MySQL, and Oracle. Description (last modified by TheTerrasque) Problem When using models.JSONField() haskey lookup with numerical keys on SQLite database it fails to find the keys. Versions: Django: 4.0.3 Python: 3.9.6 (tags/v3.9.6:, Jun 28 2021, 15:26:21) [MSC v.1929 64 bit (AMD64)] on win32 sqlite3.version: '2.6.0' sqlite3.sqliteversion: '3.35.5' Example: Database DATABASES = { 'default': { 'ENGINE': '', 'NAME': 'db.sqlite3', } } Model class JsonFieldHasKeyTest(models.Model): data = models.JSONField() Test from import TestCase from .models import JsonFieldHasKeyTest class JsonFieldHasKeyTestCase(TestCase): def setUp(self) -None: test = JsonFieldHasKeyTest(data={'foo': 'bar'}) test2 = JsonFieldHasKeyTest(data={'1111': 'bar'}) () () def testjsonfieldhaskey(self): c1 = (datahaskey='foo').count() c2 = (datahaskey='1111').count() self.assertEqual(c1, 1, "Should have found 1 entry with key 'foo'") self.assertEqual(c2, 1, "Should have found 1 entry with key '1111'") Result FAIL: testjsonfieldhaskey () Traceback (most recent call last): File "H:\Files\Projects\Electaco\Webservice\elecserve\markers\", line 16, in testjsonfieldhaskey self.assertEqual(c2, 1, "Should have found 1 entry with key '1111'") AssertionError: 0 != 1 : Should have found 1 entry with key '1111' Additional info This has been tested on SQLite and Postgresql backend, it works on postgresql but fails on sqlite.

class KeyTransformIsNull(lookups.IsNull):     # key__isnull=False is the same as has_key='key'     def as_oracle(self, compiler, connection):        sql, params = HasKey(        sql, params = HasKeyOrArrayIndex(             self.lhs.lhs,             self.lhs.key_name,         ).as_oracle(compiler, connection)


Question: haskey, haskeys, and hasanykeys JSONField() lookups don't handle numeric keys on SQLite, MySQL, and Oracle. Description (last modified by TheTerrasque) Problem When using models.JSONField() haskey lookup with numerical keys on SQLite database it fails to find the keys. Versions: Django: 4.0.3 Python: 3.9.6 (tags/v3.9.6:, Jun 28 2021, 15:26:21) [MSC v.1929 64 bit (AMD64)] on win32 sqlite3.version: '2.6.0' sqlite3.sqliteversion: '3.35.5' Example: Database DATABASES = { 'default': { 'ENGINE': '', 'NAME': 'db.sqlite3', } } Model class JsonFieldHasKeyTest(models.Model): data = models.JSONField() Test from import TestCase from .models import JsonFieldHasKeyTest class JsonFieldHasKeyTestCase(TestCase): def setUp(self) -None: test = JsonFieldHasKeyTest(data={'foo': 'bar'}) test2 = JsonFieldHasKeyTest(data={'1111': 'bar'}) () () def testjsonfieldhaskey(self): c1 = (datahaskey='foo').count() c2 = (datahaskey='1111').count() self.assertEqual(c1, 1, "Should have found 1 entry with key 'foo'") self.assertEqual(c2, 1, "Should have found 1 entry with key '1111'") Result FAIL: testjsonfieldhaskey () Traceback (most recent call last): File "H:\Files\Projects\Electaco\Webservice\elecserve\markers\", line 16, in testjsonfieldhaskey self.assertEqual(c2, 1, "Should have found 1 entry with key '1111'") AssertionError: 0 != 1 : Should have found 1 entry with key '1111' Additional info This has been tested on SQLite and Postgresql backend, it works on postgresql but fails on sqlite.

template = "JSON_TYPE(%s, %%s) IS NULL"         if not self.rhs:             template = "JSON_TYPE(%s, %%s) IS NOT NULL"        return HasKey(self.lhs.lhs, self.lhs.key_name).as_sql(        return HasKeyOrArrayIndex(self.lhs.lhs, self.lhs.key_name).as_sql(             compiler,             connection,             template=template,


Question: haskey, haskeys, and hasanykeys JSONField() lookups don't handle numeric keys on SQLite, MySQL, and Oracle. Description (last modified by TheTerrasque) Problem When using models.JSONField() haskey lookup with numerical keys on SQLite database it fails to find the keys. Versions: Django: 4.0.3 Python: 3.9.6 (tags/v3.9.6:, Jun 28 2021, 15:26:21) [MSC v.1929 64 bit (AMD64)] on win32 sqlite3.version: '2.6.0' sqlite3.sqliteversion: '3.35.5' Example: Database DATABASES = { 'default': { 'ENGINE': '', 'NAME': 'db.sqlite3', } } Model class JsonFieldHasKeyTest(models.Model): data = models.JSONField() Test from import TestCase from .models import JsonFieldHasKeyTest class JsonFieldHasKeyTestCase(TestCase): def setUp(self) -None: test = JsonFieldHasKeyTest(data={'foo': 'bar'}) test2 = JsonFieldHasKeyTest(data={'1111': 'bar'}) () () def testjsonfieldhaskey(self): c1 = (datahaskey='foo').count() c2 = (datahaskey='1111').count() self.assertEqual(c1, 1, "Should have found 1 entry with key 'foo'") self.assertEqual(c2, 1, "Should have found 1 entry with key '1111'") Result FAIL: testjsonfieldhaskey () Traceback (most recent call last): File "H:\Files\Projects\Electaco\Webservice\elecserve\markers\", line 16, in testjsonfieldhaskey self.assertEqual(c2, 1, "Should have found 1 entry with key '1111'") AssertionError: 0 != 1 : Should have found 1 entry with key '1111' Additional info This has been tested on SQLite and Postgresql backend, it works on postgresql but fails on sqlite.

rhs, rhs_params = super().process_rhs(compiler, connection)         if rhs_params == ["null"]:             # Field has key and it's NULL.            has_key_expr = HasKey(self.lhs.lhs, self.lhs.key_name)            has_key_expr = HasKeyOrArrayIndex(self.lhs.lhs, self.lhs.key_name)             has_key_sql, has_key_params = has_key_expr.as_oracle(compiler, connection)             is_null_expr = self.lhs.get_lookup("isnull")(self.lhs, True)             is_null_sql, is_null_params = is_null_expr.as_sql(compiler, connection)


Question: makemessages command skips the nl-nl-x-informal folder Description There is a mismatch on the DjangoTranslation folder and the makemessages command for the value nl-nl-x-informal DjangoTranslation uses the tolocale method to determine the language folder to read the tolocale translates nl-nl-x-informal correctly to the nlNL-x-informal folder. However makemessages skips the nlNL-x-informal folder and displays the following message invalid locale nlNL-x-informal, did you mean nlNLxinformal?  The check for - in the locale should only be for the first section a.k.a. nl_NL

)  <ins>def is_valid_locale(locale):    return re.match(r"^[a-z]+$", locale) or re.match(r"^[a-z]+_[A-Z].*$", locale)</ins> @total_ordering class TranslatableFile:     def __init__(self, dirpath, file_name, locale_dir):


Question: makemessages command skips the nl-nl-x-informal folder Description There is a mismatch on the DjangoTranslation folder and the makemessages command for the value nl-nl-x-informal DjangoTranslation uses the tolocale method to determine the language folder to read the tolocale translates nl-nl-x-informal correctly to the nlNL-x-informal folder. However makemessages skips the nlNL-x-informal folder and displays the following message invalid locale nlNL-x-informal, did you mean nlNLxinformal?  The check for - in the locale should only be for the first section a.k.a. nl_NL

# Build po files for each selected locale             for locale in locales:                if "-" in locale:                    self.stdout.write(                        "invalid locale %s, did you mean %s?"                        % (                            locale,                            locale.replace("-", "_"),                        ),                if not is_valid_locale(locale):                    # Try to guess what valid locale it could be                    # Valid examples are: en_GB, shi_Latn_MA and nl_NL-x-informal                    # Search for characters followed by a non character (i.e. separator)                    match = re.match(                        r"^(?P<language>[a-zA-Z]+)"                        r"(?P<separator>[^a-zA-Z])"                        r"(?P<territory>.+)$",                        locale,                     )                    if match:                        locale_parts = match.groupdict()                        language = locale_parts["language"].lower()                        territory = (                            locale_parts["territory"][:2].upper()                            + locale_parts["territory"][2:]                        )                        proposed_locale = f"{language}_{territory}"                    else:                        # It could be a language in uppercase                        proposed_locale = locale.lower()                    # Recheck if the proposed locale is valid                    if is_valid_locale(proposed_locale):                        self.stdout.write(                            "invalid locale %s, did you mean %s?"                            % (                                locale,                                proposed_locale,                            ),                        )                    else:                        self.stdout.write("invalid locale %s" % locale)                     continue                 if self.verbosity > 0:                     self.stdout.write("processing locale %s" % locale)


Question: loaddata fails on non-default database when natural keys uses foreign keys. Description (last modified by François Granade) I've got a one-to-many relationship between two models Book and Author, that define a natural keys in both models. I'm loading some data from a fixture. It works in the default database, but when I use it a second database, then I get an exception. I'm relatively new to natural keys and to serializers, but I wouldn't expect things to work differently in the default DB and others ? I've committed a test project here:  (The problem doesn't appear if the data is already present in the default DB) The error: % cat | loaddata --database other --format json - Traceback (most recent call last): File "/Users/francois/Library/Caches/pypoetry/virtualenvs/exportbug-PGt-cwXF-py3.9/lib/python3.9/site-", line 187, in get relobj = (instance) File "/Users/francois/Library/Caches/pypoetry/virtualenvs/exportbug-PGt-cwXF-py3.9/lib/python3.9/site-", line 15, in getcachedvalue return [cachename] KeyError: 'author' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/Users/francois/Library/Caches/pypoetry/virtualenvs/exportbug-PGt-cwXF-py3.9/lib/python3.9/site-", line 70, in Deserializer yield from PythonDeserializer(objects, options) File "/Users/francois/Library/Caches/pypoetry/virtualenvs/exportbug-PGt-cwXF-py3.9/lib/python3.9/site-", line 174, in Deserializer obj = base.buildinstance(Model, data, using) File "/Users/francois/Library/Caches/pypoetry/virtualenvs/exportbug-PGt-cwXF-py3.9/lib/python3.9/site-", line 332, in buildinstance naturalkey = Model(data).naturalkey() File "/Users/francois/lmad/src/django-", line 33, in naturalkey return (,) + () File "/Users/francois/Library/Caches/pypoetry/virtualenvs/exportbug-PGt-cwXF-py3.9/lib/python3.9/site-", line 205, in get relobj = self.getobject(instance) File "/Users/francois/Library/Caches/pypoetry/virtualenvs/exportbug-PGt-cwXF-py3.9/lib/python3.9/site-", line 168, in getobject return ((instance)) File "/Users/francois/Library/Caches/pypoetry/virtualenvs/exportbug-PGt-cwXF-py3.9/lib/python3.9/site-", line 496, in get raise ( Author matching query does not exist. the model: from import models class AuthorManager(models.Manager): def getbynaturalkey(self, name): return (name=name) class Author(models.Model): id = models.AutoField(primarykey=True) name = models.CharField(maxlength=255, unique=True) objects = AuthorManager() def naturalkey(self): return (,) def str(self): return f"{} {}" class BookManager(models.Manager): def getbynaturalkey(self, title, author): # OR title, author ?? return (title=title, authorname=author) class Book(models.Model): id = models.AutoField(primarykey=True) title = models.CharField(maxlength=255) author = models.ForeignKey(Author, models.DONOTHING, relatedname="books") objects = BookManager() def naturalkey(self): return (,) + () naturalkey.dependencies = ["testbug.Author"] class Meta: uniquetogether = [["title", "author"]] def str_(self): return f"{}: '{}' by {}" the data (generated with from import serializers; from import Book, Author; print(serializers.serialize("json", list(()) + list(()), indent=2, usenaturalforeignkeys=True, usenaturalprimary_keys=True)) in the shell): [ { "model": "", "fields": { "name": "JR Tolkien" } }, { "model": "", "fields": { "title": "The Ring", "author": [ "JR Tolkien" ] } } ]

and hasattr(default_manager, "get_by_natural_key")         and hasattr(Model, "natural_key")     ):        natural_key = Model(**data).natural_key()        obj = Model(**data)        obj._state.db = db        natural_key = obj.natural_key()         try:             data[Model._meta.pk.attname] = Model._meta.pk.to_python(                 default_manager.db_manager(db).get_by_natural_key(*natural_key).pk


Question: viewonsite redirect does not work for custom admin site. Description All reverse calls in have a currentapp keyword argument, so they resolve with the current (possibly custom) admin site – except the reverse call in getviewonsiteurl. This lead to custom admin sites using the default admin to redirect via `viewon_site. This is clearly a bug. When the default admin is handled with extra protections outside of Django (e.g. additional basic auth), users of the custom admin cannot use the "view on site" link because it routes through the default admin. Patch is easy and already here:  I'll try to provide a regression test as well.

"content_type_id": get_content_type_for_model(obj).pk,                     "object_id": obj.pk,                 },<ins>                current_app=self.admin_site.name,</ins>             )      def get_empty_value_display(self):


Question: Using multiple FilteredRelation with different filters but for same relation is ignored. Description (last modified by lind-marcus) I have a relation that ALWAYS have at least 1 entry with isall=True and then I have an optional entry that could have isall=False but instead have zone set. I'm trying to use FilteredRelation together with Case(When()) to ensure that it use the zone level one (if exist) and fall back on "All" if zone do not exist. from import FilteredRelation ( relationzone=FilteredRelation( "myrelationnested", condition=Q(myrelationnestedzone=F("zone")) ), relationall=FilteredRelation( "myrelationnested", condition=Q(myrelationnestedisall=True) ), pricezone=F("relationzoneprice") ).annotate( pricefinal=Case( When( pricezoneisnull=True, then=F("relationallprice"), ), default=F("pricezone") ) ) I noticed that when using multiple FilteredRelation with the same relation (myrelationnested) it actually just generates a single SQL JOIN (if inspecting the raw SQL) and ignores the other. So in this case if I do print(str()) I would only see a join for relationzone. Not for relation_all. Is this intended behavior or should I be able to do the thing above?

"""         return len([1 for count in self.alias_refcount.values() if count])     def join(self, join, reuse=None):    def join(self, join, reuse=None, reuse_with_filtered_relation=False):         """         Return an alias for the 'join', either reusing an existing alias for         that join or creating a new one. 'join' is either a base_table_class or


Question: Using multiple FilteredRelation with different filters but for same relation is ignored. Description (last modified by lind-marcus) I have a relation that ALWAYS have at least 1 entry with isall=True and then I have an optional entry that could have isall=False but instead have zone set. I'm trying to use FilteredRelation together with Case(When()) to ensure that it use the zone level one (if exist) and fall back on "All" if zone do not exist. from import FilteredRelation ( relationzone=FilteredRelation( "myrelationnested", condition=Q(myrelationnestedzone=F("zone")) ), relationall=FilteredRelation( "myrelationnested", condition=Q(myrelationnestedisall=True) ), pricezone=F("relationzoneprice") ).annotate( pricefinal=Case( When( pricezoneisnull=True, then=F("relationallprice"), ), default=F("pricezone") ) ) I noticed that when using multiple FilteredRelation with the same relation (myrelationnested) it actually just generates a single SQL JOIN (if inspecting the raw SQL) and ignores the other. So in this case if I do print(str()) I would only see a join for relationzone. Not for relation_all. Is this intended behavior or should I be able to do the thing above?

The 'reuse' parameter can be either None which means all joins are         reusable, or it can be a set containing the aliases that can be reused.         The 'reuse_with_filtered_relation' parameter is used when computing        FilteredRelation instances.         A join is always created as LOUTER if the lhs alias is LOUTER to make         sure chains like t1 LOUTER t2 INNER t3 aren't generated. All new         joins are created as LOUTER if the join is nullable.         """        reuse_aliases = [            a            for a, j in self.alias_map.items()            if (reuse is None or a in reuse) and j.equals(join)        ]        if reuse_with_filtered_relation and reuse:            reuse_aliases = [                a for a, j in self.alias_map.items() if a in reuse and j.equals(join)            ]        else:            reuse_aliases = [                a                for a, j in self.alias_map.items()                if (reuse is None or a in reuse) and j == join            ]         if reuse_aliases:             if join.table_alias in reuse_aliases:                 reuse_alias = join.table_alias


Question: Using multiple FilteredRelation with different filters but for same relation is ignored. Description (last modified by lind-marcus) I have a relation that ALWAYS have at least 1 entry with isall=True and then I have an optional entry that could have isall=False but instead have zone set. I'm trying to use FilteredRelation together with Case(When()) to ensure that it use the zone level one (if exist) and fall back on "All" if zone do not exist. from import FilteredRelation ( relationzone=FilteredRelation( "myrelationnested", condition=Q(myrelationnestedzone=F("zone")) ), relationall=FilteredRelation( "myrelationnested", condition=Q(myrelationnestedisall=True) ), pricezone=F("relationzoneprice") ).annotate( pricefinal=Case( When( pricezoneisnull=True, then=F("relationallprice"), ), default=F("pricezone") ) ) I noticed that when using multiple FilteredRelation with the same relation (myrelationnested) it actually just generates a single SQL JOIN (if inspecting the raw SQL) and ignores the other. So in this case if I do print(str()) I would only see a join for relationzone. Not for relation_all. Is this intended behavior or should I be able to do the thing above?

can_reuse=None,         allow_joins=True,         split_subq=True,<ins>        reuse_with_filtered_relation=False,</ins>         check_filterable=True,     ):         """


Question: Using multiple FilteredRelation with different filters but for same relation is ignored. Description (last modified by lind-marcus) I have a relation that ALWAYS have at least 1 entry with isall=True and then I have an optional entry that could have isall=False but instead have zone set. I'm trying to use FilteredRelation together with Case(When()) to ensure that it use the zone level one (if exist) and fall back on "All" if zone do not exist. from import FilteredRelation ( relationzone=FilteredRelation( "myrelationnested", condition=Q(myrelationnestedzone=F("zone")) ), relationall=FilteredRelation( "myrelationnested", condition=Q(myrelationnestedisall=True) ), pricezone=F("relationzoneprice") ).annotate( pricefinal=Case( When( pricezoneisnull=True, then=F("relationallprice"), ), default=F("pricezone") ) ) I noticed that when using multiple FilteredRelation with the same relation (myrelationnested) it actually just generates a single SQL JOIN (if inspecting the raw SQL) and ignores the other. So in this case if I do print(str()) I would only see a join for relationzone. Not for relation_all. Is this intended behavior or should I be able to do the thing above?

The 'can_reuse' is a set of reusable joins for multijoins. <ins>        If 'reuse_with_filtered_relation' is True, then only joins in can_reuse        will be reused.</ins>         The method will create a filter clause that can be added to the current         query. However, if the filter isn't added to the query then the caller         is responsible for unreffing the joins used.


Question: Using multiple FilteredRelation with different filters but for same relation is ignored. Description (last modified by lind-marcus) I have a relation that ALWAYS have at least 1 entry with isall=True and then I have an optional entry that could have isall=False but instead have zone set. I'm trying to use FilteredRelation together with Case(When()) to ensure that it use the zone level one (if exist) and fall back on "All" if zone do not exist. from import FilteredRelation ( relationzone=FilteredRelation( "myrelationnested", condition=Q(myrelationnestedzone=F("zone")) ), relationall=FilteredRelation( "myrelationnested", condition=Q(myrelationnestedisall=True) ), pricezone=F("relationzoneprice") ).annotate( pricefinal=Case( When( pricezoneisnull=True, then=F("relationallprice"), ), default=F("pricezone") ) ) I noticed that when using multiple FilteredRelation with the same relation (myrelationnested) it actually just generates a single SQL JOIN (if inspecting the raw SQL) and ignores the other. So in this case if I do print(str()) I would only see a join for relationzone. Not for relation_all. Is this intended behavior or should I be able to do the thing above?

alias,                 can_reuse=can_reuse,                 allow_many=allow_many,<ins>                reuse_with_filtered_relation=reuse_with_filtered_relation,</ins>             )              # Prevent iterator from being consumed by check_related_objects()


Question: Using multiple FilteredRelation with different filters but for same relation is ignored. Description (last modified by lind-marcus) I have a relation that ALWAYS have at least 1 entry with isall=True and then I have an optional entry that could have isall=False but instead have zone set. I'm trying to use FilteredRelation together with Case(When()) to ensure that it use the zone level one (if exist) and fall back on "All" if zone do not exist. from import FilteredRelation ( relationzone=FilteredRelation( "myrelationnested", condition=Q(myrelationnestedzone=F("zone")) ), relationall=FilteredRelation( "myrelationnested", condition=Q(myrelationnestedisall=True) ), pricezone=F("relationzoneprice") ).annotate( pricefinal=Case( When( pricezoneisnull=True, then=F("relationallprice"), ), default=F("pricezone") ) ) I noticed that when using multiple FilteredRelation with the same relation (myrelationnested) it actually just generates a single SQL JOIN (if inspecting the raw SQL) and ignores the other. So in this case if I do print(str()) I would only see a join for relationzone. Not for relation_all. Is this intended behavior or should I be able to do the thing above?

current_negated=current_negated,                     allow_joins=True,                     split_subq=False,<ins>                    reuse_with_filtered_relation=True,</ins>                 )             target_clause.add(child_clause, connector)         return target_clause


Question: Using multiple FilteredRelation with different filters but for same relation is ignored. Description (last modified by lind-marcus) I have a relation that ALWAYS have at least 1 entry with isall=True and then I have an optional entry that could have isall=False but instead have zone set. I'm trying to use FilteredRelation together with Case(When()) to ensure that it use the zone level one (if exist) and fall back on "All" if zone do not exist. from import FilteredRelation ( relationzone=FilteredRelation( "myrelationnested", condition=Q(myrelationnestedzone=F("zone")) ), relationall=FilteredRelation( "myrelationnested", condition=Q(myrelationnestedisall=True) ), pricezone=F("relationzoneprice") ).annotate( pricefinal=Case( When( pricezoneisnull=True, then=F("relationallprice"), ), default=F("pricezone") ) ) I noticed that when using multiple FilteredRelation with the same relation (myrelationnested) it actually just generates a single SQL JOIN (if inspecting the raw SQL) and ignores the other. So in this case if I do print(str()) I would only see a join for relationzone. Not for relation_all. Is this intended behavior or should I be able to do the thing above?

break         return path, final_field, targets, names[pos + 1 :]     def setup_joins(self, names, opts, alias, can_reuse=None, allow_many=True):    def setup_joins(        self,        names,        opts,        alias,        can_reuse=None,        allow_many=True,        reuse_with_filtered_relation=False,    ):         """         Compute the necessary table joins for the passage through the fields         given in 'names'. 'opts' is the Options class for the current model


Question: Using multiple FilteredRelation with different filters but for same relation is ignored. Description (last modified by lind-marcus) I have a relation that ALWAYS have at least 1 entry with isall=True and then I have an optional entry that could have isall=False but instead have zone set. I'm trying to use FilteredRelation together with Case(When()) to ensure that it use the zone level one (if exist) and fall back on "All" if zone do not exist. from import FilteredRelation ( relationzone=FilteredRelation( "myrelationnested", condition=Q(myrelationnestedzone=F("zone")) ), relationall=FilteredRelation( "myrelationnested", condition=Q(myrelationnestedisall=True) ), pricezone=F("relationzoneprice") ).annotate( pricefinal=Case( When( pricezoneisnull=True, then=F("relationallprice"), ), default=F("pricezone") ) ) I noticed that when using multiple FilteredRelation with the same relation (myrelationnested) it actually just generates a single SQL JOIN (if inspecting the raw SQL) and ignores the other. So in this case if I do print(str()) I would only see a join for relationzone. Not for relation_all. Is this intended behavior or should I be able to do the thing above?

that can be reused. Note that non-reverse foreign keys are always         reusable when using setup_joins(). <ins>        The 'reuse_with_filtered_relation' can be used to force 'can_reuse'        parameter and force the relation on the given connections.</ins>         If 'allow_many' is False, then any reverse foreign key seen will         generate a MultiJoin exception.


Question: Using multiple FilteredRelation with different filters but for same relation is ignored. Description (last modified by lind-marcus) I have a relation that ALWAYS have at least 1 entry with isall=True and then I have an optional entry that could have isall=False but instead have zone set. I'm trying to use FilteredRelation together with Case(When()) to ensure that it use the zone level one (if exist) and fall back on "All" if zone do not exist. from import FilteredRelation ( relationzone=FilteredRelation( "myrelationnested", condition=Q(myrelationnestedzone=F("zone")) ), relationall=FilteredRelation( "myrelationnested", condition=Q(myrelationnestedisall=True) ), pricezone=F("relationzoneprice") ).annotate( pricefinal=Case( When( pricezoneisnull=True, then=F("relationallprice"), ), default=F("pricezone") ) ) I noticed that when using multiple FilteredRelation with the same relation (myrelationnested) it actually just generates a single SQL JOIN (if inspecting the raw SQL) and ignores the other. So in this case if I do print(str()) I would only see a join for relationzone. Not for relation_all. Is this intended behavior or should I be able to do the thing above?

nullable,                 filtered_relation=filtered_relation,             )            reuse = can_reuse if join.m2m else None            alias = self.join(connection, reuse=reuse)            reuse = can_reuse if join.m2m or reuse_with_filtered_relation else None            alias = self.join(                connection,                reuse=reuse,                reuse_with_filtered_relation=reuse_with_filtered_relation,            )             joins.append(alias)             if filtered_relation:                 filtered_relation.path = joins[:]


Question: createsuperuser does not validate usernames that use a UniqueConstraint. Description With a custom User model that uses a UniqueConstraint instead of unique=True, the createsuperuser command does not validate usernames at all. class CustomUser(AbstractBaseUser): customusername = models.CharField(maxlength=255) USERNAMEFIELD = 'customusername' class Meta: constraints = [ models.UniqueConstraint(fields=['customusername'], name='uniqueusernames'), ] Running createsuperuser to create a user with a username that already exists then results in an IntegrityError: IntegrityError: duplicate key value violates unique constraint "uniqueusernames" DETAIL: Key (customusername)=(foo) already exists.

from django.core import exceptions from django.core.management.base import BaseCommand, CommandError from django.db import DEFAULT_DB_ALIAS<ins>from django.utils.functional import cached_property</ins> from django.utils.text import capfirst


Question: createsuperuser does not validate usernames that use a UniqueConstraint. Description With a custom User model that uses a UniqueConstraint instead of unique=True, the createsuperuser command does not validate usernames at all. class CustomUser(AbstractBaseUser): customusername = models.CharField(maxlength=255) USERNAMEFIELD = 'customusername' class Meta: constraints = [ models.UniqueConstraint(fields=['customusername'], name='uniqueusernames'), ] Running createsuperuser to create a user with a username that already exists then results in an IntegrityError: IntegrityError: duplicate key value violates unique constraint "uniqueusernames" DETAIL: Key (customusername)=(foo) already exists.

else "",         )     @cached_property    def username_is_unique(self):        if self.username_field.unique:            return True        for unique_constraint in self.UserModel._meta.total_unique_constraints:            if (                len(unique_constraint.fields) == 1                and unique_constraint.fields[0] == self.username_field.name            ):                return True        return False     def _validate_username(self, username, verbose_field_name, database):         """Validate username. If invalid, return a string error message."""        if self.username_field.unique:        if self.username_is_unique:             try:                 self.UserModel._default_manager.db_manager(database).get_by_natural_key(                     username


Question: AlterField operation should be noop when adding/changing choices on SQLite. Description while writing a test case for i found that for sqlite, even a seemingly db-transparent change like adding choices still generates sql (new table + insert + drop + rename) even though this shouldn't be needed. on e.g. postgres the same migration generates no sql

# - changing only a field name         # - changing an attribute that doesn't affect the schema         # - adding only a db_column and the column name is not changed        non_database_attrs = [            "blank",            "db_column",            "editable",            "error_messages",            "help_text",            "limit_choices_to",            # Database-level options are not supported, see #21961.            "on_delete",            "related_name",            "related_query_name",            "validators",            "verbose_name",        ]        for attr in non_database_attrs:        for attr in old_field.non_db_attrs:             old_kwargs.pop(attr, None)        for attr in new_field.non_db_attrs:             new_kwargs.pop(attr, None)         return self.quote_name(old_field.column) != self.quote_name(             new_field.columndiff --git a/django/db/models/fields/__init__.py b/django/db/models/fields/__init__.py-- a/django/db/models/fields/__init__.py++ b/django/db/models/fields/__init__.py


Question: AlterField operation should be noop when adding/changing choices on SQLite. Description while writing a test case for i found that for sqlite, even a seemingly db-transparent change like adding choices still generates sql (new table + insert + drop + rename) even though this shouldn't be needed. on e.g. postgres the same migration generates no sql

system_check_deprecated_details = None     system_check_removed_details = None <ins>    # Attributes that don't affect a column definition.    # These attributes are ignored when altering the field.    non_db_attrs = (        "blank",        "choices",        "db_column",        "editable",        "error_messages",        "help_text",        "limit_choices_to",        # Database-level options are not supported, see #21961.        "on_delete",        "related_name",        "related_query_name",        "validators",        "verbose_name",    )</ins>     # Field flags     hidden = False


Question: Wrong behavior on queryset update when multiple inheritance Description Queryset update has a wrong behavior when queryset class inherits multiple classes. The update happens not on child class but on other parents class instances. Here an easy example to show the problem: class Base(models.Model): baseid = models.AutoField(primarykey=True) fieldbase = models.IntegerField() class OtherBase(models.Model): otherbaseid = models.AutoField(primarykey=True) fieldotherbase = models.IntegerField() class Child(Base, OtherBase): pass Then in django shell: In [1]: (fieldotherbase=100) <QuerySet [{'otherbaseid': 1, 'fieldotherbase': 100}]In [2]: (fieldotherbase=101) <QuerySet [{'otherbaseid': 2, 'fieldotherbase': 101}]In [3]: (fieldbase=0, fieldotherbase=0) <Child: Child object (1)In [4]: (fieldbase=1, fieldotherbase=1) <Child: Child object (2)In [5]: (fieldotherbase=55) SELECT "applianceschild"."baseptrid" FROM "applianceschild" Execution time: 0.000647s [Database: default] UPDATE "appliancesotherbase" SET "fieldotherbase" = 55 WHERE "appliancesotherbase"."otherbaseid" IN (1, 2) Execution time: 0.001414s [Database: default] Out[5]: 2 In [6]: ('fieldotherbase') <QuerySet [{'fieldotherbase': 0}, {'fieldotherbase': 1}]In [7]: (otherbaseidin=[1,2]).values('fieldotherbase') <QuerySet [{'fieldotherbase': 55}, {'fieldotherbase': 55}]As seen on the above code, updating Child fields from second parent has no effect. Worse is that OtherBase fields where modifed because query is using primiary keys from Base class.

query.clear_ordering(force=True)         query.extra = {}         query.select = []        query.add_fields([query.get_meta().pk.name])        meta = query.get_meta()        fields = [meta.pk.name]        related_ids_index = []        for related in self.query.related_updates:            if all(                path.join_field.primary_key for path in meta.get_path_to_parent(related)            ):                # If a primary key chain exists to the targeted related update,                # then the meta.pk value can be used for it.                related_ids_index.append((related, 0))            else:                # This branch will only be reached when updating a field of an                # ancestor that is not part of the primary key chain of a MTI                # tree.                related_ids_index.append((related, len(fields)))                fields.append(related._meta.pk.name)        query.add_fields(fields)         super().pre_sql_setup()          must_pre_select = (