Question: Filter by window expression should raise a descriptive error. Description Django has a check that filter does not contain window expressions. But it is shallow, neither right side of the expression nor combined expressions are checked. class Employee(models.Model): grade = models.IntegerField() Employee.objects.annotate( prevgrade=Window(expression=Lag('grade')) ).filter(prevgrade=F('grade')) Employee.objects.annotate( prevgrade=Window(expression=Lag('grade')) ).filter(grade=F('prevgrade')) Employee.objects.annotate( prevgrade=Window(expression=Lag('grade')), decgrade=F('prevgrade') - Value(1) ).filter(decgrade=F('grade'))

for v in value:                     self.check_query_object_type(v, opts, field) <ins>    def check_filterable(self, expression):        """Raise an error if expression cannot be used in a WHERE clause."""        if not getattr(expression, 'filterable', 'True'):            raise NotSupportedError(                expression.__class__.__name__ + ' is disallowed in the filter '                'clause.'            )        if hasattr(expression, 'get_source_expressions'):            for expr in expression.get_source_expressions():                self.check_filterable(expr)</ins>     def build_lookup(self, lookups, lhs, rhs):         """         Try to extract transforms and lookup from given lhs.


Question: Filter by window expression should raise a descriptive error. Description Django has a check that filter does not contain window expressions. But it is shallow, neither right side of the expression nor combined expressions are checked. class Employee(models.Model): grade = models.IntegerField() Employee.objects.annotate( prevgrade=Window(expression=Lag('grade')) ).filter(prevgrade=F('grade')) Employee.objects.annotate( prevgrade=Window(expression=Lag('grade')) ).filter(grade=F('prevgrade')) Employee.objects.annotate( prevgrade=Window(expression=Lag('grade')), decgrade=F('prevgrade') - Value(1) ).filter(decgrade=F('grade'))

raise FieldError("Cannot parse keyword query %r" % arg)         lookups, parts, reffed_expression = self.solve_lookup_type(arg)         if not getattr(reffed_expression, 'filterable', True):            raise NotSupportedError(                reffed_expression.__class__.__name__ + ' is disallowed in '                'the filter clause.'            )        self.check_filterable(reffed_expression)          if not allow_joins and len(parts) > 1:             raise FieldError("Joined field references are not permitted in this query")


Question: Filter by window expression should raise a descriptive error. Description Django has a check that filter does not contain window expressions. But it is shallow, neither right side of the expression nor combined expressions are checked. class Employee(models.Model): grade = models.IntegerField() Employee.objects.annotate( prevgrade=Window(expression=Lag('grade')) ).filter(prevgrade=F('grade')) Employee.objects.annotate( prevgrade=Window(expression=Lag('grade')) ).filter(grade=F('prevgrade')) Employee.objects.annotate( prevgrade=Window(expression=Lag('grade')), decgrade=F('prevgrade') - Value(1) ).filter(decgrade=F('grade'))

value = self.resolve_lookup_value(value, can_reuse, allow_joins, simple_col)         used_joins = {k for k, v in self.alias_refcount.items() if v > pre_joins.get(k, 0)} <ins>        self.check_filterable(value)</ins>         clause = self.where_class()         if reffed_expression:             condition = self.build_lookup(lookups, reffed_expression, value)


Question: SQLite3 migrations can fail when used quoted dbtable. Description (last modified by Maciej Olko) If model's Meta dbtable is quoted, e.g. '"tablewithquotedname"', SQLite3 migration with this table creation with can fail with near "tablewithquotedname": syntax error. I suppose following generated query causes the error: CREATE TABLE "new"tablewithquotedname"" ("objid" integer NOT NULL PRIMARY KEY, "objnum" varchar(20) NULL, "countryid" integer NOT NULL REFERENCES "countries" ("countryid") DEFERRABLE INITIALLY DEFERRED) To reproduce table with quoted name should have at least one foreign key. Django documentation says it supports quoted names (). Quoted names can also be used with Django’s other supported database backends; except for Oracle, however, the quotes have no effect. Traceback: Traceback (most recent call last): File "…", line 82, in execute return (sql) File "…", line 382, in execute return Database.Cursor.execute(self, query) sqlite3.OperationalError: near "tablewithquotedname": syntax error The above exception was the direct cause of the following exception: Traceback (most recent call last): File "", line 15, in <moduleexecutefromcommandline() … File "…", line 234, in handle fakeinitial=fakeinitial, File "…", line 117, in migrate state = self.migrateallforwards(state, plan, fullplan, fake=fake, fakeinitial=fakeinitial) File "…", line 147, in migrateallforwards state = self.applymigration(state, migration, fake=fake, fakeinitial=fakeinitial) File "…", line 245, in applymigration state = (state, schemaeditor) File "…", line 124, in apply operation.databaseforwards(self.applabel, schemaeditor, oldstate, projectstate) File "…", line 112, in databaseforwards field, File "…", line 327, in addfield self.remaketable(model, createfield=field) File "…", line 279, in remaketable self.createmodel(newmodel) File "…", line 307, in createmodel self.execute(sql, params or None) File "…", line 137, in execute cursor.execute(sql, params) File "…", line 99, in execute return super().execute(sql, params) File "…", line 67, in execute return self.executewithwrappers(sql, params, many=False, executor=self.execute) File "…", line 76, in executewithwrappers return executor(sql, params, many, context) File "…", line 84, in execute return (sql, params) File "…", line 89, in exit raise djexcvalue.withtraceback(traceback) from excvalue File "…", line 82, in execute return (sql) File "…", line 382, in execute return Database.Cursor.execute(self, query) near "tablewithquoted_name": syntax error

from django.db.backends.base.schema import BaseDatabaseSchemaEditor from django.db.backends.ddl_references import IndexColumns<ins>from django.db.backends.utils import strip_quotes</ins>   class DatabaseSchemaEditor(BaseDatabaseSchemaEditor):


Question: SQLite3 migrations can fail when used quoted dbtable. Description (last modified by Maciej Olko) If model's Meta dbtable is quoted, e.g. '"tablewithquotedname"', SQLite3 migration with this table creation with can fail with near "tablewithquotedname": syntax error. I suppose following generated query causes the error: CREATE TABLE "new"tablewithquotedname"" ("objid" integer NOT NULL PRIMARY KEY, "objnum" varchar(20) NULL, "countryid" integer NOT NULL REFERENCES "countries" ("countryid") DEFERRABLE INITIALLY DEFERRED) To reproduce table with quoted name should have at least one foreign key. Django documentation says it supports quoted names (). Quoted names can also be used with Django’s other supported database backends; except for Oracle, however, the quotes have no effect. Traceback: Traceback (most recent call last): File "…", line 82, in execute return (sql) File "…", line 382, in execute return Database.Cursor.execute(self, query) sqlite3.OperationalError: near "tablewithquotedname": syntax error The above exception was the direct cause of the following exception: Traceback (most recent call last): File "", line 15, in <moduleexecutefromcommandline() … File "…", line 234, in handle fakeinitial=fakeinitial, File "…", line 117, in migrate state = self.migrateallforwards(state, plan, fullplan, fake=fake, fakeinitial=fakeinitial) File "…", line 147, in migrateallforwards state = self.applymigration(state, migration, fake=fake, fakeinitial=fakeinitial) File "…", line 245, in applymigration state = (state, schemaeditor) File "…", line 124, in apply operation.databaseforwards(self.applabel, schemaeditor, oldstate, projectstate) File "…", line 112, in databaseforwards field, File "…", line 327, in addfield self.remaketable(model, createfield=field) File "…", line 279, in remaketable self.createmodel(newmodel) File "…", line 307, in createmodel self.execute(sql, params or None) File "…", line 137, in execute cursor.execute(sql, params) File "…", line 99, in execute return super().execute(sql, params) File "…", line 67, in execute return self.executewithwrappers(sql, params, many=False, executor=self.execute) File "…", line 76, in executewithwrappers return executor(sql, params, many, context) File "…", line 84, in execute return (sql, params) File "…", line 89, in exit raise djexcvalue.withtraceback(traceback) from excvalue File "…", line 82, in execute return (sql) File "…", line 382, in execute return Database.Cursor.execute(self, query) near "tablewithquoted_name": syntax error

if self._field_data_type(old_field) != self._field_data_type(new_field):             self.sql_alter_column_type += ' USING %(column)s::%(type)s'         # Make ALTER TYPE with SERIAL make sense.        table = model._meta.db_table        table = strip_quotes(model._meta.db_table)         if new_type.lower() in ("serial", "bigserial"):            column = new_field.column            column = strip_quotes(new_field.column)             sequence_name = "%s_%s_seq" % (table, column)             col_type = "integer" if new_type.lower() == "serial" else "bigint"             return (diff --git a/django/db/backends/sqlite3/schema.py b/django/db/backends/sqlite3/schema.py-- a/django/db/backends/sqlite3/schema.py++ b/django/db/backends/sqlite3/schema.py


Question: SQLite3 migrations can fail when used quoted dbtable. Description (last modified by Maciej Olko) If model's Meta dbtable is quoted, e.g. '"tablewithquotedname"', SQLite3 migration with this table creation with can fail with near "tablewithquotedname": syntax error. I suppose following generated query causes the error: CREATE TABLE "new"tablewithquotedname"" ("objid" integer NOT NULL PRIMARY KEY, "objnum" varchar(20) NULL, "countryid" integer NOT NULL REFERENCES "countries" ("countryid") DEFERRABLE INITIALLY DEFERRED) To reproduce table with quoted name should have at least one foreign key. Django documentation says it supports quoted names (). Quoted names can also be used with Django’s other supported database backends; except for Oracle, however, the quotes have no effect. Traceback: Traceback (most recent call last): File "…", line 82, in execute return (sql) File "…", line 382, in execute return Database.Cursor.execute(self, query) sqlite3.OperationalError: near "tablewithquotedname": syntax error The above exception was the direct cause of the following exception: Traceback (most recent call last): File "", line 15, in <moduleexecutefromcommandline() … File "…", line 234, in handle fakeinitial=fakeinitial, File "…", line 117, in migrate state = self.migrateallforwards(state, plan, fullplan, fake=fake, fakeinitial=fakeinitial) File "…", line 147, in migrateallforwards state = self.applymigration(state, migration, fake=fake, fakeinitial=fakeinitial) File "…", line 245, in applymigration state = (state, schemaeditor) File "…", line 124, in apply operation.databaseforwards(self.applabel, schemaeditor, oldstate, projectstate) File "…", line 112, in databaseforwards field, File "…", line 327, in addfield self.remaketable(model, createfield=field) File "…", line 279, in remaketable self.createmodel(newmodel) File "…", line 307, in createmodel self.execute(sql, params or None) File "…", line 137, in execute cursor.execute(sql, params) File "…", line 99, in execute return super().execute(sql, params) File "…", line 67, in execute return self.executewithwrappers(sql, params, many=False, executor=self.execute) File "…", line 76, in executewithwrappers return executor(sql, params, many, context) File "…", line 84, in execute return (sql, params) File "…", line 89, in exit raise djexcvalue.withtraceback(traceback) from excvalue File "…", line 82, in execute return (sql) File "…", line 382, in execute return Database.Cursor.execute(self, query) near "tablewithquoted_name": syntax error

from django.apps.registry import Apps from django.db.backends.base.schema import BaseDatabaseSchemaEditor from django.db.backends.ddl_references import Statement<ins>from django.db.backends.utils import strip_quotes</ins> from django.db.models import UniqueConstraint from django.db.transaction import atomic from django.db.utils import NotSupportedError


Question: SQLite3 migrations can fail when used quoted dbtable. Description (last modified by Maciej Olko) If model's Meta dbtable is quoted, e.g. '"tablewithquotedname"', SQLite3 migration with this table creation with can fail with near "tablewithquotedname": syntax error. I suppose following generated query causes the error: CREATE TABLE "new"tablewithquotedname"" ("objid" integer NOT NULL PRIMARY KEY, "objnum" varchar(20) NULL, "countryid" integer NOT NULL REFERENCES "countries" ("countryid") DEFERRABLE INITIALLY DEFERRED) To reproduce table with quoted name should have at least one foreign key. Django documentation says it supports quoted names (). Quoted names can also be used with Django’s other supported database backends; except for Oracle, however, the quotes have no effect. Traceback: Traceback (most recent call last): File "…", line 82, in execute return (sql) File "…", line 382, in execute return Database.Cursor.execute(self, query) sqlite3.OperationalError: near "tablewithquotedname": syntax error The above exception was the direct cause of the following exception: Traceback (most recent call last): File "", line 15, in <moduleexecutefromcommandline() … File "…", line 234, in handle fakeinitial=fakeinitial, File "…", line 117, in migrate state = self.migrateallforwards(state, plan, fullplan, fake=fake, fakeinitial=fakeinitial) File "…", line 147, in migrateallforwards state = self.applymigration(state, migration, fake=fake, fakeinitial=fakeinitial) File "…", line 245, in applymigration state = (state, schemaeditor) File "…", line 124, in apply operation.databaseforwards(self.applabel, schemaeditor, oldstate, projectstate) File "…", line 112, in databaseforwards field, File "…", line 327, in addfield self.remaketable(model, createfield=field) File "…", line 279, in remaketable self.createmodel(newmodel) File "…", line 307, in createmodel self.execute(sql, params or None) File "…", line 137, in execute cursor.execute(sql, params) File "…", line 99, in execute return super().execute(sql, params) File "…", line 67, in execute return self.executewithwrappers(sql, params, many=False, executor=self.execute) File "…", line 76, in executewithwrappers return executor(sql, params, many, context) File "…", line 84, in execute return (sql, params) File "…", line 89, in exit raise djexcvalue.withtraceback(traceback) from excvalue File "…", line 82, in execute return (sql) File "…", line 382, in execute return Database.Cursor.execute(self, query) near "tablewithquoted_name": syntax error

body_copy = copy.deepcopy(body)         meta_contents = {             'app_label': model._meta.app_label,            'db_table': 'new__%s' % model._meta.db_table,            'db_table': 'new__%s' % strip_quotes(model._meta.db_table),             'unique_together': unique_together,             'index_together': index_together,             'indexes': indexes,


Question: Cloaking PermissionErrors raised in ManifestFilesMixin.readmanifest(). Description While using the ManifestStaticFilesStorage, I encountered the ValueError shown below. <trimFile "/<some venv/site-", line 134, in url hashedname = hashednamefunc(*args) File "/<some venv/site-", line 422, in storedname raise ValueError("Missing staticfiles manifest entry for '%s'" % cleanname) ValueError: Missing staticfiles manifest entry for '' Further debugging showed that staticfilestorage.hashedfiles was empty. This was odd to me because existed on my system in its proper location in STATICROOT. Additionally, the '' existed as a key in the so I was very confused. I did more debugging of the manifest loading process and hit the root problem. Reading the manifest file raised a PermissionError because the user account running the Django app did not have proper permission to read from where the file was stored on the filesystem. Unfortunately,  catches any OSError and proceeds by returning no content. This was in a Vagrant virtual machine so I switched to the root user to bypass any permission errors and the manifest loading worked. I confirmed that I was able to load a page without any issue. I'd like to suggest that ManifestFilesMixin.read_manifest either a) not catch OSError at all to let the system fail so a developer can fix it, b) have more granular error handling of different OSError exception subclasses, or c) do some kind of logging to hint to the app developer that there was a problem. I'm making this suggestion because catching this error and proceeding left the app in an unrecoverable state. The ValueError listed at the beginning of the issue was actually a side effect of the deeper underlying PermissionError problem from attempting to read the manifest file. If there is any more info you need from me, please let me know. Thanks!

try:             with self.open(self.manifest_name) as manifest:                 return manifest.read().decode()        except OSError:        except FileNotFoundError:             return None      def load_manifest(self):


Question: When DEBUG is True, raising Http404 in a path converter's topython method does not result in a technical response Description This is the response I get (plain text): A server error occurred. Please contact the administrator. I understand a ValueError should be raised which tells the URL resolver "this path does not match, try next one" but Http404 is what came to my mind intuitively and the error message was not very helpful. One could also make a point that raising a Http404 should be valid way to tell the resolver "this is indeed the right path but the current parameter value does not match anything so stop what you are doing and let the handler return the 404 page (including a helpful error message when DEBUG is True instead of the default 'Django tried these URL patterns')". This would prove useful for example to implement a path converter that uses getobjector404.

from pathlib import Path  from django.conf import settingsfrom django.http import HttpResponse, HttpResponseNotFoundfrom django.http import Http404, HttpResponse, HttpResponseNotFound from django.template import Context, Engine, TemplateDoesNotExist from django.template.defaultfilters import pprintfrom django.urls import Resolver404, resolvefrom django.urls import resolve from django.utils import timezone from django.utils.datastructures import MultiValueDict from django.utils.encoding import force_str


Question: When DEBUG is True, raising Http404 in a path converter's topython method does not result in a technical response Description This is the response I get (plain text): A server error occurred. Please contact the administrator. I understand a ValueError should be raised which tells the URL resolver "this path does not match, try next one" but Http404 is what came to my mind intuitively and the error message was not very helpful. One could also make a point that raising a Http404 should be valid way to tell the resolver "this is indeed the right path but the current parameter value does not match anything so stop what you are doing and let the handler return the 404 page (including a helpful error message when DEBUG is True instead of the default 'Django tried these URL patterns')". This would prove useful for example to implement a path converter that uses getobjector404.

caller = ''     try:         resolver_match = resolve(request.path)    except Resolver404:    except Http404:         pass     else:         obj = resolver_match.func


Question: Add a helpful exception for invalid values passed to AutoField/FloatField/IntegerField. Description (last modified by Nick Pope) When a large model is updated and saved with invalid values, Django produces a traceback deep within the ORM, with no clue which field assignment caused the error. Developers are with: "TypeError: int() argument must be a string, a bytes-like object or a number, not 'tuple'" This change displays the field name which makes spotting errors a lot easier.

value = super().get_prep_value(value)         if value is None or isinstance(value, OuterRef):             return value        return int(value)        try:            return int(value)        except (TypeError, ValueError) as e:            raise e.__class__(                "Field '%s' expected a number but got %r." % (self.name, value),            ) from e      def contribute_to_class(self, cls, name, **kwargs):         assert not cls._meta.auto_field, "Model %s can't have more than one AutoField." % cls._meta.label


Question: Add a helpful exception for invalid values passed to AutoField/FloatField/IntegerField. Description (last modified by Nick Pope) When a large model is updated and saved with invalid values, Django produces a traceback deep within the ORM, with no clue which field assignment caused the error. Developers are with: "TypeError: int() argument must be a string, a bytes-like object or a number, not 'tuple'" This change displays the field name which makes spotting errors a lot easier.

value = super().get_prep_value(value)         if value is None:             return None        return float(value)        try:            return float(value)        except (TypeError, ValueError) as e:            raise e.__class__(                "Field '%s' expected a number but got %r." % (self.name, value),            ) from e      def get_internal_type(self):         return "FloatField"


Question: Add a helpful exception for invalid values passed to AutoField/FloatField/IntegerField. Description (last modified by Nick Pope) When a large model is updated and saved with invalid values, Django produces a traceback deep within the ORM, with no clue which field assignment caused the error. Developers are with: "TypeError: int() argument must be a string, a bytes-like object or a number, not 'tuple'" This change displays the field name which makes spotting errors a lot easier.

value = super().get_prep_value(value)         if value is None:             return None        return int(value)        try:            return int(value)        except (TypeError, ValueError) as e:            raise e.__class__(                "Field '%s' expected a number but got %r." % (self.name, value),            ) from e      def get_internal_type(self):         return "IntegerField"


Question: Django throws error when different apps with different models have the same name table name. Description Error message: tablename: (models.E028) dbtable 'table_name' is used by multiple models: base.ModelName, app2.ModelName. We have a Base app that points to a central database and that has its own tables. We then have multiple Apps that talk to their own databases. Some share the same table names. We have used this setup for a while, but after upgrading to Django 2.2 we're getting an error saying we're not allowed 2 apps, with 2 different models to have the same table names. Is this correct behavior? We've had to roll back to Django 2.0 for now.

from itertools import chain  from django.apps import appsfrom django.core.checks import Error, Tags, registerfrom django.conf import settingsfrom django.core.checks import Error, Tags, Warning, register   @register(Tags.models)


Question: Django throws error when different apps with different models have the same name table name. Description Error message: tablename: (models.E028) dbtable 'table_name' is used by multiple models: base.ModelName, app2.ModelName. We have a Base app that points to a central database and that has its own tables. We then have multiple Apps that talk to their own databases. Some share the same table names. We have used this setup for a while, but after upgrading to Django 2.2 we're getting an error saying we're not allowed 2 apps, with 2 different models to have the same table names. Is this correct behavior? We've had to roll back to Django 2.0 for now.

indexes[model_index.name].append(model._meta.label)         for model_constraint in model._meta.constraints:             constraints[model_constraint.name].append(model._meta.label)    if settings.DATABASE_ROUTERS:        error_class, error_id = Warning, 'models.W035'        error_hint = (            'You have configured settings.DATABASE_ROUTERS. Verify that %s '            'are correctly routed to separate databases.'        )    else:        error_class, error_id = Error, 'models.E028'        error_hint = None     for db_table, model_labels in db_table_models.items():         if len(model_labels) != 1:            model_labels_str = ', '.join(model_labels)             errors.append(                Error(                error_class(                     "db_table '%s' is used by multiple models: %s."                    % (db_table, ', '.join(db_table_models[db_table])),                    % (db_table, model_labels_str),                     obj=db_table,                    id='models.E028',                    hint=(error_hint % model_labels_str) if error_hint else None,                    id=error_id,                 )             )     for index_name, model_labels in indexes.items():


Question: Improve Exception Message In Test Client and urlencode() when None is passed as data. Description (last modified by Keith Gray) I am upgrading to 2.2.1 from 2.0.5 and my tests are failing due to a change in the test client:  It now throws an exception if a None value is provided in data given to a POST. I would like to propose an improvement to the message to display the offending Key and Value that generate the exception. I have a proposed change in my fork on github:

for key, value in query:         if value is None:             raise TypeError(                'Cannot encode None in a query string. Did you mean to pass '                'an empty string or omit the value?'                "Cannot encode None for key '%s' in a query string. Did you "                "mean to pass an empty string or omit the value?" % key             )         elif not doseq or isinstance(value, (str, bytes)):             query_val = value


Question: Improve Exception Message In Test Client and urlencode() when None is passed as data. Description (last modified by Keith Gray) I am upgrading to 2.2.1 from 2.0.5 and my tests are failing due to a change in the test client:  It now throws an exception if a None value is provided in data given to a POST. I would like to propose an improvement to the message to display the offending Key and Value that generate the exception. I have a proposed change in my fork on github:

for item in itr:                     if item is None:                         raise TypeError(                            'Cannot encode None in a query string. Did you '                            'mean to pass an empty string or omit the value?'                            "Cannot encode None for key '%s' in a query "                            "string. Did you mean to pass an empty string or "                            "omit the value?" % key                         )                     elif not isinstance(item, bytes):                         item = str(item)


Question: Allowing patchvaryheaders() caching utility to handle '' value. Description Function "patchvaryheaders", simply appends new headers to list. If view code sets Vary header to asterisk, the resulting header (after applying SessionMiddleware and LocaleMiddleware) looks like this: Vary: , Accept-Language, Cookie This is unnecessary and possible violates HTTP spec: The "Vary" header field in a response describes what parts of a request message, aside from the method, Host header field, and request target, might influence the origin server's process for selecting and representing this response. The value consists of either a single asterisk ("") or a list of header field names (case-insensitive). Vary = "" / 1#field-name (from ) I am using Django to implement REST API, so I'd like it to speak robust HTTP, that works with all present and future caching libraries, — even if widely used browsers and Nginx can correctly interpret current form of the header.

def patch_vary_headers(response, newheaders):     """     Add (or update) the "Vary" header in the given HttpResponse object.    newheaders is a list of header names that should be in "Vary". Existing    headers in "Vary" aren't removed.    newheaders is a list of header names that should be in "Vary". If headers    contains an asterisk, then "Vary" header will consist of a single asterisk    '*'. Otherwise, existing headers in "Vary" aren't removed.     """     # Note that we need to keep the original order intact, because cache     # implementations may rely on the order of the Vary contents in, say,


Question: Allowing patchvaryheaders() caching utility to handle '' value. Description Function "patchvaryheaders", simply appends new headers to list. If view code sets Vary header to asterisk, the resulting header (after applying SessionMiddleware and LocaleMiddleware) looks like this: Vary: , Accept-Language, Cookie This is unnecessary and possible violates HTTP spec: The "Vary" header field in a response describes what parts of a request message, aside from the method, Host header field, and request target, might influence the origin server's process for selecting and representing this response. The value consists of either a single asterisk ("") or a list of header field names (case-insensitive). Vary = "" / 1#field-name (from ) I am using Django to implement REST API, so I'd like it to speak robust HTTP, that works with all present and future caching libraries, — even if widely used browsers and Nginx can correctly interpret current form of the header.

existing_headers = {header.lower() for header in vary_headers}     additional_headers = [newheader for newheader in newheaders                           if newheader.lower() not in existing_headers]    response['Vary'] = ', '.join(vary_headers + additional_headers)    vary_headers += additional_headers    if '*' in vary_headers:        response['Vary'] = '*'    else:        response['Vary'] = ', '.join(vary_headers)   def has_vary_header(response, header_query):


Question: Stop TemplateView automatically passing kwargs into the context Description Only TemplateView pushes to the context. ListView does not, I yet have to check others. This is inconsistency and, I think, it should be fixed.

import logging<ins>import warnings</ins> from functools import update_wrapper  from django.core.exceptions import ImproperlyConfigured


Question: Stop TemplateView automatically passing kwargs into the context Description Only TemplateView pushes to the context. ListView does not, I yet have to check others. This is inconsistency and, I think, it should be fixed.

from django.template.response import TemplateResponse from django.urls import reverse from django.utils.decorators import classonlymethod<ins>from django.utils.deprecation import RemovedInDjango40Warningfrom django.utils.functional import SimpleLazyObject</ins>  logger = logging.getLogger('django.request')


Question: Stop TemplateView automatically passing kwargs into the context Description Only TemplateView pushes to the context. ListView does not, I yet have to check others. This is inconsistency and, I think, it should be fixed.

class TemplateView(TemplateResponseMixin, ContextMixin, View):    """    Render a template. Pass keyword arguments from the URLconf to the context.    """    """Render a template."""     def get(self, request, *args, **kwargs):        context = self.get_context_data(**kwargs)        # RemovedInDjango40Warning: when the deprecation ends, replace with:        #   context = self.get_context_data()        context_kwargs = _wrap_url_kwargs_with_deprecation_warning(kwargs)        context = self.get_context_data(**context_kwargs)         return self.render_to_response(context)  # RemovedInDjango40Warningdef _wrap_url_kwargs_with_deprecation_warning(url_kwargs):    context_kwargs = {}    for key, value in url_kwargs.items():        # Bind into function closure.        @SimpleLazyObject        def access_value(key=key, value=value):            warnings.warn(                'TemplateView passing URL kwargs to the context is '                'deprecated. Reference %s in your template through '                'view.kwargs instead.' % key,                RemovedInDjango40Warning, stacklevel=2,            )            return value        context_kwargs[key] = access_value    return context_kwargs class RedirectView(View):     """Provide a redirect on any GET request."""     permanent = False


Question: Nested OuterRef not looking on the right model for the field. Description (last modified by Aaron Lisman) I ran into this and made a test case for it. It seems similar to the test case above it. I'm not sure what's different about it and causing it to fail. Let me know if the query is just built wrong. diff --git index .. --- +++ -83,6 +83,28 class SimulationRun(models.Model): return "%s (%s to %s)" % (self.midpoint, , ) +class Customer(models.Model): name = models.TextField() + +class Item(models.Model): pass + +class Invoice(models.Model): INVOICE = 'invoice' EXPENSE = 'expense' + KINDCHOICES = ( (INVOICE, 'Invoice'), (EXPENSE, 'Expense'), ) + kind = models.CharField(choices=KINDCHOICES, maxlength=255, default=None) owner = models.ForeignKey(Customer, models.CASCADE) items = models.ManyToManyField(Item, relatedname='invoices') + class UUIDPK(models.Model): id = models.UUIDField(primarykey=True, default=uuid.uuid4) diff --git index .. --- +++ -24,7 +24,7 from import Approximate from .models import ( UUID, UUIDPK, Company, Employee, Experiment, Number, Result, SimulationRun, Time, Time, Customer, Item, Invoice ) -536,6 +536,55 class BasicExpressionsTests(TestCase): # This is a contrived example. It exercises the double OuterRef form. self.assertCountEqual(outer, [first, second, third]) def testnestedsubqueryouterref3(self): customer = (name='Test Customer') othercustomer = (name='Ohter Customer') + unexpensedinvoice = (kind=Invoice.INVOICE, owner=customer) unexpenseditem1 = () (unexpenseditem1) + partiallyexpensedinvoice = (kind=Invoice.INVOICE, owner=customer) expense1 = (kind=Invoice.EXPENSE, owner=customer) expenseditem1 = () unexpenseditem2 = () (expenseditem1, unexpenseditem2) (expenseditem1) + expensedinvoice = (kind=Invoice.INVOICE, owner=customer) (kind=Invoice.EXPENSE, owner=customer) expenseditem2 = () (expenseditem2) (expenseditem2) + otherinvoice = (kind=Invoice.INVOICE, owner=othercustomer) (unexpenseditem1) otherexpense = (kind=Invoice.EXPENSE, owner=othercustomer) (unexpenseditem1) + inner = ( kind=Invoice.EXPENSE, owner=OuterRef(OuterRef('owner')), items=OuterRef('id'), ) middle = ( invoices=OuterRef('id'), ).annotate( expensed=Exists(inner), ).filter( expensed=False, ) outer = ( kind=Invoice.INVOICE, owner=customer, ).annotate( unexpensed=Exists(middle), ) + self.assertTrue(().unexpensed) self.assertTrue(().unexpensed) self.assertFalse(().unexpensed) def testnestedsubqueryouterrefwithautofield(self): first = (time='09:00') second = (time='17:00') Cannot resolve keyword 'owner' into field. Choices are: expensed, id, invoices

return any(child.is_summary for child in self.children)      @staticmethod    def _resolve_rhs(rhs, query, *args, **kwargs):        if hasattr(rhs, 'resolve_expression'):            rhs = rhs.resolve_expression(query, *args, **kwargs)        return rhs    def _resolve_leaf(expr, query, *args, **kwargs):        if hasattr(expr, 'resolve_expression'):            expr = expr.resolve_expression(query, *args, **kwargs)        return expr      @classmethod     def _resolve_node(cls, node, query, *args, **kwargs):         if hasattr(node, 'children'):             for child in node.children:                 cls._resolve_node(child, query, *args, **kwargs)        if hasattr(node, 'lhs'):            node.lhs = cls._resolve_leaf(node.lhs, query, *args, **kwargs)         if hasattr(node, 'rhs'):            node.rhs = cls._resolve_rhs(node.rhs, query, *args, **kwargs)            node.rhs = cls._resolve_leaf(node.rhs, query, *args, **kwargs)      def resolve_expression(self, *args, **kwargs):         clone = self.clone()


Question: Remove UPDATE query when saving a new model instance with a primary key that has a default Description (last modified by user0007) Using a model's instance: class Account(models.Model): id = models.UUIDField( primarykey=True, default=uuid.uuid4, editable=False ) title = models.TextField() account = Account() = "abc" () "appaccount" SET "title" = \'\', WHERE "appaccount"."id" = \'-150e-419f-b493-\'::uuid', INTO "appaccount" ("title", "id") VALUES (\'abc\', \'-214a-4798-a0fa-\'::uuid) Using a model's manager method: (title="abc") INTO "appaccount" ("title", "id") VALUES (\'abc\', \'-214a-4798-a0fa-\'::uuid) Using a model's instance with forceinsert argument: account = Account() = "abc" (forceinsert=true) INTO "appaccount" ("title", "id") VALUES (\'abc\', \'-214a-4798-a0fa-\'::uuid) Related issue?

DEFAULT_DB_ALIAS, DJANGO_VERSION_PICKLE_KEY, DatabaseError, connection,     connections, router, transaction, )<ins>from django.db.models import NOT_PROVIDED</ins> from django.db.models.constants import LOOKUP_SEP from django.db.models.constraints import CheckConstraint, UniqueConstraint from django.db.models.deletion import CASCADE, Collector


Question: Remove UPDATE query when saving a new model instance with a primary key that has a default Description (last modified by user0007) Using a model's instance: class Account(models.Model): id = models.UUIDField( primarykey=True, default=uuid.uuid4, editable=False ) title = models.TextField() account = Account() = "abc" () "appaccount" SET "title" = \'\', WHERE "appaccount"."id" = \'-150e-419f-b493-\'::uuid', INTO "appaccount" ("title", "id") VALUES (\'abc\', \'-214a-4798-a0fa-\'::uuid) Using a model's manager method: (title="abc") INTO "appaccount" ("title", "id") VALUES (\'abc\', \'-214a-4798-a0fa-\'::uuid) Using a model's instance with forceinsert argument: account = Account() = "abc" (forceinsert=true) INTO "appaccount" ("title", "id") VALUES (\'abc\', \'-214a-4798-a0fa-\'::uuid) Related issue?

if not pk_set and (force_update or update_fields):             raise ValueError("Cannot force an update in save() with no primary key.")         updated = False<ins>        # Skip an UPDATE when adding an instance and primary key has a default.        if (            not force_insert and            self._state.adding and            self._meta.pk.default and            self._meta.pk.default is not NOT_PROVIDED        ):            force_insert = True</ins>         # If possible, try an UPDATE. If that doesn't update anything, do an INSERT.         if pk_set and not force_insert:             base_qs = cls._base_manager.using(using)


Question: path converters don't handle spaces well. Description This came up for someone on IRC last week, but I can't see that they raised a ticket about it. Correct: from import routetoregex routetoregex("<uuid:test") ('^(?P<test[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12})', {'test': at 0x1055e8c88}) Also working correctly: from import routetoregex routetoregex("<uuid:2") ImproperlyConfigured: URL route '<uuid:2' uses parameter name '2' which isn't a valid Python identifier. however, constructing a valid looking converter reference apparently hits neither the happy nor the unhappy path, and also I presume passes any system checks in place that might otherwise warn about the sensitivity: from import routetoregex routetoregex("<uuid: test") # note the preceeding space ('^\<uuid\:\ test\', {}) the regex is invalid the kwargs dictionary is (sort of rightly) empty. the same is true with "test " and "te st" Unless I'm misunderstanding the code therein, "test ", " test" and "te st" should all be hitting the invalid identifier part, and personally I feel like leading/trailing spaces at least could just be sanitised (stripped) as they're almost certainly accidental or for display purposes. Tested in a shell against master @ .

import functools import inspect import re<ins>import string</ins> from importlib import import_module from urllib.parse import quote


Question: path converters don't handle spaces well. Description This came up for someone on IRC last week, but I can't see that they raised a ticket about it. Correct: from import routetoregex routetoregex("<uuid:test") ('^(?P<test[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12})', {'test': at 0x1055e8c88}) Also working correctly: from import routetoregex routetoregex("<uuid:2") ImproperlyConfigured: URL route '<uuid:2' uses parameter name '2' which isn't a valid Python identifier. however, constructing a valid looking converter reference apparently hits neither the happy nor the unhappy path, and also I presume passes any system checks in place that might otherwise warn about the sensitivity: from import routetoregex routetoregex("<uuid: test") # note the preceeding space ('^\<uuid\:\ test\', {}) the regex is invalid the kwargs dictionary is (sort of rightly) empty. the same is true with "test " and "te st" Unless I'm misunderstanding the code therein, "test ", " test" and "te st" should all be hitting the invalid identifier part, and personally I feel like leading/trailing spaces at least could just be sanitised (stripped) as they're almost certainly accidental or for display purposes. Tested in a shell against master @ .

For example, 'foo/<int:pk>' returns '^foo\\/(?P<pk>[0-9]+)'     and {'pk': <django.urls.converters.IntConverter>}.     """<ins>    if not set(route).isdisjoint(string.whitespace):        raise ImproperlyConfigured("URL route '%s' cannot contain whitespace." % route)</ins>     original_route = route     parts = ['^']     converters = {}


Question: Can't use OuterRef in union Subquery Description When you make a QuerySet using the union method or the | operator, the QuerySet passed into the union method cannot reference OuterRef even when wrapped with Subquery. For example: cls = ( checklistisnull=False, part=OuterRef('id') ).values('checklistcustomer', 'created') ots = ( ownershiptransferisnull=False, part=OuterRef('id') ).values('ownershiptransfershiptargetcontact', 'created') return self.annotate( owner=Subquery((ots).values('owner')[:1]) ) Returns this error: ValueError This queryset contains a reference to an outer query and may only be used in a subquery. I get the same error with this statement: return self.annotate( owner=Subquery((cls | ots).values('owner')[:1]) ) (As an aside, I also get an error when I try to apply an orderby clause.)

part_sql = 'SELECT * FROM ({})'.format(part_sql)                     # Add parentheses when combining with compound query if not                     # already added for all compound queries.                    elif not features.supports_slicing_ordering_in_compound:                    elif (                        self.query.subquery or                        not features.supports_slicing_ordering_in_compound                    ):                         part_sql = '({})'.format(part_sql)                 parts += ((part_sql, part_args),)             except EmptyResultSet:


Question: Can't use OuterRef in union Subquery Description When you make a QuerySet using the union method or the | operator, the QuerySet passed into the union method cannot reference OuterRef even when wrapped with Subquery. For example: cls = ( checklistisnull=False, part=OuterRef('id') ).values('checklistcustomer', 'created') ots = ( ownershiptransferisnull=False, part=OuterRef('id') ).values('ownershiptransfershiptargetcontact', 'created') return self.annotate( owner=Subquery((ots).values('owner')[:1]) ) Returns this error: ValueError This queryset contains a reference to an outer query and may only be used in a subquery. I get the same error with this statement: return self.annotate( owner=Subquery((cls | ots).values('owner')[:1]) ) (As an aside, I also get an error when I try to apply an orderby clause.)

combinator_sql = self.connection.ops.set_operators[combinator]         if all and combinator == 'union':             combinator_sql += ' ALL'        braces = '({})' if features.supports_slicing_ordering_in_compound else '{}'        braces = '{}'        if not self.query.subquery and features.supports_slicing_ordering_in_compound:            braces = '({})'         sql_parts, args_parts = zip(*((braces.format(sql), args) for sql, args in parts))         result = [' {} '.format(combinator_sql).join(sql_parts)]         params = []diff --git a/django/db/models/sql/query.py b/django/db/models/sql/query.py-- a/django/db/models/sql/query.py++ b/django/db/models/sql/query.py


Question: Can't use OuterRef in union Subquery Description When you make a QuerySet using the union method or the | operator, the QuerySet passed into the union method cannot reference OuterRef even when wrapped with Subquery. For example: cls = ( checklistisnull=False, part=OuterRef('id') ).values('checklistcustomer', 'created') ots = ( ownershiptransferisnull=False, part=OuterRef('id') ).values('ownershiptransfershiptargetcontact', 'created') return self.annotate( owner=Subquery((ots).values('owner')[:1]) ) Returns this error: ValueError This queryset contains a reference to an outer query and may only be used in a subquery. I get the same error with this statement: return self.annotate( owner=Subquery((cls | ots).values('owner')[:1]) ) (As an aside, I also get an error when I try to apply an orderby clause.)

clone.bump_prefix(query)         clone.subquery = True         clone.where.resolve_expression(query, *args, **kwargs)<ins>        # Resolve combined queries.        if clone.combinator:            clone.combined_queries = tuple([                combined_query.resolve_expression(query, *args, **kwargs)                for combined_query in clone.combined_queries            ])</ins>         for key, value in clone.annotations.items():             resolved = value.resolve_expression(query, *args, **kwargs)             if hasattr(resolved, 'external_aliases'):


Question: Rate-limit autocomplete widgets Ajax requests Description (last modified by Federico Jaramillo Martínez) The current implementation of the Ajax autocomplete widget using Select2 in Django triggers a request for every key-press. This creates unnecessary load on servers. This patch rate-limit the requests by adding a delay using the option provided by Select2.

attrs.setdefault('class', '')         attrs.update({             'data-ajax--cache': 'true',<ins>            'data-ajax--delay': 250,</ins>             'data-ajax--type': 'GET',             'data-ajax--url': self.get_url(),             'data-theme': 'admin-autocomplete',


Question: Admin search with a null character crashes with "A string literal cannot contain NUL (0x00) characters." on PostgreSQL Description Input following URL to browser URL field and access. http://localhost/admin/auth/user/?q=%00 Crash with following Error. Environment: Request Method: GET Request URL: http://localhost/admin/auth/user/?q=%00 Django Version: 2.1.4 Python Version: 3.6.7 Installed Applications: ['', 'django.contrib.contenttypes', 'django.contrib.sessions', 'django.contrib.messages', 'django.contrib.staticfiles', 'djangoextensions', 'restframework', 'select2', 'corsheaders', .......] Installed Middleware: ['django.middleware.security.SecurityMiddleware', 'django.contrib.sessions.middleware.SessionMiddleware', 'corsheaders.middleware.CorsMiddleware', '', '', '', 'django.contrib.messages.middleware.MessageMiddleware', 'django.middleware.clickjacking.XFrameOptionsMiddleware'] Traceback: File "" in inner response = getresponse(request) File "" in getresponse response = self.processexceptionbymiddleware(e, request) File "" in getresponse response = wrappedcallback(request, callbackargs, callbackkwargs) File "" in wrapper return self.adminsite.adminview(view)(args, kwargs) File "" in wrappedview response = viewfunc(request, args, kwargs) File "" in wrappedviewfunc response = viewfunc(request, args, kwargs) File "" in inner return view(request, args, kwargs) File "" in wrapper return boundmethod(args, kwargs) File "" in wrappedview response = viewfunc(request, args, *kwargs) File "" in changelistview cl = self.getchangelistinstance(request) File "" in getchangelistinstance sortableby, File "" in init self.getresults(request) File "" in getresults resultcount = File "" in get res = [] = (instance) File "" in count return () File "" in count return () File "" in getcount number = obj.getaggregation(using, ['count'])['count'] File "" in getaggregation result = compiler.executesql(SINGLE) File "" in executesql cursor.execute(sql, params) File "" in execute return super().execute(sql, params) File "" in execute return self.executewithwrappers(sql, params, many=False, executor=self.execute) File "" in executewithwrappers return executor(sql, params, many, context) File "" in _execute return (sql, params) Exception Type: ValueError at /admin/auth/user/ Exception Value: A string literal cannot contain NUL (0x00) characters.

from datetime import datetime, timedelta from django import forms from django.conf import settingsfrom django.contrib import messages from django.contrib.admin import FieldListFilter from django.contrib.admin.exceptions import (     DisallowedModelAdminLookup, DisallowedModelAdminToField,


Question: Admin search with a null character crashes with "A string literal cannot contain NUL (0x00) characters." on PostgreSQL Description Input following URL to browser URL field and access. http://localhost/admin/auth/user/?q=%00 Crash with following Error. Environment: Request Method: GET Request URL: http://localhost/admin/auth/user/?q=%00 Django Version: 2.1.4 Python Version: 3.6.7 Installed Applications: ['', 'django.contrib.contenttypes', 'django.contrib.sessions', 'django.contrib.messages', 'django.contrib.staticfiles', 'djangoextensions', 'restframework', 'select2', 'corsheaders', .......] Installed Middleware: ['django.middleware.security.SecurityMiddleware', 'django.contrib.sessions.middleware.SessionMiddleware', 'corsheaders.middleware.CorsMiddleware', '', '', '', 'django.contrib.messages.middleware.MessageMiddleware', 'django.middleware.clickjacking.XFrameOptionsMiddleware'] Traceback: File "" in inner response = getresponse(request) File "" in getresponse response = self.processexceptionbymiddleware(e, request) File "" in getresponse response = wrappedcallback(request, callbackargs, callbackkwargs) File "" in wrapper return self.adminsite.adminview(view)(args, kwargs) File "" in wrappedview response = viewfunc(request, args, kwargs) File "" in wrappedviewfunc response = viewfunc(request, args, kwargs) File "" in inner return view(request, args, kwargs) File "" in wrapper return boundmethod(args, kwargs) File "" in wrappedview response = viewfunc(request, args, *kwargs) File "" in changelistview cl = self.getchangelistinstance(request) File "" in getchangelistinstance sortableby, File "" in init self.getresults(request) File "" in getresults resultcount = File "" in get res = [] = (instance) File "" in count return () File "" in count return () File "" in getcount number = obj.getaggregation(using, ['count'])['count'] File "" in getaggregation result = compiler.executesql(SINGLE) File "" in executesql cursor.execute(sql, params) File "" in execute return super().execute(sql, params) File "" in execute return self.executewithwrappers(sql, params, many=False, executor=self.execute) File "" in executewithwrappers return executor(sql, params, many, context) File "" in _execute return (sql, params) Exception Type: ValueError at /admin/auth/user/ Exception Value: A string literal cannot contain NUL (0x00) characters.

ALL_VAR, ORDER_VAR, ORDER_TYPE_VAR, SEARCH_VAR, IS_POPUP_VAR, TO_FIELD_VAR)  class ChangeListSearchForm(forms.Form):    def __init__(self, *args, **kwargs):        super().__init__(*args, **kwargs)        # Populate "fields" dynamically because SEARCH_VAR is a variable:        self.fields = {            SEARCH_VAR: forms.CharField(required=False, strip=False),        } class ChangeList:    search_form_class = ChangeListSearchForm     def __init__(self, request, model, list_display, list_display_links,                  list_filter, date_hierarchy, search_fields, list_select_related,                  list_per_page, list_max_show_all, list_editable, model_admin, sortable_by):


Question: Admin search with a null character crashes with "A string literal cannot contain NUL (0x00) characters." on PostgreSQL Description Input following URL to browser URL field and access. http://localhost/admin/auth/user/?q=%00 Crash with following Error. Environment: Request Method: GET Request URL: http://localhost/admin/auth/user/?q=%00 Django Version: 2.1.4 Python Version: 3.6.7 Installed Applications: ['', 'django.contrib.contenttypes', 'django.contrib.sessions', 'django.contrib.messages', 'django.contrib.staticfiles', 'djangoextensions', 'restframework', 'select2', 'corsheaders', .......] Installed Middleware: ['django.middleware.security.SecurityMiddleware', 'django.contrib.sessions.middleware.SessionMiddleware', 'corsheaders.middleware.CorsMiddleware', '', '', '', 'django.contrib.messages.middleware.MessageMiddleware', 'django.middleware.clickjacking.XFrameOptionsMiddleware'] Traceback: File "" in inner response = getresponse(request) File "" in getresponse response = self.processexceptionbymiddleware(e, request) File "" in getresponse response = wrappedcallback(request, callbackargs, callbackkwargs) File "" in wrapper return self.adminsite.adminview(view)(args, kwargs) File "" in wrappedview response = viewfunc(request, args, kwargs) File "" in wrappedviewfunc response = viewfunc(request, args, kwargs) File "" in inner return view(request, args, kwargs) File "" in wrapper return boundmethod(args, kwargs) File "" in wrappedview response = viewfunc(request, args, *kwargs) File "" in changelistview cl = self.getchangelistinstance(request) File "" in getchangelistinstance sortableby, File "" in init self.getresults(request) File "" in getresults resultcount = File "" in get res = [] = (instance) File "" in count return () File "" in count return () File "" in getcount number = obj.getaggregation(using, ['count'])['count'] File "" in getaggregation result = compiler.executesql(SINGLE) File "" in executesql cursor.execute(sql, params) File "" in execute return super().execute(sql, params) File "" in execute return self.executewithwrappers(sql, params, many=False, executor=self.execute) File "" in executewithwrappers return executor(sql, params, many, context) File "" in _execute return (sql, params) Exception Type: ValueError at /admin/auth/user/ Exception Value: A string literal cannot contain NUL (0x00) characters.

self.sortable_by = sortable_by          # Get search parameters from the query string.<ins>        _search_form = self.search_form_class(request.GET)        if not _search_form.is_valid():            for error in _search_form.errors.values():                messages.error(request, ', '.join(error))        self.query = _search_form.cleaned_data.get(SEARCH_VAR) or ''</ins>         try:             self.page_num = int(request.GET.get(PAGE_VAR, 0))         except ValueError:


Question: Admin search with a null character crashes with "A string literal cannot contain NUL (0x00) characters." on PostgreSQL Description Input following URL to browser URL field and access. http://localhost/admin/auth/user/?q=%00 Crash with following Error. Environment: Request Method: GET Request URL: http://localhost/admin/auth/user/?q=%00 Django Version: 2.1.4 Python Version: 3.6.7 Installed Applications: ['', 'django.contrib.contenttypes', 'django.contrib.sessions', 'django.contrib.messages', 'django.contrib.staticfiles', 'djangoextensions', 'restframework', 'select2', 'corsheaders', .......] Installed Middleware: ['django.middleware.security.SecurityMiddleware', 'django.contrib.sessions.middleware.SessionMiddleware', 'corsheaders.middleware.CorsMiddleware', '', '', '', 'django.contrib.messages.middleware.MessageMiddleware', 'django.middleware.clickjacking.XFrameOptionsMiddleware'] Traceback: File "" in inner response = getresponse(request) File "" in getresponse response = self.processexceptionbymiddleware(e, request) File "" in getresponse response = wrappedcallback(request, callbackargs, callbackkwargs) File "" in wrapper return self.adminsite.adminview(view)(args, kwargs) File "" in wrappedview response = viewfunc(request, args, kwargs) File "" in wrappedviewfunc response = viewfunc(request, args, kwargs) File "" in inner return view(request, args, kwargs) File "" in wrapper return boundmethod(args, kwargs) File "" in wrappedview response = viewfunc(request, args, *kwargs) File "" in changelistview cl = self.getchangelistinstance(request) File "" in getchangelistinstance sortableby, File "" in init self.getresults(request) File "" in getresults resultcount = File "" in get res = [] = (instance) File "" in count return () File "" in count return () File "" in getcount number = obj.getaggregation(using, ['count'])['count'] File "" in getaggregation result = compiler.executesql(SINGLE) File "" in executesql cursor.execute(sql, params) File "" in execute return super().execute(sql, params) File "" in execute return self.executewithwrappers(sql, params, many=False, executor=self.execute) File "" in executewithwrappers return executor(sql, params, many, context) File "" in _execute return (sql, params) Exception Type: ValueError at /admin/auth/user/ Exception Value: A string literal cannot contain NUL (0x00) characters.

self.list_editable = ()         else:             self.list_editable = list_editable<del>        self.query = request.GET.get(SEARCH_VAR, '')</del>         self.queryset = self.get_queryset(request)         self.get_results(request)         if self.is_popup:


Question: Pickling a QuerySet evaluates the querysets given to Subquery in annotate. Description I wrote a test case for modeled after the test from bug which is very similar. def testpicklesubqueryquerysetnotevaluated(self): """ Verifies that querysets passed into Subquery expressions are not evaluated when pickled """ groups = Group.objects.annotate( hasevent=models.Exists((groupid=models.OuterRef('id'))) ) with self.assertNumQueries(0): (()) The Subquery class (which is the base for Exists) only stores the underlying query object and throws the QuerySet away (as of this commit, although I don't think it worked before that). So in theory it shouldn't be pickling the QuerySet. However, the QuerySet is still stored on the instance within the constructorargs attribute by the decorator on the BaseExpression base class. So when the inner query object gets pickled, so does the QuerySet, which attempts to evaluate it. In this case, it gets the error "ValueError: This queryset contains a reference to an outer query and may only be used in a subquery." since the inner queryset is being evaluated independently when called from pickle: it calls QuerySet.getstate(). I'm not sure what the best solution is here. I made a patch that adds the following override to getstate to the SubQuery class, which fixes the problem and passes all other Django tests on my machine. I'll submit a PR shortly, but welcome any better approaches since I'm not sure what else that may effect. class Subquery(Expression): ... def getstate(self): objdict = super().getstate() ('constructorargs', None) return obj_dict

self.extra = extra         super().__init__(output_field) <ins>    def __getstate__(self):        state = super().__getstate__()        state.pop('_constructor_args', None)        return state</ins>     def get_source_expressions(self):         return [self.query]


Question: Allow hiding the "Save and Add Another" button with a context variable Description To provide better adjustability, to introduce new context var - showsaveandaddanother. E.g. if I want to hide button "Save and add another", I can just modify extra_context - write False to the variable. For other buttons - "Save" and "Save and continue editing", this already works exactly in this manner.

is_popup = context['is_popup']     save_as = context['save_as']     show_save = context.get('show_save', True)    show_save_and_add_another = context.get('show_save_and_add_another', True)     show_save_and_continue = context.get('show_save_and_continue', True)     has_add_permission = context['has_add_permission']     has_change_permission = context['has_change_permission']     has_view_permission = context['has_view_permission']     has_editable_inline_admin_formsets = context['has_editable_inline_admin_formsets']     can_save = (has_change_permission and change) or (has_add_permission and add) or has_editable_inline_admin_formsets    can_save_and_add_another = (        has_add_permission and        not is_popup and        (not save_as or add) and        can_save and        show_save_and_add_another    )     can_save_and_continue = not is_popup and can_save and has_view_permission and show_save_and_continue     can_change = has_change_permission or has_editable_inline_admin_formsets     ctx = Context(context)


Question: Allow hiding the "Save and Add Another" button with a context variable Description To provide better adjustability, to introduce new context var - showsaveandaddanother. E.g. if I want to hide button "Save and add another", I can just modify extra_context - write False to the variable. For other buttons - "Save" and "Save and continue editing", this already works exactly in this manner.

change and context.get('show_delete', True)         ),         'show_save_as_new': not is_popup and has_change_permission and change and save_as,        'show_save_and_add_another': (            has_add_permission and not is_popup and            (not save_as or add) and can_save        ),        'show_save_and_add_another': can_save_and_add_another,         'show_save_and_continue': can_save_and_continue,         'show_save': show_save and can_save,         'show_close': not(show_save and can_save)


Question: simplifyregexp() doesn't replace trailing groups. Description replacenamedgroups() fails to replace the final named group if the urlpattern passed in is missing a trailing '/'. For example, with input r'entries/(?P<pk[^/.]+)/relationships/(?P<relatedfield\w+)' the "related_field" does not get properly replaced. A workaround is to tack on a '/' at the end and then it works. Code that reproduces this is attached. This function is used downstream in Django REST Framework. See issue 6888

Find named groups in `pattern` and replace them with the group name. E.g.,     1. ^(?P<a>\w+)/b/(\w+)$ ==> ^<a>/b/(\w+)$     2. ^(?P<a>\w+)/b/(?P<c>\w+)/$ ==> ^<a>/b/<c>/$<ins>    3. ^(?P<a>\w+)/b/(\w+) ==> ^<a>/b/(\w+)    4. ^(?P<a>\w+)/b/(?P<c>\w+) ==> ^<a>/b/<c></ins>     """     named_group_indices = [         (m.start(0), m.end(0), m.group(1))


Question: simplifyregexp() doesn't replace trailing groups. Description replacenamedgroups() fails to replace the final named group if the urlpattern passed in is missing a trailing '/'. For example, with input r'entries/(?P<pk[^/.]+)/relationships/(?P<relatedfield\w+)' the "related_field" does not get properly replaced. A workaround is to tack on a '/' at the end and then it works. Code that reproduces this is attached. This function is used downstream in Django REST Framework. See issue 6888

# Handle nested parentheses, e.g. '^(?P<a>(x|y))/b'.         unmatched_open_brackets, prev_char = 1, None         for idx, val in enumerate(pattern[end:]):<del>            # If brackets are balanced, the end of the string for the current            # named capture group pattern has been reached.            if unmatched_open_brackets == 0:                group_pattern_and_name.append((pattern[start:end + idx], group_name))                break</del>             # Check for unescaped `(` and `)`. They mark the start and end of a             # nested group.             if val == '(' and prev_char != '\\':


Question: simplifyregexp() doesn't replace trailing groups. Description replacenamedgroups() fails to replace the final named group if the urlpattern passed in is missing a trailing '/'. For example, with input r'entries/(?P<pk[^/.]+)/relationships/(?P<relatedfield\w+)' the "related_field" does not get properly replaced. A workaround is to tack on a '/' at the end and then it works. Code that reproduces this is attached. This function is used downstream in Django REST Framework. See issue 6888

elif val == ')' and prev_char != '\\':                 unmatched_open_brackets -= 1             prev_char = val<ins>            # If brackets are balanced, the end of the string for the current            # named capture group pattern has been reached.            if unmatched_open_brackets == 0:                group_pattern_and_name.append((pattern[start:end + idx + 1], group_name))                break</ins>      # Replace the string for named capture groups with their group names.     for group_pattern, group_name in group_pattern_and_name:


Question: simplifyregexp() doesn't replace trailing groups. Description replacenamedgroups() fails to replace the final named group if the urlpattern passed in is missing a trailing '/'. For example, with input r'entries/(?P<pk[^/.]+)/relationships/(?P<relatedfield\w+)' the "related_field" does not get properly replaced. A workaround is to tack on a '/' at the end and then it works. Code that reproduces this is attached. This function is used downstream in Django REST Framework. See issue 6888

Find unnamed groups in `pattern` and replace them with '<var>'. E.g.,     1. ^(?P<a>\w+)/b/(\w+)$ ==> ^(?P<a>\w+)/b/<var>$     2. ^(?P<a>\w+)/b/((x|y)\w+)$ ==> ^(?P<a>\w+)/b/<var>$<ins>    3. ^(?P<a>\w+)/b/(\w+) ==> ^(?P<a>\w+)/b/<var>    4. ^(?P<a>\w+)/b/((x|y)\w+) ==> ^(?P<a>\w+)/b/<var></ins>     """     unnamed_group_indices = [m.start(0) for m in unnamed_group_matcher.finditer(pattern)]     # Indices of the start of unnamed capture groups.


Question: simplifyregexp() doesn't replace trailing groups. Description replacenamedgroups() fails to replace the final named group if the urlpattern passed in is missing a trailing '/'. For example, with input r'entries/(?P<pk[^/.]+)/relationships/(?P<relatedfield\w+)' the "related_field" does not get properly replaced. A workaround is to tack on a '/' at the end and then it works. Code that reproduces this is attached. This function is used downstream in Django REST Framework. See issue 6888

# Handle nested parentheses, e.g. '^b/((x|y)\w+)$'.         unmatched_open_brackets, prev_char = 1, None         for idx, val in enumerate(pattern[start + 1:]):<del>            if unmatched_open_brackets == 0:                group_indices.append((start, start + 1 + idx))                break</del>             # Check for unescaped `(` and `)`. They mark the start and end of             # a nested group.             if val == '(' and prev_char != '\\':


Question: simplifyregexp() doesn't replace trailing groups. Description replacenamedgroups() fails to replace the final named group if the urlpattern passed in is missing a trailing '/'. For example, with input r'entries/(?P<pk[^/.]+)/relationships/(?P<relatedfield\w+)' the "related_field" does not get properly replaced. A workaround is to tack on a '/' at the end and then it works. Code that reproduces this is attached. This function is used downstream in Django REST Framework. See issue 6888

unmatched_open_brackets -= 1             prev_char = val <ins>            if unmatched_open_brackets == 0:                group_indices.append((start, start + 2 + idx))                break</ins>     # Remove unnamed group matches inside other unnamed capture groups.     group_start_end_indices = []     prev_end = None


Question: OuterRef in exclude() or ~Q() uses wrong model. Description The following test ( to tests/queries/testqscombinators) fails when trying to exclude results using OuterRef() def testexistsexclude(self): # filter() qs = Number.objects.annotate( foo=Exists( (tagscategoryid=OuterRef('pk')) ) ).filter(foo=True) print(qs) # works # exclude() qs = Number.objects.annotate( foo =Exists( Item.objects.exclude(tagscategoryid=OuterRef('pk')) ) ).filter(foo=True) print(qs) # crashes # filter(~Q()) qs = Number.objects.annotate( foo =Exists( (~Q(tags_categoryid=OuterRef('pk'))) ) ).filter(foo=True) print(qs) # crashes It results in the following error ValueError: This queryset contains a reference to an outer query and may only be used in a subquery

value = connection.ops.validate_autopk_value(value)         return value     def get_prep_value(self, value):        from django.db.models.expressions import OuterRef        return value if isinstance(value, OuterRef) else super().get_prep_value(value)     def contribute_to_class(self, cls, name, **kwargs):         assert not cls._meta.auto_field, (             "Model %s can't have more than one auto-generated field."diff --git a/django/db/models/fields/related_lookups.py b/django/db/models/fields/related_lookups.py-- a/django/db/models/fields/related_lookups.py++ b/django/db/models/fields/related_lookups.py


Question: OuterRef in exclude() or ~Q() uses wrong model. Description The following test ( to tests/queries/testqscombinators) fails when trying to exclude results using OuterRef() def testexistsexclude(self): # filter() qs = Number.objects.annotate( foo=Exists( (tagscategoryid=OuterRef('pk')) ) ).filter(foo=True) print(qs) # works # exclude() qs = Number.objects.annotate( foo =Exists( Item.objects.exclude(tagscategoryid=OuterRef('pk')) ) ).filter(foo=True) print(qs) # crashes # filter(~Q()) qs = Number.objects.annotate( foo =Exists( (~Q(tags_categoryid=OuterRef('pk'))) ) ).filter(foo=True) print(qs) # crashes It results in the following error ValueError: This queryset contains a reference to an outer query and may only be used in a subquery

class RelatedLookupMixin:     def get_prep_lookup(self):        if not isinstance(self.lhs, MultiColSource) and self.rhs_is_direct_value():        if not isinstance(self.lhs, MultiColSource) and not hasattr(self.rhs, 'resolve_expression'):             # If we get here, we are dealing with single-column relations.             self.rhs = get_normalized_value(self.rhs, self.lhs)[0]             # We need to run the related field's get_prep_value(). Consider casediff --git a/django/db/models/sql/query.py b/django/db/models/sql/query.py-- a/django/db/models/sql/query.py++ b/django/db/models/sql/query.py


Question: OuterRef in exclude() or ~Q() uses wrong model. Description The following test ( to tests/queries/testqscombinators) fails when trying to exclude results using OuterRef() def testexistsexclude(self): # filter() qs = Number.objects.annotate( foo=Exists( (tagscategoryid=OuterRef('pk')) ) ).filter(foo=True) print(qs) # works # exclude() qs = Number.objects.annotate( foo =Exists( Item.objects.exclude(tagscategoryid=OuterRef('pk')) ) ).filter(foo=True) print(qs) # crashes # filter(~Q()) qs = Number.objects.annotate( foo =Exists( (~Q(tags_categoryid=OuterRef('pk'))) ) ).filter(foo=True) print(qs) # crashes It results in the following error ValueError: This queryset contains a reference to an outer query and may only be used in a subquery

handle.         """         filter_lhs, filter_rhs = filter_expr        if isinstance(filter_rhs, F):        if isinstance(filter_rhs, OuterRef):            filter_expr = (filter_lhs, OuterRef(filter_rhs))        elif isinstance(filter_rhs, F):             filter_expr = (filter_lhs, OuterRef(filter_rhs.name))         # Generate the inner query.         query = Query(self.model)


Question: Change uuid field to FK does not create dependency Description (last modified by Simon Charette) Hi! I am new in django community, so please help me, because i really dont know is it really "bug". I have a django project named "testproject" and two apps: testapp1, testapp2. It will be simpler to understand, with this example: class App1(models.Model): id = models.UUIDField(primarykey=True, unique=True, default=uuid.uuid4, editable=False, verbosename=('identifier')) text = models.CharField(maxlength=100, verbosename=('text')) anotherapp = models.UUIDField(null=True, blank=True, verbosename=('another app')) class App2(models.Model): id = models.UUIDField(primarykey=True, unique=True, default=uuid.uuid4, editable=False, verbosename=('identifier')) text = models.CharField(maxlength=100, verbosename=('text')) First model named "App1" has UUID field named "anotherapp": anotherapp = models.UUIDField(null=True, blank=True, verbosename=('another app')) After some time i change field from UUID to FK, like this: anotherapp = models.ForeignKey(App2, null=True, blank=True, ondelete=models.SETNULL, verbosename=('another app')) And as result i create new migration, but Migration class was unexpected result, because it does not create any "dependencies" for App2, because of FK. I think the correct solution will be create dependency for App2. This project use django version 2.2 and postgresql. Attach archive with sources. Project contains small test, after running him, you will get exception like this: ValueError: Related model 'testapp2.App2' cannot be resolved. So is it problem in django or maybe i dont understand something ? Here is my post in django users: !searchin/django-users/Django3A20uuid20to20does20create$20dependency%7Csort:date/django-users/-h9LZxFomLU/yz-NLi1cDgAJ Regards, Viktor Lomakin

old_field_name = self.renamed_fields.get((app_label, model_name, field_name), field_name)             old_field = self.old_apps.get_model(app_label, old_model_name)._meta.get_field(old_field_name)             new_field = self.new_apps.get_model(app_label, model_name)._meta.get_field(field_name)<ins>            dependencies = []</ins>             # Implement any model renames on relations; these are handled by RenameModel             # so we need to exclude them from the comparison             if hasattr(new_field, "remote_field") and getattr(new_field.remote_field, "model", None):


Question: Change uuid field to FK does not create dependency Description (last modified by Simon Charette) Hi! I am new in django community, so please help me, because i really dont know is it really "bug". I have a django project named "testproject" and two apps: testapp1, testapp2. It will be simpler to understand, with this example: class App1(models.Model): id = models.UUIDField(primarykey=True, unique=True, default=uuid.uuid4, editable=False, verbosename=('identifier')) text = models.CharField(maxlength=100, verbosename=('text')) anotherapp = models.UUIDField(null=True, blank=True, verbosename=('another app')) class App2(models.Model): id = models.UUIDField(primarykey=True, unique=True, default=uuid.uuid4, editable=False, verbosename=('identifier')) text = models.CharField(maxlength=100, verbosename=('text')) First model named "App1" has UUID field named "anotherapp": anotherapp = models.UUIDField(null=True, blank=True, verbosename=('another app')) After some time i change field from UUID to FK, like this: anotherapp = models.ForeignKey(App2, null=True, blank=True, ondelete=models.SETNULL, verbosename=('another app')) And as result i create new migration, but Migration class was unexpected result, because it does not create any "dependencies" for App2, because of FK. I think the correct solution will be create dependency for App2. This project use django version 2.2 and postgresql. Attach archive with sources. Project contains small test, after running him, you will get exception like this: ValueError: Related model 'testapp2.App2' cannot be resolved. So is it problem in django or maybe i dont understand something ? Here is my post in django users: !searchin/django-users/Django3A20uuid20to20does20create$20dependency%7Csort:date/django-users/-h9LZxFomLU/yz-NLi1cDgAJ Regards, Viktor Lomakin

self.renamed_fields.get(rename_key + (to_field,), to_field)                         for to_field in new_field.to_fields                     ])<ins>                dependencies.extend(self._get_dependencies_for_foreign_key(new_field))</ins>             if hasattr(new_field, "remote_field") and getattr(new_field.remote_field, "through", None):                 rename_key = (                     new_field.remote_field.through._meta.app_label,


Question: Change uuid field to FK does not create dependency Description (last modified by Simon Charette) Hi! I am new in django community, so please help me, because i really dont know is it really "bug". I have a django project named "testproject" and two apps: testapp1, testapp2. It will be simpler to understand, with this example: class App1(models.Model): id = models.UUIDField(primarykey=True, unique=True, default=uuid.uuid4, editable=False, verbosename=('identifier')) text = models.CharField(maxlength=100, verbosename=('text')) anotherapp = models.UUIDField(null=True, blank=True, verbosename=('another app')) class App2(models.Model): id = models.UUIDField(primarykey=True, unique=True, default=uuid.uuid4, editable=False, verbosename=('identifier')) text = models.CharField(maxlength=100, verbosename=('text')) First model named "App1" has UUID field named "anotherapp": anotherapp = models.UUIDField(null=True, blank=True, verbosename=('another app')) After some time i change field from UUID to FK, like this: anotherapp = models.ForeignKey(App2, null=True, blank=True, ondelete=models.SETNULL, verbosename=('another app')) And as result i create new migration, but Migration class was unexpected result, because it does not create any "dependencies" for App2, because of FK. I think the correct solution will be create dependency for App2. This project use django version 2.2 and postgresql. Attach archive with sources. Project contains small test, after running him, you will get exception like this: ValueError: Related model 'testapp2.App2' cannot be resolved. So is it problem in django or maybe i dont understand something ? Here is my post in django users: !searchin/django-users/Django3A20uuid20to20does20create$20dependency%7Csort:date/django-users/-h9LZxFomLU/yz-NLi1cDgAJ Regards, Viktor Lomakin

name=field_name,                             field=field,                             preserve_default=preserve_default,                        )                        ),                        dependencies=dependencies,                     )                 else:                     # We cannot alter between m2m and concrete fields


Question: Add check to ensure maxlength fits longest choice. Description There is currently no check to ensure that Field.maxlength is large enough to fit the longest value in Field.choices. This would be very helpful as often this mistake is not noticed until an attempt is made to save a record with those values that are too long.

)             ] <ins>        choice_max_length = 0</ins>         # Expect [group_name, [value, display]]         for choices_group in self.choices:             try:


Question: Add check to ensure maxlength fits longest choice. Description There is currently no check to ensure that Field.maxlength is large enough to fit the longest value in Field.choices. This would be very helpful as often this mistake is not noticed until an attempt is made to save a record with those values that are too long.

for value, human_name in group_choices                 ):                     break                if self.max_length is not None and group_choices:                    choice_max_length = max(                        choice_max_length,                        *(len(value) for value, _ in group_choices if isinstance(value, str)),                    )             except (TypeError, ValueError):                 # No groups, choices in the form [value, display]                 value, human_name = group_name, group_choices                 if not is_value(value) or not is_value(human_name):                     break                if self.max_length is not None and isinstance(value, str):                    choice_max_length = max(choice_max_length, len(value))              # Special case: choices=['ab']             if isinstance(choices_group, str):                 break         else:            if self.max_length is not None and choice_max_length > self.max_length:                return [                    checks.Error(                        "'max_length' is too small to fit the longest value "                        "in 'choices' (%d characters)." % choice_max_length,                        obj=self,                        id='fields.E009',                    ),                ]             return []          return [


Question: callcommand fails when argument of required mutually exclusive group is passed in kwargs. Description This error Error: one of the arguments --shop-id --shop is required is raised when I run callcommand('mycommand', shopid=1) the argument 'shopid' is part of a required mutually exclusive group: shop = parser.addmutuallyexclusivegroup(required=True) shop.addargument('--shop-id', nargs='?', type=int, default=None, dest='shopid') shop.addargument('--shop', nargs='?', type=str, default=None, dest='shopname') However, everything is fine when I call this command in this way: callcommand('mycommand, '--shop-id=1') In django sources I found that only those keyword arguments of callcommand are passed to the parser that are defined as required: parseargs += [ '{}={}'.format(min(opt.optionstrings), argoptions[]) for opt in parser.actions if opt.required and in options ] but in this special case both of them individually are not required, they are actually part of a group that is required. And the code of callcommand does nothing with groups defined in the parser.

yield opt      parser_actions = list(get_actions(parser))    mutually_exclusive_required_options = {        opt        for group in parser._mutually_exclusive_groups        for opt in group._group_actions if group.required    }     # Any required arguments which are passed in via **options must be passed     # to parse_args().     parse_args += [         '{}={}'.format(min(opt.option_strings), arg_options[opt.dest])        for opt in parser_actions if opt.required and opt.dest in options        for opt in parser_actions if (            opt.dest in options and            (opt.required or opt in mutually_exclusive_required_options)        )     ]     defaults = parser.parse_args(args=parse_args)     defaults = dict(defaults._get_kwargs(), **arg_options)


Question: Make security headers default. Description (last modified by Adam Johnson) Following my security headers talk at DjangoCon Europe and its related blog post (  ), I'd like to make Django use more of these security headers by default on new projects. They're always harder to roll out on existing projects than to just bake in to the new project template. On current master, running python check --deploy on a fresh project created with startproject yields these warnings: System check identified some issues: WARNINGS: ?: (security.W004) You have not set a value for the SECUREHSTSSECONDS setting. If your entire site is served only over SSL, you may want to consider setting a value and enabling HTTP Strict Transport Security. Be sure to read the documentation first; enabling HSTS carelessly can cause serious, irreversible problems. ?: (security.W006) Your SECURECONTENTTYPENOSNIFF setting is not set to True, so your pages will not be served with an 'X-Content-Type-Options: nosniff' header. You should consider enabling this header to prevent the browser from identifying content types incorrectly. ?: (security.W007) Your SECUREBROWSERXSSFILTER setting is not set to True, so your pages will not be served with an 'X-XSS-Protection: 1; mode=block' header. You should consider enabling this header to activate the browser's XSS filtering and help prevent XSS attacks. ?: (security.W008) Your SECURESSLREDIRECT setting is not set to True. Unless your site should be available over both SSL and non-SSL connections, you may want to either set this setting True or configure a load balancer or reverse-proxy server to redirect all connections to HTTPS. ?: (security.W012) SESSIONCOOKIESECURE is not set to True. Using a secure-only session cookie makes it more difficult for network traffic sniffers to hijack user sessions. ?: (security.W016) You have '' in your MIDDLEWARE, but you have not set CSRFCOOKIESECURE to True. Using a secure-only CSRF cookie makes it more difficult for network traffic sniffers to steal the CSRF token. ?: (security.W018) You should not have DEBUG set to True in deployment. ?: (security.W019) You have 'django.middleware.clickjacking.XFrameOptionsMiddleware' in your MIDDLEWARE, but XFRAMEOPTIONS is not set to 'DENY'. The default is 'SAMEORIGIN', but unless there is a good reason for your site to serve other parts of itself in a frame, you should change it to 'DENY'. ?: (security.W020) ALLOWEDHOSTS must not be empty in deployment. System check identified 9 issues (0 silenced). Three of these come from security headers that we could activate by default in the settings SECURECONTENTTYPENOSNIFF, SECUREBROWSERXSSFILTER, and XFRAME_OPTIONS. I'd like to propose making them default in the startproject settings and even changing their global defaults (through a deprecation period) so they are activated by default.

DEFAULT_INDEX_TABLESPACE = ''  # Default X-Frame-Options header valueX_FRAME_OPTIONS = 'SAMEORIGIN'X_FRAME_OPTIONS = 'DENY'  USE_X_FORWARDED_HOST = False USE_X_FORWARDED_PORT = Falsediff --git a/django/core/checks/security/base.py b/django/core/checks/security/base.py-- a/django/core/checks/security/base.py++ b/django/core/checks/security/base.py


Question: Make security headers default. Description (last modified by Adam Johnson) Following my security headers talk at DjangoCon Europe and its related blog post (  ), I'd like to make Django use more of these security headers by default on new projects. They're always harder to roll out on existing projects than to just bake in to the new project template. On current master, running python check --deploy on a fresh project created with startproject yields these warnings: System check identified some issues: WARNINGS: ?: (security.W004) You have not set a value for the SECUREHSTSSECONDS setting. If your entire site is served only over SSL, you may want to consider setting a value and enabling HTTP Strict Transport Security. Be sure to read the documentation first; enabling HSTS carelessly can cause serious, irreversible problems. ?: (security.W006) Your SECURECONTENTTYPENOSNIFF setting is not set to True, so your pages will not be served with an 'X-Content-Type-Options: nosniff' header. You should consider enabling this header to prevent the browser from identifying content types incorrectly. ?: (security.W007) Your SECUREBROWSERXSSFILTER setting is not set to True, so your pages will not be served with an 'X-XSS-Protection: 1; mode=block' header. You should consider enabling this header to activate the browser's XSS filtering and help prevent XSS attacks. ?: (security.W008) Your SECURESSLREDIRECT setting is not set to True. Unless your site should be available over both SSL and non-SSL connections, you may want to either set this setting True or configure a load balancer or reverse-proxy server to redirect all connections to HTTPS. ?: (security.W012) SESSIONCOOKIESECURE is not set to True. Using a secure-only session cookie makes it more difficult for network traffic sniffers to hijack user sessions. ?: (security.W016) You have '' in your MIDDLEWARE, but you have not set CSRFCOOKIESECURE to True. Using a secure-only CSRF cookie makes it more difficult for network traffic sniffers to steal the CSRF token. ?: (security.W018) You should not have DEBUG set to True in deployment. ?: (security.W019) You have 'django.middleware.clickjacking.XFrameOptionsMiddleware' in your MIDDLEWARE, but XFRAMEOPTIONS is not set to 'DENY'. The default is 'SAMEORIGIN', but unless there is a good reason for your site to serve other parts of itself in a frame, you should change it to 'DENY'. ?: (security.W020) ALLOWEDHOSTS must not be empty in deployment. System check identified 9 issues (0 silenced). Three of these come from security headers that we could activate by default in the settings SECURECONTENTTYPENOSNIFF, SECUREBROWSERXSSFILTER, and XFRAME_OPTIONS. I'd like to propose making them default in the startproject settings and even changing their global defaults (through a deprecation period) so they are activated by default.

"You have "     "'django.middleware.clickjacking.XFrameOptionsMiddleware' in your "     "MIDDLEWARE, but X_FRAME_OPTIONS is not set to 'DENY'. "    "The default is 'SAMEORIGIN', but unless there is a good reason for "    "your site to serve other parts of itself in a frame, you should "    "change it to 'DENY'.",    "Unless there is a good reason for your site to serve other parts of "    "itself in a frame, you should change it to 'DENY'.",     id='security.W019', ) diff --git a/django/middleware/clickjacking.py b/django/middleware/clickjacking.py-- a/django/middleware/clickjacking.py++ b/django/middleware/clickjacking.py


Question: Make security headers default. Description (last modified by Adam Johnson) Following my security headers talk at DjangoCon Europe and its related blog post (  ), I'd like to make Django use more of these security headers by default on new projects. They're always harder to roll out on existing projects than to just bake in to the new project template. On current master, running python check --deploy on a fresh project created with startproject yields these warnings: System check identified some issues: WARNINGS: ?: (security.W004) You have not set a value for the SECUREHSTSSECONDS setting. If your entire site is served only over SSL, you may want to consider setting a value and enabling HTTP Strict Transport Security. Be sure to read the documentation first; enabling HSTS carelessly can cause serious, irreversible problems. ?: (security.W006) Your SECURECONTENTTYPENOSNIFF setting is not set to True, so your pages will not be served with an 'X-Content-Type-Options: nosniff' header. You should consider enabling this header to prevent the browser from identifying content types incorrectly. ?: (security.W007) Your SECUREBROWSERXSSFILTER setting is not set to True, so your pages will not be served with an 'X-XSS-Protection: 1; mode=block' header. You should consider enabling this header to activate the browser's XSS filtering and help prevent XSS attacks. ?: (security.W008) Your SECURESSLREDIRECT setting is not set to True. Unless your site should be available over both SSL and non-SSL connections, you may want to either set this setting True or configure a load balancer or reverse-proxy server to redirect all connections to HTTPS. ?: (security.W012) SESSIONCOOKIESECURE is not set to True. Using a secure-only session cookie makes it more difficult for network traffic sniffers to hijack user sessions. ?: (security.W016) You have '' in your MIDDLEWARE, but you have not set CSRFCOOKIESECURE to True. Using a secure-only CSRF cookie makes it more difficult for network traffic sniffers to steal the CSRF token. ?: (security.W018) You should not have DEBUG set to True in deployment. ?: (security.W019) You have 'django.middleware.clickjacking.XFrameOptionsMiddleware' in your MIDDLEWARE, but XFRAMEOPTIONS is not set to 'DENY'. The default is 'SAMEORIGIN', but unless there is a good reason for your site to serve other parts of itself in a frame, you should change it to 'DENY'. ?: (security.W020) ALLOWEDHOSTS must not be empty in deployment. System check identified 9 issues (0 silenced). Three of these come from security headers that we could activate by default in the settings SECURECONTENTTYPENOSNIFF, SECUREBROWSERXSSFILTER, and XFRAME_OPTIONS. I'd like to propose making them default in the startproject settings and even changing their global defaults (through a deprecation period) so they are activated by default.

def get_xframe_options_value(self, request, response):         """         Get the value to set for the X_FRAME_OPTIONS header. Use the value from        the X_FRAME_OPTIONS setting, or 'SAMEORIGIN' if not set.        the X_FRAME_OPTIONS setting, or 'DENY' if not set.          This method can be overridden if needed, allowing it to vary based on         the request or response.         """        return getattr(settings, 'X_FRAME_OPTIONS', 'SAMEORIGIN').upper()        return getattr(settings, 'X_FRAME_OPTIONS', 'DENY').upper()


Question: Allow using ExceptionReporter subclass in Description (last modified by Carlton Gibson) allows using an ExceptionReporter subclass with AdminEmailHandler. Ideally we'd make the similar available for the 500 debug error view. Currently the use of is hardcoded. Move this to a parameter Provide an example of using, e.g., functools.partial to configure a subclass when specifying handler500. Updated for comment:5 Add ExceptionReporter to the documentation, explaining the relation between the exception reporter class and the filter it uses, and showing a simple override of gettracebackdata(). Add a DEFAULTEXCEPTIONREPORTER setting to allow specifying this. (At that point we could review deprecating DEFAULTEXCEPTIONREPORTER_FILTER, as was discussed on — but maybe that's unnecessary. Thoughts?)

# Custom logging configuration. LOGGING = {} # Default exception reporter class used in case none has been# specifically assigned to the HttpRequest instance.DEFAULT_EXCEPTION_REPORTER = 'django.views.debug.ExceptionReporter' # Default exception reporter filter class used in case none has been # specifically assigned to the HttpRequest instance. DEFAULT_EXCEPTION_REPORTER_FILTER = 'django.views.debug.SafeExceptionReporterFilter'diff --git a/django/utils/log.py b/django/utils/log.py-- a/django/utils/log.py++ b/django/utils/log.py


Question: Allow using ExceptionReporter subclass in Description (last modified by Carlton Gibson) allows using an ExceptionReporter subclass with AdminEmailHandler. Ideally we'd make the similar available for the 500 debug error view. Currently the use of is hardcoded. Move this to a parameter Provide an example of using, e.g., functools.partial to configure a subclass when specifying handler500. Updated for comment:5 Add ExceptionReporter to the documentation, explaining the relation between the exception reporter class and the filter it uses, and showing a simple override of gettracebackdata(). Add a DEFAULTEXCEPTIONREPORTER setting to allow specifying this. (At that point we could review deprecating DEFAULTEXCEPTIONREPORTER_FILTER, as was discussed on — but maybe that's unnecessary. Thoughts?)

super().__init__()         self.include_html = include_html         self.email_backend = email_backend        self.reporter_class = import_string(reporter_class or 'django.views.debug.ExceptionReporter')        self.reporter_class = import_string(reporter_class or settings.DEFAULT_EXCEPTION_REPORTER)      def emit(self, record):         try:diff --git a/django/views/debug.py b/django/views/debug.py-- a/django/views/debug.py++ b/django/views/debug.py


Question: Allow using ExceptionReporter subclass in Description (last modified by Carlton Gibson) allows using an ExceptionReporter subclass with AdminEmailHandler. Ideally we'd make the similar available for the 500 debug error view. Currently the use of is hardcoded. Move this to a parameter Provide an example of using, e.g., functools.partial to configure a subclass when specifying handler500. Updated for comment:5 Add ExceptionReporter to the documentation, explaining the relation between the exception reporter class and the filter it uses, and showing a simple override of gettracebackdata(). Add a DEFAULTEXCEPTIONREPORTER setting to allow specifying this. (At that point we could review deprecating DEFAULTEXCEPTIONREPORTER_FILTER, as was discussed on — but maybe that's unnecessary. Thoughts?)

Create a technical server error response. The last three arguments are     the values returned from sys.exc_info() and friends.     """    reporter = ExceptionReporter(request, exc_type, exc_value, tb)    reporter = get_exception_reporter_class(request)(request, exc_type, exc_value, tb)     if request.is_ajax():         text = reporter.get_traceback_text()         return HttpResponse(text, status=status_code, content_type='text/plain; charset=utf-8')


Question: Allow using ExceptionReporter subclass in Description (last modified by Carlton Gibson) allows using an ExceptionReporter subclass with AdminEmailHandler. Ideally we'd make the similar available for the 500 debug error view. Currently the use of is hardcoded. Move this to a parameter Provide an example of using, e.g., functools.partial to configure a subclass when specifying handler500. Updated for comment:5 Add ExceptionReporter to the documentation, explaining the relation between the exception reporter class and the filter it uses, and showing a simple override of gettracebackdata(). Add a DEFAULTEXCEPTIONREPORTER setting to allow specifying this. (At that point we could review deprecating DEFAULTEXCEPTIONREPORTER_FILTER, as was discussed on — but maybe that's unnecessary. Thoughts?)

return getattr(request, 'exception_reporter_filter', default_filter)  <ins>def get_exception_reporter_class(request):    default_exception_reporter_class = import_string(settings.DEFAULT_EXCEPTION_REPORTER)    return getattr(request, 'exception_reporter_class', default_exception_reporter_class)</ins> class SafeExceptionReporterFilter:     """     Use annotations made by the sensitive_post_parameters and