public UTF8StreamJsonParser(IOContext ctxt, int features, InputStream in,
        ObjectCodec codec, BytesToNameCanonicalizer sym,
        byte[] inputBuffer, int start, int end,
        boolean bufferRecyclable)
{
    super(ctxt, features);
    _inputStream = in;
    _objectCodec = codec;
    _symbols = sym;
    _inputBuffer = inputBuffer;
    _inputPtr = start;
    _inputEnd = end;
    // If we have offset, need to omit that from byte offset, so:
    _bufferRecyclable = bufferRecyclable;
}

public UTF8StreamJsonParser(IOContext ctxt, int features, InputStream in,
        ObjectCodec codec, BytesToNameCanonicalizer sym,
        byte[] inputBuffer, int start, int end,
        boolean bufferRecyclable)
{
    super(ctxt, features);
    _inputStream = in;
    _objectCodec = codec;
    _symbols = sym;
    _inputBuffer = inputBuffer;
    _inputPtr = start;
    _inputEnd = end;
    _currInputRowStart = start;
    // If we have offset, need to omit that from byte offset, so:
    _currInputProcessed = -start;
    _bufferRecyclable = bufferRecyclable;
}

_currInputRowStart isn't initialized in UTF8StreamJsonParser() constructor. The column position will be wrong.

The UTF8StreamJson Parser constructor allows to specify the start position. But it doesn't set the  "_currInputRowStart" as the same value. It is still 0. So when raise the exception, the column calculation (ParserBase.getCurrentLocation() )will be wrong.
int col = _inputPtr - _currInputRowStart + 1; // 1-based
public UTF8StreamJsonParser(IOContext ctxt, int features, InputStream in,
ObjectCodec codec, BytesToNameCanonicalizer sym,
byte[] inputBuffer, int start, int end,
boolean bufferRecyclable)


public char[] expandCurrentSegment()
{
    final char[] curr = _currentSegment;
    // Let's grow by 50% by default
    final int len = curr.length;
    // but above intended maximum, slow to increase by 25%
    int newLen = (len == MAX_SEGMENT_LEN) ? (MAX_SEGMENT_LEN+1) : Math.min(MAX_SEGMENT_LEN, len + (len >> 1));
    return (_currentSegment = Arrays.copyOf(curr, newLen));
}

public char[] expandCurrentSegment()
{
    final char[] curr = _currentSegment;
    // Let's grow by 50% by default
    final int len = curr.length;
    int newLen = len + (len >> 1);
    // but above intended maximum, slow to increase by 25%
    if (newLen > MAX_SEGMENT_LEN) {
        newLen = len + (len >> 2);
    }
    return (_currentSegment = Arrays.copyOf(curr, newLen));
}

I noticed that even in Jackson 2.4, if a JSON key is longer than 262144 bytes, ArrayIndexOutOfBoundsException is thrown from TextBuffer. Below is the stack trace:
java.lang.ArrayIndexOutOfBoundsException
    at java.lang.System.arraycopy(Native Method)
    at com.fasterxml.jackson.core.util.TextBuffer.expandCurrentSegment(TextBuffer.java:604)
    at com.fasterxml.jackson.core.json.UTF8StreamJsonParser.addName(UTF8StreamJsonParser.java:2034)
    at com.fasterxml.jackson.core.json.UTF8StreamJsonParser.findName(UTF8StreamJsonParser.java:1928)
    at com.fasterxml.jackson.core.json.UTF8StreamJsonParser.parseLongFieldName(UTF8StreamJsonParser.java:1534)
    at com.fasterxml.jackson.core.json.UTF8StreamJsonParser.parseMediumFieldName(UTF8StreamJsonParser.java:1502)
    at com.fasterxml.jackson.core.json.UTF8StreamJsonParser._parseFieldName(UTF8StreamJsonParser.java:1437)
    at com.fasterxml.jackson.core.json.UTF8StreamJsonParser.nextToken(UTF8StreamJsonParser.java:668)
    ... <below are our code> ...

Looking at TextBuffer.expandCurrentSegment(TextBuffer.java:604), once the length of _currentSegment is increased to MAX_SEGMENT_LEN + 1 (262145) bytes, the newLen will stay at MAX_SEGMENT_LEN, which is smaller than len. Therefore System.arraycopy() will fail.
I understand it is rare to have key larger than 262144 bytes, but it would be nice if

Jackson explicitly throw exception stating that key is too long.
Document that the maximum key length is 262144 bytes.

OR

Update TextBuffer to support super long key.

Thanks!


private final static int _parseIndex(String str) {
    final int len = str.length();
    // [Issue#133]: beware of super long indexes; assume we never
    // have arrays over 2 billion entries so ints are fine.
    if (len == 0 || len > 10) {
        return -1;
    }
    for (int i = 0; i < len; ++i) {
        char c = str.charAt(i++);
        if (c > '9' || c < '0') {
            return -1;
        }
    }
    if (len == 10) {
        long l = NumberInput.parseLong(str);
        if (l > Integer.MAX_VALUE) {
            return -1;
        }
    }
    return NumberInput.parseInt(str);
}

private final static int _parseIndex(String str) {
    final int len = str.length();
    // [Issue#133]: beware of super long indexes; assume we never
    // have arrays over 2 billion entries so ints are fine.
    if (len == 0 || len > 10) {
        return -1;
    }
    for (int i = 0; i < len; ++i) {
        char c = str.charAt(i);
        if (c > '9' || c < '0') {
            return -1;
        }
    }
    if (len == 10) {
        long l = NumberInput.parseLong(str);
        if (l > Integer.MAX_VALUE) {
            return -1;
        }
    }
    return NumberInput.parseInt(str);
}

An exception is thrown for a valid JsonPointer expression

Json-Patch project leader has noted me that there is a bug on JsonPointer implementation and I have decided to investigate.
Basically if you do something like JsonPointer.compile("/1e0"); it throws a NumberFormatExpcetion which is not true. This is because this piece of code:
private final static int _parseInt(String str)
    {
        final int len = str.length();
        if (len == 0) {
            return -1;
        }
        for (int i = 0; i < len; ++i) {
            char c = str.charAt(i++);
            if (c > '9' || c < '0') {
                return -1;
            }
        }
        // for now, we'll assume 32-bit indexes are fine
        return NumberInput.parseInt(str);
    }
When they found a number it interprets the segment as integer but in reality it should be the whole expression. For this reason I think that the condition should be changed to the inverse condition  (if it doesn't found any char then it is a number.
If you want I can send you a PR as well.
Alex.


private final static int _parseIndex(String str) {
    final int len = str.length();
    // [core#133]: beware of super long indexes; assume we never
    // have arrays over 2 billion entries so ints are fine.
    if (len == 0 || len > 10) {
        return -1;
    }
    // [core#176]: no leading zeroes allowed
    for (int i = 0; i < len; ++i) {
        char c = str.charAt(i);
        if (c > '9' || c < '0') {
            return -1;
        }
    }
    if (len == 10) {
        long l = NumberInput.parseLong(str);
        if (l > Integer.MAX_VALUE) {
            return -1;
        }
    }
    return NumberInput.parseInt(str);
}

private final static int _parseIndex(String str) {
    final int len = str.length();
    // [core#133]: beware of super long indexes; assume we never
    // have arrays over 2 billion entries so ints are fine.
    if (len == 0 || len > 10) {
        return -1;
    }
    // [core#176]: no leading zeroes allowed
    char c = str.charAt(0);
    if (c <= '0') {
        return (len == 1 && c == '0') ? 0 : -1;
    }
    if (c > '9') {
        return -1;
    }
    for (int i = 1; i < len; ++i) {
        c = str.charAt(i);
        if (c > '9' || c < '0') {
            return -1;
        }
    }
    if (len == 10) {
        long l = NumberInput.parseLong(str);
        if (l > Integer.MAX_VALUE) {
            return -1;
        }
    }
    return NumberInput.parseInt(str);
}

JsonPointer should not consider "00" to be valid index

Although 00 can be parsed as 0 in some cases, it is not a valid JSON number; and is also not legal numeric index for JSON Pointer. As such, JsonPointer class should ensure it can only match property name "00" and not array index.


public int writeValue() {
    // Most likely, object:
    if (_type == TYPE_OBJECT) {
        _gotName = false;
        ++_index;
        return STATUS_OK_AFTER_COLON;
    }

    // Ok, array?
    if (_type == TYPE_ARRAY) {
        int ix = _index;
        ++_index;
        return (ix < 0) ? STATUS_OK_AS_IS : STATUS_OK_AFTER_COMMA;
    }

    // Nope, root context
    // No commas within root context, but need space
    ++_index;
    return (_index == 0) ? STATUS_OK_AS_IS : STATUS_OK_AFTER_SPACE;
}

public int writeValue() {
    // Most likely, object:
    if (_type == TYPE_OBJECT) {
        if (!_gotName) {
            return STATUS_EXPECT_NAME;
        }
        _gotName = false;
        ++_index;
        return STATUS_OK_AFTER_COLON;
    }

    // Ok, array?
    if (_type == TYPE_ARRAY) {
        int ix = _index;
        ++_index;
        return (ix < 0) ? STATUS_OK_AS_IS : STATUS_OK_AFTER_COMMA;
    }

    // Nope, root context
    // No commas within root context, but need space
    ++_index;
    return (_index == 0) ? STATUS_OK_AS_IS : STATUS_OK_AFTER_SPACE;
}

Add a check so JsonGenerator.writeString() won't work if writeFieldName() expected.

Looks like calling writeString() (and perhaps other scalar write methods) results in writing invalid output, instead of throwing an exception. It should instead fail; in future we may want to consider allowing this as an alias, but at any rate it should not produce invalid output.


public char[] getTextBuffer()
{
    // Are we just using shared input buffer?
    if (_inputStart >= 0) return _inputBuffer;
    if (_resultArray != null)  return _resultArray;
    if (_resultString != null) {
        return (_resultArray = _resultString.toCharArray());
    }
    // Nope; but does it fit in just one segment?
    if (!_hasSegments)  return _currentSegment;
    // Nope, need to have/create a non-segmented array and return it
    return contentsAsArray();
}

public char[] getTextBuffer()
{
    // Are we just using shared input buffer?
    if (_inputStart >= 0) return _inputBuffer;
    if (_resultArray != null)  return _resultArray;
    if (_resultString != null) {
        return (_resultArray = _resultString.toCharArray());
    }
    // Nope; but does it fit in just one segment?
    if (!_hasSegments && _currentSegment != null)  return _currentSegment;
    // Nope, need to have/create a non-segmented array and return it
    return contentsAsArray();
}

Hi, I'm using 2.4.2. While I'm working on CBORParser, I noticed that CBORParser#getTextCharacters() returns sometimes null sometimes [] (empty array) when it's parsing empty string "".
While debugging, I noticed that TextBuffer#getTextBuffer behaves inconsistently.
TextBuffer buffer = new TextBuffer(new BufferRecycler());
buffer.resetWithEmpty();
buffer.getTextBuffer(); // returns null
buffer.contentsAsString(); // returns empty string ""
buffer.getTextBuffer(); // returns empty array []

I think getTextBuffer should return the same value. Not sure which (null or []) is expected though.


public byte[] getBinaryValue(Base64Variant b64variant)
    throws IOException, JsonParseException
{
    // Multiple possibilities...
    JsonNode n = currentNode();
    if (n != null) {
        // [databind#2096]: although `binaryValue()` works for real binary node
        // and embedded "POJO" node, coercion from TextNode may require variant, so:
        byte[] data = n.binaryValue();
        if (data != null) {
            return data;
        }
        if (n.isPojo()) {
            Object ob = ((POJONode) n).getPojo();
            if (ob instanceof byte[]) {
                return (byte[]) ob;
            }
        }
    }
    // otherwise return null to mark we have no binary content
    return null;
}

public byte[] getBinaryValue(Base64Variant b64variant)
    throws IOException, JsonParseException
{
    // Multiple possibilities...
    JsonNode n = currentNode();
    if (n != null) {
        // [databind#2096]: although `binaryValue()` works for real binary node
        // and embedded "POJO" node, coercion from TextNode may require variant, so:
        if (n instanceof TextNode) {
            return ((TextNode) n).getBinaryValue(b64variant);
        }
        return n.binaryValue();
    }
    // otherwise return null to mark we have no binary content
    return null;
}

TreeTraversingParser does not take base64 variant into account

This affects at least 2.6.4 to current versions. In TreeTraversingParser#getBinaryValue, a Base64Variant is accepted but ignored. The call to n.binaryValue(), when n is a TextNode, then uses the default Base64 variant instead of what's specified. It seems the correct behavior would be to call TextNode#getBinaryValue instead.


public static JsonDeserializer<?> find(Class<?> rawType, String clsName)
    {
        if (_classNames.contains(clsName)) {
            JsonDeserializer<?> d = FromStringDeserializer.findDeserializer(rawType);
            if (d != null) {
                return d;
            }
            if (rawType == UUID.class) {
                return new UUIDDeserializer();
            }
            if (rawType == StackTraceElement.class) {
                return new StackTraceElementDeserializer();
            }
            if (rawType == AtomicBoolean.class) {
                // (note: AtomicInteger/Long work due to single-arg constructor. For now?
                return new AtomicBooleanDeserializer();
            }
            if (rawType == ByteBuffer.class) {
                return new ByteBufferDeserializer();
            }
        }
        return null;
    }

public static JsonDeserializer<?> find(Class<?> rawType, String clsName)
    {
        if (_classNames.contains(clsName)) {
            JsonDeserializer<?> d = FromStringDeserializer.findDeserializer(rawType);
            if (d != null) {
                return d;
            }
            if (rawType == UUID.class) {
                return new UUIDDeserializer();
            }
            if (rawType == StackTraceElement.class) {
                return new StackTraceElementDeserializer();
            }
            if (rawType == AtomicBoolean.class) {
                // (note: AtomicInteger/Long work due to single-arg constructor. For now?
                return new AtomicBooleanDeserializer();
            }
            if (rawType == ByteBuffer.class) {
                return new ByteBufferDeserializer();
            }
            if (rawType == Void.class) {
                return NullifyingDeserializer.instance;
            }
        }
        return null;
    }

Illegal reflective access operation warning when using `java.lang.Void` as value type

I'm using Jackson (**2.9.7**) through Spring's RestTemplate:

```java
ResponseEntity<Void> response = getRestTemplate().exchange(
		requestUrl,
		HttpMethod.PATCH,
		new HttpEntity<>(dto, authHeaders),
		Void.class
);
```

When [`Void`](https://docs.oracle.com/javase/7/docs/api/java/lang/Void.html) is used to indicate that the ResponseEntity has no body, the following warning appears in the console:

```
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.fasterxml.jackson.databind.util.ClassUtil (file:/<snip>repository/com/fasterxml/jackson/core/jackson-databind/2.9.7/jackson-databind-2.9.7.jar) to constructor java.lang.Void()
WARNING: Please consider reporting this to the maintainers of com.fasterxml.jackson.databind.util.ClassUtil
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
```

The problem disappears if `String` is used as generic type.


protected final JsonDeserializer<Object> _findDeserializer(DeserializationContext ctxt,
        String typeId) throws IOException
{
    JsonDeserializer<Object> deser = _deserializers.get(typeId);
    if (deser == null) {
        /* As per [databind#305], need to provide contextual info. But for
         * backwards compatibility, let's start by only supporting this
         * for base class, not via interface. Later on we can add this
         * to the interface, assuming deprecation at base class helps.
         */
        JavaType type = _idResolver.typeFromId(ctxt, typeId);
        if (type == null) {
            // use the default impl if no type id available:
            deser = _findDefaultImplDeserializer(ctxt);
            if (deser == null) {
                // 10-May-2016, tatu: We may get some help...
                JavaType actual = _handleUnknownTypeId(ctxt, typeId);
                if (actual == null) { // what should this be taken to mean?
                    // 17-Jan-2019, tatu: As per [databind#2221], better NOT return `null` but...
                    return null;
                }
                // ... would this actually work?
                deser = ctxt.findContextualValueDeserializer(actual, _property);
            }
        } else {
            /* 16-Dec-2010, tatu: Since nominal type we get here has no (generic) type parameters,
             *   we actually now need to explicitly narrow from base type (which may have parameterization)
             *   using raw type.
             *
             *   One complication, though; cannot change 'type class' (simple type to container); otherwise
             *   we may try to narrow a SimpleType (Object.class) into MapType (Map.class), losing actual
             *   type in process (getting SimpleType of Map.class which will not work as expected)
             */
            if ((_baseType != null)
                    && _baseType.getClass() == type.getClass()) {
                /* 09-Aug-2015, tatu: Not sure if the second part of the check makes sense;
                 *   but it appears to check that JavaType impl class is the same which is
                 *   important for some reason?
                 *   Disabling the check will break 2 Enum-related tests.
                 */
                // 19-Jun-2016, tatu: As per [databind#1270] we may actually get full
                //   generic type with custom type resolvers. If so, should try to retain them.
                //  Whether this is sufficient to avoid problems remains to be seen, but for
                //  now it should improve things.
                if (!type.hasGenericTypes()) {
                    type = ctxt.getTypeFactory().constructSpecializedType(_baseType, type.getRawClass());
                }
            }
            deser = ctxt.findContextualValueDeserializer(type, _property);
        }
        _deserializers.put(typeId, deser);
    }
    return deser;
}

protected final JsonDeserializer<Object> _findDeserializer(DeserializationContext ctxt,
        String typeId) throws IOException
{
    JsonDeserializer<Object> deser = _deserializers.get(typeId);
    if (deser == null) {
        /* As per [databind#305], need to provide contextual info. But for
         * backwards compatibility, let's start by only supporting this
         * for base class, not via interface. Later on we can add this
         * to the interface, assuming deprecation at base class helps.
         */
        JavaType type = _idResolver.typeFromId(ctxt, typeId);
        if (type == null) {
            // use the default impl if no type id available:
            deser = _findDefaultImplDeserializer(ctxt);
            if (deser == null) {
                // 10-May-2016, tatu: We may get some help...
                JavaType actual = _handleUnknownTypeId(ctxt, typeId);
                if (actual == null) { // what should this be taken to mean?
                    // 17-Jan-2019, tatu: As per [databind#2221], better NOT return `null` but...
                    return NullifyingDeserializer.instance;
                }
                // ... would this actually work?
                deser = ctxt.findContextualValueDeserializer(actual, _property);
            }
        } else {
            /* 16-Dec-2010, tatu: Since nominal type we get here has no (generic) type parameters,
             *   we actually now need to explicitly narrow from base type (which may have parameterization)
             *   using raw type.
             *
             *   One complication, though; cannot change 'type class' (simple type to container); otherwise
             *   we may try to narrow a SimpleType (Object.class) into MapType (Map.class), losing actual
             *   type in process (getting SimpleType of Map.class which will not work as expected)
             */
            if ((_baseType != null)
                    && _baseType.getClass() == type.getClass()) {
                /* 09-Aug-2015, tatu: Not sure if the second part of the check makes sense;
                 *   but it appears to check that JavaType impl class is the same which is
                 *   important for some reason?
                 *   Disabling the check will break 2 Enum-related tests.
                 */
                // 19-Jun-2016, tatu: As per [databind#1270] we may actually get full
                //   generic type with custom type resolvers. If so, should try to retain them.
                //  Whether this is sufficient to avoid problems remains to be seen, but for
                //  now it should improve things.
                if (!type.hasGenericTypes()) {
                    type = ctxt.getTypeFactory().constructSpecializedType(_baseType, type.getRawClass());
                }
            }
            deser = ctxt.findContextualValueDeserializer(type, _property);
        }
        _deserializers.put(typeId, deser);
    }
    return deser;
}

DeserializationProblemHandler.handleUnknownTypeId() returning Void.class, enableDefaultTyping causing NPE

Returning Void.class from com.fasterxml.jackson.databind.deser.HandleUnknowTypeIdTest.testDeserializationWithDeserializationProblemHandler().new DeserializationProblemHandler() {...}.handleUnknownTypeId(DeserializationContext, JavaType, String, TypeIdResolver, String) is causing a NPE in jackson 2.9. I'll provide a pull request illustrating the issue in a test.


@SuppressWarnings("unchecked")
    @Override
    public <T extends TreeNode> T readTree(JsonParser p) throws IOException {
        return (T) _bindAsTree(p);
    }

@SuppressWarnings("unchecked")
    @Override
    public <T extends TreeNode> T readTree(JsonParser p) throws IOException {
        return (T) _bindAsTreeOrNull(p);
    }

Change of behavior (2.8 -> 2.9) with `ObjectMapper.readTree(input)` with no content

So, it looks like `readTree()` methods in `ObjectMapper`, `ObjectReader` that take input OTHER than `JsonParser`, and are given "empty input" (only white-space available before end), will

* Return `NullNode` (Jackson 2.x up to and including 2.8)
* Return `null` (Jackson 2.9)

Latter behavior is what `readTree(JsonParser)` has and will do; but this accidentally changed other methods due to refactoring that unified underlying call handling (and add checking for new `DeserializationFeature.FAIL_ON_TRAILING_TOKENS`).
Behavior for this edge case was not being tested, apparently.

Now: since behavior has been changed for all 2.9.x patch versions, I am not sure it should be changed for 2.9 branch. But it seems sub-optimal as behavior, and something to definitely change for 3.0... but probably also for 2.10.

There are multiple things we could do.

1. Change it back to 2.8, to return `NullNode`
2. Change to throw exception, as "not valid" use case
3. Change it to return `MissingNode`
4. Leave as-is, for rest of 2.x.

Although it might seem best to revert it to (1), that seems somewhat wrong, problematic, as it would now not be possible to distinguish between JSON `null` value and missing content.
And although (2) would probably make sense, if designing API from scratch, it is probably too intrusive.

So I think (3) is the best way: it avoids returning `null` or throwing Exception (both being likely to break 2.9 code), but still allows distinguishing between all possible input cases.


protected JavaType _fromVariable(TypeVariable<?> type, TypeBindings context)
{
    final String name = type.getName();
    // 19-Mar-2015: Without context, all we can check are bounds.
    if (context == null) {
        // And to prevent infinite loops, now need this:
        return _unknownType();
    } else {
        // Ok: here's where context might come in handy!
        /* 19-Mar-2015, tatu: As per [databind#609], may need to allow
         *   unresolved type variables to handle some cases where bounds
         *   are enough. Let's hope it does not hide real fail cases.
         */
        JavaType actualType = context.findType(name);
        if (actualType != null) {
            return actualType;
        }
    }

    /* 29-Jan-2010, tatu: We used to throw exception here, if type was
     *   bound: but the problem is that this can occur for generic "base"
     *   method, overridden by sub-class. If so, we will want to ignore
     *   current type (for method) since it will be masked.
     */
    Type[] bounds = type.getBounds();

    // With type variables we must use bound information.
    // Theoretically this gets tricky, as there may be multiple
    // bounds ("... extends A & B"); and optimally we might
    // want to choose the best match. Also, bounds are optional;
    // but here we are lucky in that implicit "Object" is
    // added as bounds if so.
    // Either way let's just use the first bound, for now, and
    // worry about better match later on if there is need.

    /* 29-Jan-2010, tatu: One more problem are recursive types
     *   (T extends Comparable<T>). Need to add "placeholder"
     *   for resolution to catch those.
     */
    context._addPlaceholder(name);
    return _constructType(bounds[0], context);
}

protected JavaType _fromVariable(TypeVariable<?> type, TypeBindings context)
{
    final String name = type.getName();
    // 19-Mar-2015: Without context, all we can check are bounds.
    if (context == null) {
        // And to prevent infinite loops, now need this:
        context = new TypeBindings(this, (Class<?>) null);
    } else {
        // Ok: here's where context might come in handy!
        /* 19-Mar-2015, tatu: As per [databind#609], may need to allow
         *   unresolved type variables to handle some cases where bounds
         *   are enough. Let's hope it does not hide real fail cases.
         */
        JavaType actualType = context.findType(name, false);
        if (actualType != null) {
            return actualType;
        }
    }

    /* 29-Jan-2010, tatu: We used to throw exception here, if type was
     *   bound: but the problem is that this can occur for generic "base"
     *   method, overridden by sub-class. If so, we will want to ignore
     *   current type (for method) since it will be masked.
     */
    Type[] bounds = type.getBounds();

    // With type variables we must use bound information.
    // Theoretically this gets tricky, as there may be multiple
    // bounds ("... extends A & B"); and optimally we might
    // want to choose the best match. Also, bounds are optional;
    // but here we are lucky in that implicit "Object" is
    // added as bounds if so.
    // Either way let's just use the first bound, for now, and
    // worry about better match later on if there is need.

    /* 29-Jan-2010, tatu: One more problem are recursive types
     *   (T extends Comparable<T>). Need to add "placeholder"
     *   for resolution to catch those.
     */
    context._addPlaceholder(name);
    return _constructType(bounds[0], context);
}

Problem resolving locally declared generic type

(reported by Hal H)
Case like:
class Something {
    public <T extends Ruleform> T getEntity()
    public <T extends Ruleform> void setEntity(T entity)
}
appears to fail on deserialization.


public JsonDeserializer<?> createContextual(DeserializationContext ctxt,
        BeanProperty property) throws JsonMappingException
{
    // May need to resolve types for delegate-based creators:
    JsonDeserializer<Object> delegate = null;
    if (_valueInstantiator != null) {
        // [databind#2324]: check both array-delegating and delegating
        AnnotatedWithParams delegateCreator = _valueInstantiator.getDelegateCreator();
        if (delegateCreator != null) {
            JavaType delegateType = _valueInstantiator.getDelegateType(ctxt.getConfig());
            delegate = findDeserializer(ctxt, delegateType, property);
        }
    }
    JsonDeserializer<?> valueDeser = _valueDeserializer;
    final JavaType valueType = _containerType.getContentType();
    if (valueDeser == null) {
        // [databind#125]: May have a content converter
        valueDeser = findConvertingContentDeserializer(ctxt, property, valueDeser);
        if (valueDeser == null) {
        // And we may also need to get deserializer for String
            valueDeser = ctxt.findContextualValueDeserializer(valueType, property);
        }
    } else { // if directly assigned, probably not yet contextual, so:
        valueDeser = ctxt.handleSecondaryContextualization(valueDeser, property, valueType);
    }
    // 11-Dec-2015, tatu: Should we pass basic `Collection.class`, or more refined? Mostly
    //   comes down to "List vs Collection" I suppose... for now, pass Collection
    Boolean unwrapSingle = findFormatFeature(ctxt, property, Collection.class,
            JsonFormat.Feature.ACCEPT_SINGLE_VALUE_AS_ARRAY);
    NullValueProvider nuller = findContentNullProvider(ctxt, property, valueDeser);
    if (isDefaultDeserializer(valueDeser)) {
        valueDeser = null;
    }
    return withResolved(delegate, valueDeser, nuller, unwrapSingle);
}

public JsonDeserializer<?> createContextual(DeserializationContext ctxt,
        BeanProperty property) throws JsonMappingException
{
    // May need to resolve types for delegate-based creators:
    JsonDeserializer<Object> delegate = null;
    if (_valueInstantiator != null) {
        // [databind#2324]: check both array-delegating and delegating
        AnnotatedWithParams delegateCreator = _valueInstantiator.getArrayDelegateCreator();
        if (delegateCreator != null) {
            JavaType delegateType = _valueInstantiator.getArrayDelegateType(ctxt.getConfig());
            delegate = findDeserializer(ctxt, delegateType, property);
        } else if ((delegateCreator = _valueInstantiator.getDelegateCreator()) != null) {
            JavaType delegateType = _valueInstantiator.getDelegateType(ctxt.getConfig());
            delegate = findDeserializer(ctxt, delegateType, property);
        }
    }
    JsonDeserializer<?> valueDeser = _valueDeserializer;
    final JavaType valueType = _containerType.getContentType();
    if (valueDeser == null) {
        // [databind#125]: May have a content converter
        valueDeser = findConvertingContentDeserializer(ctxt, property, valueDeser);
        if (valueDeser == null) {
        // And we may also need to get deserializer for String
            valueDeser = ctxt.findContextualValueDeserializer(valueType, property);
        }
    } else { // if directly assigned, probably not yet contextual, so:
        valueDeser = ctxt.handleSecondaryContextualization(valueDeser, property, valueType);
    }
    // 11-Dec-2015, tatu: Should we pass basic `Collection.class`, or more refined? Mostly
    //   comes down to "List vs Collection" I suppose... for now, pass Collection
    Boolean unwrapSingle = findFormatFeature(ctxt, property, Collection.class,
            JsonFormat.Feature.ACCEPT_SINGLE_VALUE_AS_ARRAY);
    NullValueProvider nuller = findContentNullProvider(ctxt, property, valueDeser);
    if (isDefaultDeserializer(valueDeser)) {
        valueDeser = null;
    }
    return withResolved(delegate, valueDeser, nuller, unwrapSingle);
}

StringCollectionDeserializer fails with custom collection

Seeing this with Jackson 2.9.8.
We have a custom collection implementation, which is wired to use its "immutable" version for deserialization. The rationale is that we don't want accidental modifications to the data structures that come from the wire, so they all are forced to be immutable.
After upgrade from 2.6.3 to 2.9.8, the deserialization started breaking with the message:

Cannot construct instance of XXX (although at least one Creator exists): no default no-arguments constructor found

This happens ONLY when you deserialize a custom collection of strings as a property of the other object. Deserializing the custom collection of strings directly works fine, and so does the deserialization of custom collection of non-strings. I believe either the StringCollectionDeserializer should not be invoked for custom collections, or perhaps it does not handle the delegation as expected.
Please see comments for repro and workaround.
Thanks!


public boolean isCachable() {
    /* As per [databind#735], existence of value or key deserializer (only passed
     * if annotated to use non-standard one) should also prevent caching.
     */
    return (_valueTypeDeserializer == null)
            && (_ignorableProperties == null);
}

public boolean isCachable() {
    /* As per [databind#735], existence of value or key deserializer (only passed
     * if annotated to use non-standard one) should also prevent caching.
     */
    return (_valueDeserializer == null)
            && (_keyDeserializer == null)
            && (_valueTypeDeserializer == null)
            && (_ignorableProperties == null);
}

@JsonDeserialize on Map with contentUsing custom deserializer overwrites default behavior

I recently updated from version 2.3.3 to 2.5.1 and encountered a new issue with our custom deserializers. They either seemed to stop working or were active on the wrong fields.
I could narrow it down to some change in version 2.4.4 (2.4.3 is still working for me)
I wrote a test to show this behavior. It seems to appear when there a two maps with the same key and value types in a bean, and only one of them has a custom deserializer. The deserializer is then falsely used either for both or none of the maps.
This test works for me in version 2.4.3 and fails with higher versions.
import static org.junit.Assert.assertEquals;

import java.io.IOException;
import java.util.Map;

import org.junit.Test;

import com.fasterxml.jackson.annotation.JsonProperty;
import com.fasterxml.jackson.core.JsonParser;
import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.DeserializationContext;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.annotation.JsonDeserialize;
import com.fasterxml.jackson.databind.deser.std.StdDeserializer;

public class DeserializeTest {

    @Test
    public void testIt() throws Exception {
        ObjectMapper om = new ObjectMapper();
        String json = "{\"map1\":{\"a\":1},\"map2\":{\"a\":1}}";
        TestBean bean = om.readValue(json.getBytes(), TestBean.class);

        assertEquals(100, bean.getMap1().get("a").intValue());
        assertEquals(1, bean.getMap2().get("a").intValue());
    }

    public static class TestBean {

        @JsonProperty("map1")
        @JsonDeserialize(contentUsing = CustomDeserializer.class)
        Map<String, Integer> map1;

        @JsonProperty("map2")
        Map<String, Integer> map2;

        public Map<String, Integer> getMap1() {
            return map1;
        }

        public void setMap1(Map<String, Integer> map1) {
            this.map1 = map1;
        }

        public Map<String, Integer> getMap2() {
            return map2;
        }

        public void setMap2(Map<String, Integer> map2) {
            this.map2 = map2;
        }
    }

    public static class CustomDeserializer extends StdDeserializer<Integer> {

        public CustomDeserializer() {
            super(Integer.class);
        }

        @Override
        public Integer deserialize(JsonParser p, DeserializationContext ctxt) throws IOException, JsonProcessingException {
            Integer value = p.readValueAs(Integer.class);
            return value * 100;
        }
    }
}


protected final boolean _add(Annotation ann) {
    if (_annotations == null) {
        _annotations = new HashMap<Class<? extends Annotation>,Annotation>();
    }
    Annotation previous = _annotations.put(ann.annotationType(), ann);
    return (previous != null) && previous.equals(ann);
}

protected final boolean _add(Annotation ann) {
    if (_annotations == null) {
        _annotations = new HashMap<Class<? extends Annotation>,Annotation>();
    }
    Annotation previous = _annotations.put(ann.annotationType(), ann);
    return (previous == null) || !previous.equals(ann);
}

Annotation bundles ignored when added to Mixin

When updating from v 2.4.4 to 2.5.* it appears as though annotation bundles created with @JacksonAnnotationsInside are ignored when placed on a mixin.  Moving the annotation bundel to the actual class seems to resolve the issue.  Below is a simple test that attempts to rename a property.  I have more complicated test cases that are also failing but this should provide some context.
public class Fun {

    @Test
    public void test() throws JsonProcessingException {
        ObjectMapper mapper = new ObjectMapper().addMixIn(Foo.class, FooMixin.class);
        String result = mapper.writeValueAsString(new Foo("result"));
        Assert.assertEquals("{\"bar\":\"result\"}", result);
    }

    @Target(value={ ElementType.CONSTRUCTOR, ElementType.FIELD, ElementType.METHOD })
    @Retention(value=RetentionPolicy.RUNTIME)
    @JacksonAnnotationsInside
    @JsonProperty("bar")
    public @interface ExposeStuff {

    }

    public abstract class FooMixin {
        @ExposeStuff
        public abstract String getStuff();
    }

    public class Foo {

        private String stuff;

        Foo(String stuff) {
            this.stuff = stuff;
        }

        public String getStuff() {
            return stuff;
        }
    }
}
I'm expecting the "stuff" property to be serialized as "bar".
I apologize I haven't been able to identify the culprit (and perhaps it's in my usage).  Let me know your thoughts. I'm always happy to provide more details!


private JavaType _mapType(Class<?> rawClass)
{
    // 28-May-2015, tatu: Properties are special, as per [databind#810]
    JavaType[] typeParams = findTypeParameters(rawClass, Map.class);
    // ok to have no types ("raw")
    if (typeParams == null) {
        return MapType.construct(rawClass, _unknownType(), _unknownType());
    }
    // but exactly 2 types if any found
    if (typeParams.length != 2) {
        throw new IllegalArgumentException("Strange Map type "+rawClass.getName()+": can not determine type parameters");
    }
    return MapType.construct(rawClass, typeParams[0], typeParams[1]);
}

private JavaType _mapType(Class<?> rawClass)
{
    // 28-May-2015, tatu: Properties are special, as per [databind#810]
    if (rawClass == Properties.class) {
        return MapType.construct(rawClass, CORE_TYPE_STRING, CORE_TYPE_STRING);
    }
    JavaType[] typeParams = findTypeParameters(rawClass, Map.class);
    // ok to have no types ("raw")
    if (typeParams == null) {
        return MapType.construct(rawClass, _unknownType(), _unknownType());
    }
    // but exactly 2 types if any found
    if (typeParams.length != 2) {
        throw new IllegalArgumentException("Strange Map type "+rawClass.getName()+": can not determine type parameters");
    }
    return MapType.construct(rawClass, typeParams[0], typeParams[1]);
}

Force value coercion for java.util.Properties, so that values are Strings

Currently there is no custom handling for java.util.Properties, and although it is possible to use it (since it really is a Map under the hood), results are only good if values are already Strings.
The problem here is that Properties is actually declared as Map<String,Object>, probably due to backwards-compatibility constraints.
But Jackson should know better: perhaps by TypeFactory tweaking parameterizations a bit?


@JsonIgnore // work-around for [databind#815]
    public JsonNode setAll(Map<String,? extends JsonNode> properties)
    {
        for (Map.Entry<String,? extends JsonNode> en : properties.entrySet()) {
            JsonNode n = en.getValue();
            if (n == null) {
                n = nullNode();
            }
            _children.put(en.getKey(), n);
        }
        return this;
    }

Presence of PropertyNamingStrategy Makes Deserialization Fail

I originally came across this issue using Dropwizard - https://github.com/dropwizard/dropwizard/issues/1095.  But it looks like this is a Jackson issue.  Here's the rerproducer:

``` java
public class TestPropertyNamingStrategyIssue {
  public static class ClassWithObjectNodeField {
    public String id;
    public ObjectNode json;
  }

  @Test
  public void reproducer() throws Exception {
    ObjectMapper mapper = new ObjectMapper();
    mapper.setPropertyNamingStrategy(PropertyNamingStrategy.LOWER_CASE);
    ClassWithObjectNodeField deserialized =
        mapper.readValue(
            "{ \"id\": \"1\", \"json\": { \"foo\": \"bar\", \"baz\": \"bing\" } }",
            ClassWithObjectNodeField.class);
  }
}
```

Looks like the presence of any PropertyNamingStrategy make deserialization to ObjectNode fail.  This works fine if I remove the property naming strategy.


public BaseSettings withDateFormat(DateFormat df) {
    if (_dateFormat == df) {
        return this;
    }
    TimeZone tz = (df == null) ? _timeZone : df.getTimeZone();
    return new BaseSettings(_classIntrospector, _annotationIntrospector, _visibilityChecker, _propertyNamingStrategy, _typeFactory,
            _typeResolverBuilder, df, _handlerInstantiator, _locale,
            tz, _defaultBase64);
}

public BaseSettings withDateFormat(DateFormat df) {
    if (_dateFormat == df) {
        return this;
    }
    return new BaseSettings(_classIntrospector, _annotationIntrospector, _visibilityChecker, _propertyNamingStrategy, _typeFactory,
            _typeResolverBuilder, df, _handlerInstantiator, _locale,
            _timeZone, _defaultBase64);
}

Configuring an ObjectMapper's DateFormat changes time zone when serialising Joda DateTime

The serialisation of Joda DateTime instances behaves differently in 2.6.0 vs 2.5.4 when the ObjectMapper's had its DateFormat configured. The behaviour change is illustrated by the following code:
public static void main(String[] args) throws JsonProcessingException {
    System.out.println(createObjectMapper()
            .writeValueAsString(new DateTime(1988, 6, 25, 20, 30, DateTimeZone.UTC)));
}

private static ObjectMapper createObjectMapper() {
    ObjectMapper mapper = new ObjectMapper();
    mapper.registerModule(createJodaModule());
    mapper.configure(SerializationFeature.WRITE_DATES_AS_TIMESTAMPS, false);
    System.out.println(mapper.getSerializationConfig().getTimeZone());
    mapper.setDateFormat(new SimpleDateFormat("yyyy-MM-dd HH:mm:ss"));
    System.out.println(mapper.getSerializationConfig().getTimeZone());
    return mapper;
}

private static SimpleModule createJodaModule() {
    SimpleModule module = new SimpleModule();
    module.addSerializer(DateTime.class, new DateTimeSerializer(
            new JacksonJodaDateFormat(DateTimeFormat.forPattern("yyyy-MM-dd HH:mm:ss")
                    .withZoneUTC())));
        return module;
    }
When run with Jackson 2.5.4 the output is:
sun.util.calendar.ZoneInfo[id="GMT",offset=0,dstSavings=0,useDaylight=false,transitions=0,lastRule=null]
sun.util.calendar.ZoneInfo[id="GMT",offset=0,dstSavings=0,useDaylight=false,transitions=0,lastRule=null]
"1988-06-25 20:30:00"

When run with Jackson 2.6.0 the output is:
sun.util.calendar.ZoneInfo[id="GMT",offset=0,dstSavings=0,useDaylight=false,transitions=0,lastRule=null]
sun.util.calendar.ZoneInfo[id="Europe/London",offset=0,dstSavings=3600000,useDaylight=true,transitions=242,lastRule=java.util.SimpleTimeZone[id=Europe/London,offset=0,dstSavings=3600000,useDaylight=true,startYear=0,startMode=2,startMonth=2,startDay=-1,startDayOfWeek=1,startTime=3600000,startTimeMode=2,endMode=2,endMonth=9,endDay=-1,endDayOfWeek=1,endTime=3600000,endTimeMode=2]]
"1988-06-25 21:30:00"

It looks like the fix for #824 is the cause. In 2.6, the call to mapper.setDateFormat causes the ObjectMapper's time zone to be set to the JVM's default time zone. In 2.5.x, calling mapper.setDateFormat has no effect on its time zone.


public PropertyName findNameForSerialization(Annotated a)
{
    String name = null;

    JsonGetter jg = _findAnnotation(a, JsonGetter.class);
    if (jg != null) {
        name = jg.value();
    } else {
        JsonProperty pann = _findAnnotation(a, JsonProperty.class);
        if (pann != null) {
            name = pann.value();
            /* 22-Apr-2014, tatu: Should figure out a better way to do this, but
             *   it's actually bit tricky to do it more efficiently (meta-annotations
             *   add more lookups; AnnotationMap costs etc)
             */
        } else if (_hasAnnotation(a, JsonSerialize.class)
                || _hasAnnotation(a, JsonView.class)
                || _hasAnnotation(a, JsonRawValue.class)) {
            name = "";
        } else {
            return null;
        }
    }
    return PropertyName.construct(name);
}

public PropertyName findNameForSerialization(Annotated a)
{
    String name = null;

    JsonGetter jg = _findAnnotation(a, JsonGetter.class);
    if (jg != null) {
        name = jg.value();
    } else {
        JsonProperty pann = _findAnnotation(a, JsonProperty.class);
        if (pann != null) {
            name = pann.value();
            /* 22-Apr-2014, tatu: Should figure out a better way to do this, but
             *   it's actually bit tricky to do it more efficiently (meta-annotations
             *   add more lookups; AnnotationMap costs etc)
             */
        } else if (_hasAnnotation(a, JsonSerialize.class)
                || _hasAnnotation(a, JsonView.class)
                || _hasAnnotation(a, JsonRawValue.class)
                || _hasAnnotation(a, JsonUnwrapped.class)
                || _hasAnnotation(a, JsonBackReference.class)
                || _hasAnnotation(a, JsonManagedReference.class)) {
            name = "";
        } else {
            return null;
        }
    }
    return PropertyName.construct(name);
}

@JsonUnwrapped is not treated as assuming @JsonProperty("")

See discussion here but basically @JsonUnwrapped on a private field by itself does not cause that field to be serialized, currently,  You need to add an explicit @JsonProperty.  You shouldn't have to do that.  (Following test fails currently, should pass, though you can make it pass by commenting out the line with @JsonProperty.  Uses TestNG and AssertJ.)
package com.bakins_bits;

import static org.assertj.core.api.Assertions.assertThat;

import org.testng.annotations.Test;

import com.fasterxml.jackson.annotation.JsonUnwrapped;
import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.ObjectMapper;

public class TestJsonUnwrappedShouldMakePrivateFieldsSerializable
{
    public static class Inner
    {
        public String animal;
    }

    public static class Outer
    {
        // @JsonProperty
        @JsonUnwrapped
        private Inner inner;
    }

    @Test
    public void jsonUnwrapped_should_make_private_fields_serializable() throws JsonProcessingException {
        // ARRANGE
        Inner inner = new Inner();
        inner.animal = "Zebra";

        Outer outer = new Outer();
        outer.inner = inner;

        ObjectMapper sut = new ObjectMapper();

        // ACT
        String actual = sut.writeValueAsString(outer);

        // ASSERT
        assertThat(actual).contains("animal");
        assertThat(actual).contains("Zebra");
        assertThat(actual).doesNotContain("inner");
    }
}


public void acceptJsonFormatVisitor(JsonFormatVisitorWrapper visitor, JavaType typeHint) throws JsonMappingException
{
    if (_isInt) {
        visitIntFormat(visitor, typeHint, JsonParser.NumberType.BIG_INTEGER);
    } else {
        Class<?> h = handledType();
        if (h == BigDecimal.class) {
            visitFloatFormat(visitor, typeHint, JsonParser.NumberType.BIG_INTEGER);
        } else {
            // otherwise bit unclear what to call... but let's try:
            /*JsonNumberFormatVisitor v2 =*/ visitor.expectNumberFormat(typeHint);
        }
    }
}

public void acceptJsonFormatVisitor(JsonFormatVisitorWrapper visitor, JavaType typeHint) throws JsonMappingException
{
    if (_isInt) {
        visitIntFormat(visitor, typeHint, JsonParser.NumberType.BIG_INTEGER);
    } else {
        Class<?> h = handledType();
        if (h == BigDecimal.class) {
            visitFloatFormat(visitor, typeHint, JsonParser.NumberType.BIG_DECIMAL);
        } else {
            // otherwise bit unclear what to call... but let's try:
            /*JsonNumberFormatVisitor v2 =*/ visitor.expectNumberFormat(typeHint);
        }
    }
}

Regression in 2.7.0-rc2, for schema/introspection for BigDecimal

(found via Avro module, but surprisingly json schema module has not test to catch it)
Looks like schema type for BigDecimal is not correctly produced, due to an error in refactoring (made to simplify introspection for simple serializers): it is seen as BigInteger (and for Avro, for example, results in long getting written).


private final Object _deserialize(JsonParser p, DeserializationContext ctxt) throws IOException
{
    // 02-Aug-2013, tatu: May need to use native type ids
    if (p.canReadTypeId()) {
        Object typeId = p.getTypeId();
        if (typeId != null) {
            return _deserializeWithNativeTypeId(p, ctxt, typeId);
        }
    }
    // first, sanity checks
    if (p.getCurrentToken() != JsonToken.START_OBJECT) {
        throw ctxt.wrongTokenException(p, JsonToken.START_OBJECT,
                "need JSON Object to contain As.WRAPPER_OBJECT type information for class "+baseTypeName());
    }
        // should always get field name, but just in case...
        if (p.nextToken() != JsonToken.FIELD_NAME) {
            throw ctxt.wrongTokenException(p, JsonToken.FIELD_NAME,
                    "need JSON String that contains type id (for subtype of "+baseTypeName()+")");
        }
    final String typeId = p.getText();
    JsonDeserializer<Object> deser = _findDeserializer(ctxt, typeId);
    p.nextToken();

    // Minor complication: we may need to merge type id in?
    if (_typeIdVisible && p.getCurrentToken() == JsonToken.START_OBJECT) {
        // but what if there's nowhere to add it in? Error? Or skip? For now, skip.
        TokenBuffer tb = new TokenBuffer(null, false);
        tb.writeStartObject(); // recreate START_OBJECT
        tb.writeFieldName(_typePropertyName);
        tb.writeString(typeId);
        p = JsonParserSequence.createFlattened(tb.asParser(p), p);
        p.nextToken();
    }

    Object value = deser.deserialize(p, ctxt);
    // And then need the closing END_OBJECT
    if (p.nextToken() != JsonToken.END_OBJECT) {
        throw ctxt.wrongTokenException(p, JsonToken.END_OBJECT,
                "expected closing END_OBJECT after type information and deserialized value");
    }
    return value;
}

private final Object _deserialize(JsonParser p, DeserializationContext ctxt) throws IOException
{
    // 02-Aug-2013, tatu: May need to use native type ids
    if (p.canReadTypeId()) {
        Object typeId = p.getTypeId();
        if (typeId != null) {
            return _deserializeWithNativeTypeId(p, ctxt, typeId);
        }
    }
    // first, sanity checks
    JsonToken t = p.getCurrentToken();
    if (t == JsonToken.START_OBJECT) {
        // should always get field name, but just in case...
        if (p.nextToken() != JsonToken.FIELD_NAME) {
            throw ctxt.wrongTokenException(p, JsonToken.FIELD_NAME,
                    "need JSON String that contains type id (for subtype of "+baseTypeName()+")");
        }
    } else if (t != JsonToken.FIELD_NAME) {
        throw ctxt.wrongTokenException(p, JsonToken.START_OBJECT,
                "need JSON Object to contain As.WRAPPER_OBJECT type information for class "+baseTypeName());
    }
    final String typeId = p.getText();
    JsonDeserializer<Object> deser = _findDeserializer(ctxt, typeId);
    p.nextToken();

    // Minor complication: we may need to merge type id in?
    if (_typeIdVisible && p.getCurrentToken() == JsonToken.START_OBJECT) {
        // but what if there's nowhere to add it in? Error? Or skip? For now, skip.
        TokenBuffer tb = new TokenBuffer(null, false);
        tb.writeStartObject(); // recreate START_OBJECT
        tb.writeFieldName(_typePropertyName);
        tb.writeString(typeId);
        p = JsonParserSequence.createFlattened(tb.asParser(p), p);
        p.nextToken();
    }

    Object value = deser.deserialize(p, ctxt);
    // And then need the closing END_OBJECT
    if (p.nextToken() != JsonToken.END_OBJECT) {
        throw ctxt.wrongTokenException(p, JsonToken.END_OBJECT,
                "expected closing END_OBJECT after type information and deserialized value");
    }
    return value;
}

Problem with Object Id and Type Id as Wrapper Object (regression in 2.5.1)

(note: originally from FasterXML/jackson-module-jaxb-annotations#51)
Looks like fix for #669 caused a regression for the special use case of combining type and object ids, with wrapper-object type id inclusion. The problem started with 2.5.1.


private final static DateFormat _cloneFormat(DateFormat df, String format,
            TimeZone tz, Locale loc, Boolean lenient)
    {
        if (!loc.equals(DEFAULT_LOCALE)) {
            df = new SimpleDateFormat(format, loc);
            df.setTimeZone((tz == null) ? DEFAULT_TIMEZONE : tz);
        } else {
            df = (DateFormat) df.clone();
            if (tz != null) {
                df.setTimeZone(tz);
            }
        }
        return df;
    }

private final static DateFormat _cloneFormat(DateFormat df, String format,
            TimeZone tz, Locale loc, Boolean lenient)
    {
        if (!loc.equals(DEFAULT_LOCALE)) {
            df = new SimpleDateFormat(format, loc);
            df.setTimeZone((tz == null) ? DEFAULT_TIMEZONE : tz);
        } else {
            df = (DateFormat) df.clone();
            if (tz != null) {
                df.setTimeZone(tz);
            }
        }
        if (lenient != null) {
            df.setLenient(lenient.booleanValue());
        }
        return df;
    }

Allow use	of `StdDateFormat.setLenient()`

ObjectMapper uses the StdDateFormat for date serialization. Jackson date parsing is lenient by default, so 2015-01-32 gets parsed as 2015-02-01.  Jackson’s StdDateParser is matching default behavior of DateParser.

StdDateParser wasn’t really designed for extension to just enable strict date parsing.  If it were, we could just call objectMapper.setDateFormat(new StdDateFormat().setLenient(false)). But StdDateFomrat doesn't support setting lenient to false. And i.e. the reason date like 2015-01-32 gets parsed as 2015-02-01 ad Jackson date parsing is lenient by defualt.

Can StdDateFormat can be enhanced to support to non lenient date parsing?


public Object deserialize(JsonParser p, DeserializationContext ctxt) throws IOException
{
    // 29-Jan-2016, tatu: Simple skipping for all other tokens, but FIELD_NAME bit
    //    special unfortunately
        p.skipChildren();
    return null;
}

public Object deserialize(JsonParser p, DeserializationContext ctxt) throws IOException
{
    // 29-Jan-2016, tatu: Simple skipping for all other tokens, but FIELD_NAME bit
    //    special unfortunately
    if (p.hasToken(JsonToken.FIELD_NAME)) {
        while (true) {
            JsonToken t = p.nextToken();
            if ((t == null) || (t == JsonToken.END_OBJECT)) {
                break;
            }
            p.skipChildren();
        }
    } else {
        p.skipChildren();
    }
    return null;
}

Jackson not continue to parse after DeserializationFeature.FAIL_ON_INVALID_SUBTYPE error

After FAIL_ON_INVALID_SUBTYPE error, jackson should continue to parse, but seems jackson doesn't.
The output:
CallRecord [version=0.0, application=123, ] // doesn't read item2 which is valid
CallRecord [version=0.0, application=123, ]
CallRecord [version=0.0, ] // doesn't read application after invalid item.

@JsonInclude(Include.NON_NULL)
public class CallRecord {
    public float version;
    public String application;
    public Item item;
    public Item item2;
    public CallRecord() {}

    public static void main(final String[] args) throws IOException {
        final ObjectMapper objectMapper = new ObjectMapper().disable(DeserializationFeature.FAIL_ON_INVALID_SUBTYPE,
                DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, DeserializationFeature.FAIL_ON_IGNORED_PROPERTIES);
        final CallRecord call = new CallRecord();

        final Event event = new Event();
        event.location = "location1";
        call.item = event;
        call.item2 = event;
        call.application = "123";
        // System.out.println(objectMapper.writeValueAsString(call));
        String json =
                "{\"version\":0.0,\"application\":\"123\",\"item\":{\"type\":\"xevent\",\"location\":\"location1\"},\"item2\":{\"type\":\"event\",\"location\":\"location1\"}}";
        // can't read item2 - which is valid
        System.out.println(objectMapper.readValue(json, CallRecord.class));

        json = "{\"version\":0.0,\"application\":\"123\"},{\"item\":{\"type\":\"xevent\",\"location\":\"location1\"}";
        System.out.println(objectMapper.readValue(json, CallRecord.class));

        json = "{\"item\":{\"type\":\"xevent\",\"location\":\"location1\"}, \"version\":0.0,\"application\":\"123\"}";
        // order matters: move item to the fornt, now it can't read application property
        System.out.println(objectMapper.readValue(json, CallRecord.class));
    }
    @Override
    public String toString() {
        final StringBuilder builder = new StringBuilder();
        builder.append("CallRecord [version=").append(version).append(", ");
        if (application != null) {
            builder.append("application=").append(application).append(", ");
        }
        if (item != null) {
            builder.append("item=").append(item);
        }
        builder.append("]");
        return builder.toString();
    }
}

@JsonTypeInfo(use = JsonTypeInfo.Id.NAME, include = JsonTypeInfo.As.PROPERTY, property = "type", visible = true)
@JsonSubTypes({@Type(value = Event.class, name = Event.TYPE)})
public interface Item {
}

public final class Event implements Item {
    public String location;
    public static final String TYPE = "event";
    public Event() {}
}


protected Object _deserializeFromEmptyString() throws IOException {
    // As per [databind#398], URI requires special handling
    if (_kind == STD_URI) {
        return URI.create("");
    }
    // As per [databind#1123], Locale too
    return super._deserializeFromEmptyString();
}

protected Object _deserializeFromEmptyString() throws IOException {
    // As per [databind#398], URI requires special handling
    if (_kind == STD_URI) {
        return URI.create("");
    }
    // As per [databind#1123], Locale too
    if (_kind == STD_LOCALE) {
        return Locale.ROOT;
    }
    return super._deserializeFromEmptyString();
}

Serializing and Deserializing Locale.ROOT

Serializing and Deserializing Locale objects seems to work just fine, until you try on the Root Locale.
It writes it out as an empty string and when it reads it in, the value is null
@Test
    public void testLocaleDeserialization() throws IOException {
        ObjectMapper objectMapper = new ObjectMapper();
        Locale root = Locale.ROOT;
        String json = objectMapper.writeValueAsString(root);
        System.out.printf("Root Locale: '%s'", json);
        Locale actual = objectMapper.readValue(json, Locale.class);
        Assert.assertEquals(root, actual);
    }

Here is the output:
Root Locale: '""'
java.lang.AssertionError:
Expected :
Actual   :null


@Override
    public Object deserializeSetAndReturn(JsonParser p,
    		DeserializationContext ctxt, Object instance) throws IOException
    {
        Object id = _valueDeserializer.deserialize(p, ctxt);
        /* 02-Apr-2015, tatu: Actually, as per [databind#742], let it be;
         *  missing or null id is needed for some cases, such as cases where id
         *  will be generated externally, at a later point, and is not available
         *  quite yet. Typical use case is with DB inserts.
         */
        // note: no null checks (unlike usually); deserializer should fail if one found
        if (id == null) {
            return null;
        }
        ReadableObjectId roid = ctxt.findObjectId(id, _objectIdReader.generator, _objectIdReader.resolver);
        roid.bindItem(instance);
        // also: may need to set a property value as well
        SettableBeanProperty idProp = _objectIdReader.idProperty;
        if (idProp != null) {
            return idProp.setAndReturn(instance, id);
        }
        return instance;
    }

@Override
    public Object deserializeSetAndReturn(JsonParser p,
    		DeserializationContext ctxt, Object instance) throws IOException
    {
        /* 02-Apr-2015, tatu: Actually, as per [databind#742], let it be;
         *  missing or null id is needed for some cases, such as cases where id
         *  will be generated externally, at a later point, and is not available
         *  quite yet. Typical use case is with DB inserts.
         */
        // note: no null checks (unlike usually); deserializer should fail if one found
        if (p.hasToken(JsonToken.VALUE_NULL)) {
            return null;
        }
        Object id = _valueDeserializer.deserialize(p, ctxt);
        ReadableObjectId roid = ctxt.findObjectId(id, _objectIdReader.generator, _objectIdReader.resolver);
        roid.bindItem(instance);
        // also: may need to set a property value as well
        SettableBeanProperty idProp = _objectIdReader.idProperty;
        if (idProp != null) {
            return idProp.setAndReturn(instance, id);
        }
        return instance;
    }

Problem with Object id handling, explicit `null` token

According to #742, it shouldn't throw an exception if the value of the property is null


protected JavaType _narrow(Class<?> subclass)
{
    if (_class == subclass) {
        return this;
    }
    // Should we check that there is a sub-class relationship?
    // 15-Jan-2016, tatu: Almost yes, but there are some complications with
    //    placeholder values (`Void`, `NoClass`), so can not quite do yet.
    // TODO: fix in 2.8
        /*
        throw new IllegalArgumentException("Class "+subclass.getName()+" not sub-type of "
                +_class.getName());
                */
        return new SimpleType(subclass, _bindings, this, _superInterfaces,
                _valueHandler, _typeHandler, _asStatic);
    // Otherwise, stitch together the hierarchy. First, super-class
    // if not found, try a super-interface
    // should not get here but...
}

protected JavaType _narrow(Class<?> subclass)
{
    if (_class == subclass) {
        return this;
    }
    // Should we check that there is a sub-class relationship?
    // 15-Jan-2016, tatu: Almost yes, but there are some complications with
    //    placeholder values (`Void`, `NoClass`), so can not quite do yet.
    // TODO: fix in 2.8
    if (!_class.isAssignableFrom(subclass)) {
        /*
        throw new IllegalArgumentException("Class "+subclass.getName()+" not sub-type of "
                +_class.getName());
                */
        return new SimpleType(subclass, _bindings, this, _superInterfaces,
                _valueHandler, _typeHandler, _asStatic);
    }
    // Otherwise, stitch together the hierarchy. First, super-class
    Class<?> next = subclass.getSuperclass();
    if (next == _class) { // straight up parent class? Great.
        return new SimpleType(subclass, _bindings, this,
                _superInterfaces, _valueHandler, _typeHandler, _asStatic);
    }
    if ((next != null) && _class.isAssignableFrom(next)) {
        JavaType superb = _narrow(next);
        return new SimpleType(subclass, _bindings, superb,
                null, _valueHandler, _typeHandler, _asStatic);
    }
    // if not found, try a super-interface
    Class<?>[] nextI = subclass.getInterfaces();
    for (Class<?> iface : nextI) {
        if (iface == _class) { // directly implemented
            return new SimpleType(subclass, _bindings, null,
                    new JavaType[] { this }, _valueHandler, _typeHandler, _asStatic);
        }
        if (_class.isAssignableFrom(iface)) { // indirect, so recurse
            JavaType superb = _narrow(iface);
            return new SimpleType(subclass, _bindings, null,
                    new JavaType[] { superb }, _valueHandler, _typeHandler, _asStatic);
        }
    }
    // should not get here but...
    throw new IllegalArgumentException("Internal error: Can not resolve sub-type for Class "+subclass.getName()+" to "
            +_class.getName());
}

Problem with polymorphic types, losing properties from base type(s)

(background, see: dropwizard/dropwizard#1449)
Looks like sub-type resolution may be broken for one particular case: that of using defaultImpl. If so, appears like properties from super-types are not properly resolved; guessing this could be follow-up item for #1083 (even sooner than I thought...).


public JsonSerializer<?> createContextual(SerializerProvider serializers,
        BeanProperty property) throws JsonMappingException
{
    if (property != null) {
        JsonFormat.Value format = serializers.getAnnotationIntrospector().findFormat((Annotated)property.getMember());
        if (format != null) {

        	// Simple case first: serialize as numeric timestamp?
            JsonFormat.Shape shape = format.getShape();
            if (shape.isNumeric()) {
                return withFormat(Boolean.TRUE, null);
            }

            if (format.getShape() == JsonFormat.Shape.STRING) {
                TimeZone tz = format.getTimeZone();
                final String pattern = format.hasPattern()
                                ? format.getPattern()
                                : StdDateFormat.DATE_FORMAT_STR_ISO8601;
                final Locale loc = format.hasLocale()
                                ? format.getLocale()
                                : serializers.getLocale();
                SimpleDateFormat df = new SimpleDateFormat(pattern, loc);
                if (tz == null) {
                    tz = serializers.getTimeZone();
                }
                df.setTimeZone(tz);
                return withFormat(Boolean.FALSE, df);
            }
        }
    }
    return this;
}

public JsonSerializer<?> createContextual(SerializerProvider serializers,
        BeanProperty property) throws JsonMappingException
{
    if (property != null) {
        JsonFormat.Value format = serializers.getAnnotationIntrospector().findFormat((Annotated)property.getMember());
        if (format != null) {

        	// Simple case first: serialize as numeric timestamp?
            JsonFormat.Shape shape = format.getShape();
            if (shape.isNumeric()) {
                return withFormat(Boolean.TRUE, null);
            }

            if ((shape == JsonFormat.Shape.STRING) || format.hasPattern()
                            || format.hasLocale() || format.hasTimeZone()) {
                TimeZone tz = format.getTimeZone();
                final String pattern = format.hasPattern()
                                ? format.getPattern()
                                : StdDateFormat.DATE_FORMAT_STR_ISO8601;
                final Locale loc = format.hasLocale()
                                ? format.getLocale()
                                : serializers.getLocale();
                SimpleDateFormat df = new SimpleDateFormat(pattern, loc);
                if (tz == null) {
                    tz = serializers.getTimeZone();
                }
                df.setTimeZone(tz);
                return withFormat(Boolean.FALSE, df);
            }
        }
    }
    return this;
}

Looks pretty good, but would it be possible to have a unit test that would fail before fix, pass after? Would be great to have something to guard against regression.
I may want to change the logic a little bit, however; if shape is explicitly defined as NUMBER, textual representation should not be enabled even if Locale (etc) happen to be specified: explicit shape value should have precedence. I can make that change, or you can do it, either way is fine.
I'll also need to merge this again 2.7 branch instead of master, to get in 2.7.3.


public StringBuilder getGenericSignature(StringBuilder sb)
{
    _classSignature(_class, sb, false);
    sb.append('<');
    sb = _referencedType.getGenericSignature(sb);
    sb.append(';');
    return sb;
}

public StringBuilder getGenericSignature(StringBuilder sb)
{
    _classSignature(_class, sb, false);
    sb.append('<');
    sb = _referencedType.getGenericSignature(sb);
    sb.append(">;");
    return sb;
}

Incorrect signature for generic type via `JavaType.getGenericSignature

(see FasterXML/jackson-modules-base#8 for background)
It looks like generic signature generation is missing one closing > character to produce:
()Ljava/util/concurrent/atomic/AtomicReference<Ljava/lang/String;;

instead of expected
()Ljava/util/concurrent/atomic/AtomicReference<Ljava/lang/String;>;

that is, closing '>' is missing.


public Object generateId(Object forPojo) {
    // 04-Jun-2016, tatu: As per [databind#1255], need to consider possibility of
    //    id being generated for "alwaysAsId", but not being written as POJO; regardless,
    //    need to use existing id if there is one:
        id = generator.generateId(forPojo);
    return id;
}

public Object generateId(Object forPojo) {
    // 04-Jun-2016, tatu: As per [databind#1255], need to consider possibility of
    //    id being generated for "alwaysAsId", but not being written as POJO; regardless,
    //    need to use existing id if there is one:
    if (id == null) {
        id = generator.generateId(forPojo);
    }
    return id;
}

JsonIdentityInfo incorrectly serializing forward references

I wrote this small test program to demonstrate the issue:
import com.fasterxml.jackson.annotation.JsonIdentityInfo;
import com.fasterxml.jackson.annotation.JsonIdentityReference;
import com.fasterxml.jackson.annotation.ObjectIdGenerators;
import com.fasterxml.jackson.databind.ObjectMapper;

public class ObjectIdTest {

    public static class Foo {

        @JsonIdentityReference(alwaysAsId = true)
        public Bar bar1;

        @JsonIdentityReference()
        public Bar bar2;
    }

    @JsonIdentityInfo(generator = ObjectIdGenerators.IntSequenceGenerator.class)
    public static class Bar {

    }

    public static void main(String[] args) throws Exception {
        ObjectMapper mapper = new ObjectMapper();

        // create structure to serialize
        Foo mo = new Foo();
        mo.bar1 = new Bar();
        mo.bar2 = mo.bar1;

        // serialize it
        System.out.println(mapper.writeValueAsString(mo));
    }

}
When executing this test program in the latest version (2.7.4), the output will be {"bar1":1,"bar2":{"@id":2}} - the second field will be written with a new id even though both fields reference the same object. Because of this, writing forward references is essentially impossible.
The issue seems to be the fact that BeanSerializerBase will always call WritableObjectId.generateId if the referenced object has not been written in plain format yet (https://github.com/FasterXML/jackson-databind/blob/master/src/main/java/com/fasterxml/jackson/databind/ser/std/BeanSerializerBase.java#L600). This will also happen if an id has been generated before.
It might also be smarter to only generate a new id in WritableObjectId.generateId if that hasn't happened before; as that method doesn't have a javadoc I can't tell how it is supposed to work.


protected void _addMethodMixIns(Class<?> targetClass, AnnotatedMethodMap methods,
        Class<?> mixInCls, AnnotatedMethodMap mixIns)
{
    List<Class<?>> parents = new ArrayList<Class<?>>();
    parents.add(mixInCls);
    ClassUtil.findSuperTypes(mixInCls, targetClass, parents);
    for (Class<?> mixin : parents) {
        for (Method m : mixin.getDeclaredMethods()) {
            if (!_isIncludableMemberMethod(m)) {
                continue;
            }
            AnnotatedMethod am = methods.find(m);
            /* Do we already have a method to augment (from sub-class
             * that will mask this mixIn)? If so, add if visible
             * without masking (no such annotation)
             */
            if (am != null) {
                _addMixUnders(m, am);
                /* Otherwise will have precedence, but must wait
                 * until we find the real method (mixIn methods are
                 * just placeholder, can't be called)
                 */
            } else {
                // Well, or, as per [Issue#515], multi-level merge within mixins...
                    mixIns.add(_constructMethod(m));
            }
        }
    }
}

protected void _addMethodMixIns(Class<?> targetClass, AnnotatedMethodMap methods,
        Class<?> mixInCls, AnnotatedMethodMap mixIns)
{
    List<Class<?>> parents = new ArrayList<Class<?>>();
    parents.add(mixInCls);
    ClassUtil.findSuperTypes(mixInCls, targetClass, parents);
    for (Class<?> mixin : parents) {
        for (Method m : mixin.getDeclaredMethods()) {
            if (!_isIncludableMemberMethod(m)) {
                continue;
            }
            AnnotatedMethod am = methods.find(m);
            /* Do we already have a method to augment (from sub-class
             * that will mask this mixIn)? If so, add if visible
             * without masking (no such annotation)
             */
            if (am != null) {
                _addMixUnders(m, am);
                /* Otherwise will have precedence, but must wait
                 * until we find the real method (mixIn methods are
                 * just placeholder, can't be called)
                 */
            } else {
                // Well, or, as per [Issue#515], multi-level merge within mixins...
                am = mixIns.find(m);
                if (am != null) {
                    _addMixUnders(m, am);
                } else {
                    mixIns.add(_constructMethod(m));
                }
            }
        }
    }
}

Mixin annotations lost when using a mixin class hierarchy with non-mixin interfaces

In summary, mixin annotations are lost when Jackson scans a parent mixin class with Json annotations followed by an interface implemented by the parent mixin class that does not have the same Json annotations.
Jackson version: 2.4.0
Detail:
I have the following class structure
public interface Contact {
    String getCity();
}

public class ContactImpl implements Contact {
    public String getCity() { ... }
}

public class ContactMixin implements Contact {
    @JsonProperty
    public String getCity() { return null; }
}

public interface Person extends Contact {}

public class PersonImpl extends ContactImpl implements Person {}

public class PersonMixin extends ContactMixin implements Person {}
and I configure a module as
// There are other getters/properties in the Impl class that do not need to be serialized and so
// I am using the Mixin to match the interface and explicitly annotate all the inherited methods
module.disable(MapperFeature.ALLOW_FINAL_FIELDS_AS_MUTATORS)
    .disable(MapperFeature.AUTO_DETECT_FIELDS)
    .disable(MapperFeature.AUTO_DETECT_GETTERS)
    .disable(MapperFeature.AUTO_DETECT_IS_GETTERS)
    .disable(MapperFeature.INFER_PROPERTY_MUTATORS);
module.setMixInAnnotation(Person.class, PersonMixin.class);
When a PersonImpl instance is serialized, city is not included.
I debugged the code and this is what happens:
In AnnotatedClass.resolveMemberMethods() the supertypes of PersonImpl are [Person.class, Contact.class, ContactImpl.class] in that order.
It starts with Person for which it finds PersonMixin and proceeds to AnnotatedClass._addMethodMixIns(). Here the parents list has [PersonMixin, ContactMixin, Contact]. When it processes ContactMixin it adds getCity() with the JsonProperty annotation. Then it processes Contact, doesn't find getCity() in methods map and so creates a new AnnotatedMethod for getCity() with the one from the interface which has no annotation which replaces the one from ContactMixin
The workaround for this issue is to explicitly add any parent mixins to the module i.e.
module.setMixInAnnotation(Contact.class, ContactMixin.class);


protected final JsonDeserializer<Object> _findDeserializer(DeserializationContext ctxt,
        String typeId) throws IOException
{
    JsonDeserializer<Object> deser = _deserializers.get(typeId);
    if (deser == null) {
        /* As per [Databind#305], need to provide contextual info. But for
         * backwards compatibility, let's start by only supporting this
         * for base class, not via interface. Later on we can add this
         * to the interface, assuming deprecation at base class helps.
         */
        JavaType type = _idResolver.typeFromId(ctxt, typeId);
        if (type == null) {
            // As per [JACKSON-614], use the default impl if no type id available:
            deser = _findDefaultImplDeserializer(ctxt);
            if (deser == null) {
                // 10-May-2016, tatu: We may get some help...
                JavaType actual = _handleUnknownTypeId(ctxt, typeId, _idResolver, _baseType);
                if (actual == null) { // what should this be taken to mean?
                    // TODO: try to figure out something better
                    return null;
                }
                // ... would this actually work?
                deser = ctxt.findContextualValueDeserializer(actual, _property);
            }
        } else {
            /* 16-Dec-2010, tatu: Since nominal type we get here has no (generic) type parameters,
             *   we actually now need to explicitly narrow from base type (which may have parameterization)
             *   using raw type.
             *
             *   One complication, though; can not change 'type class' (simple type to container); otherwise
             *   we may try to narrow a SimpleType (Object.class) into MapType (Map.class), losing actual
             *   type in process (getting SimpleType of Map.class which will not work as expected)
             */
            if ((_baseType != null)
                    && _baseType.getClass() == type.getClass()) {
                /* 09-Aug-2015, tatu: Not sure if the second part of the check makes sense;
                 *   but it appears to check that JavaType impl class is the same which is
                 *   important for some reason?
                 *   Disabling the check will break 2 Enum-related tests.
                 */
                // 19-Jun-2016, tatu: As per [databind#1270] we may actually get full
                //   generic type with custom type resolvers. If so, should try to retain them.
                //  Whether this is sufficient to avoid problems remains to be seen, but for
                //  now it should improve things.
                    type = ctxt.getTypeFactory().constructSpecializedType(_baseType, type.getRawClass());
            }
            deser = ctxt.findContextualValueDeserializer(type, _property);
        }
        _deserializers.put(typeId, deser);
    }
    return deser;
}

protected final JsonDeserializer<Object> _findDeserializer(DeserializationContext ctxt,
        String typeId) throws IOException
{
    JsonDeserializer<Object> deser = _deserializers.get(typeId);
    if (deser == null) {
        /* As per [Databind#305], need to provide contextual info. But for
         * backwards compatibility, let's start by only supporting this
         * for base class, not via interface. Later on we can add this
         * to the interface, assuming deprecation at base class helps.
         */
        JavaType type = _idResolver.typeFromId(ctxt, typeId);
        if (type == null) {
            // As per [JACKSON-614], use the default impl if no type id available:
            deser = _findDefaultImplDeserializer(ctxt);
            if (deser == null) {
                // 10-May-2016, tatu: We may get some help...
                JavaType actual = _handleUnknownTypeId(ctxt, typeId, _idResolver, _baseType);
                if (actual == null) { // what should this be taken to mean?
                    // TODO: try to figure out something better
                    return null;
                }
                // ... would this actually work?
                deser = ctxt.findContextualValueDeserializer(actual, _property);
            }
        } else {
            /* 16-Dec-2010, tatu: Since nominal type we get here has no (generic) type parameters,
             *   we actually now need to explicitly narrow from base type (which may have parameterization)
             *   using raw type.
             *
             *   One complication, though; can not change 'type class' (simple type to container); otherwise
             *   we may try to narrow a SimpleType (Object.class) into MapType (Map.class), losing actual
             *   type in process (getting SimpleType of Map.class which will not work as expected)
             */
            if ((_baseType != null)
                    && _baseType.getClass() == type.getClass()) {
                /* 09-Aug-2015, tatu: Not sure if the second part of the check makes sense;
                 *   but it appears to check that JavaType impl class is the same which is
                 *   important for some reason?
                 *   Disabling the check will break 2 Enum-related tests.
                 */
                // 19-Jun-2016, tatu: As per [databind#1270] we may actually get full
                //   generic type with custom type resolvers. If so, should try to retain them.
                //  Whether this is sufficient to avoid problems remains to be seen, but for
                //  now it should improve things.
                if (!type.hasGenericTypes()) {
                    type = ctxt.getTypeFactory().constructSpecializedType(_baseType, type.getRawClass());
                }
            }
            deser = ctxt.findContextualValueDeserializer(type, _property);
        }
        _deserializers.put(typeId, deser);
    }
    return deser;
}

Generic type returned from type id resolver seems to be ignored

https://github.com/benson-basis/jackson-custom-mess-tc
Here's the situation, with Jackson 2.7.4.
I have a TypeIdResolver that returns a JavaType for a generic type. However, something seems to be forgetting/erasing the generic, as it is failing to use the generic type param to understand the type of a field in the class.
All the information is in the test case, so I'm not putting any code to read here in the issue.


@SuppressWarnings("unchecked")
    public static JsonSerializer<Object> getFallbackKeySerializer(SerializationConfig config,
            Class<?> rawKeyType)
    {
        if (rawKeyType != null) {
            // 29-Sep-2015, tatu: Odd case here, of `Enum`, which we may get for `EnumMap`; not sure
            //   if that is a bug or feature. Regardless, it seems to require dynamic handling
            //   (compared to getting actual fully typed Enum).
            //  Note that this might even work from the earlier point, but let's play it safe for now
            // 11-Aug-2016, tatu: Turns out we get this if `EnumMap` is the root value because
            //    then there is no static type
            if (rawKeyType == Enum.class) {
                return new Dynamic();
            }
            if (rawKeyType.isEnum()) {
                return new Default(Default.TYPE_ENUM, rawKeyType);
            }
        }
        return DEFAULT_KEY_SERIALIZER;
    }

@SuppressWarnings("unchecked")
    public static JsonSerializer<Object> getFallbackKeySerializer(SerializationConfig config,
            Class<?> rawKeyType)
    {
        if (rawKeyType != null) {
            // 29-Sep-2015, tatu: Odd case here, of `Enum`, which we may get for `EnumMap`; not sure
            //   if that is a bug or feature. Regardless, it seems to require dynamic handling
            //   (compared to getting actual fully typed Enum).
            //  Note that this might even work from the earlier point, but let's play it safe for now
            // 11-Aug-2016, tatu: Turns out we get this if `EnumMap` is the root value because
            //    then there is no static type
            if (rawKeyType == Enum.class) {
                return new Dynamic();
            }
            if (rawKeyType.isEnum()) {
                return EnumKeySerializer.construct(rawKeyType,
                        EnumValues.constructFromName(config, (Class<Enum<?>>) rawKeyType));
            }
        }
        return DEFAULT_KEY_SERIALIZER;
    }

EnumMap keys not using enum's `@JsonProperty` values unlike Enum values

Based on these issues:
https://github.com/FasterXML/jackson-databind/issues/677
https://github.com/FasterXML/jackson-databind/issues/1148
https://github.com/FasterXML/jackson-annotations/issues/96

I implemented @JsonProperty for my enum constants and they show up nicely when they are property values. But I also have an EnumMap which uses the enum, and it's generated JSON uses the original enum names for the keys and not the JsonProperty values.

Using 2.8.1 (in spring boot 4.3.2)

Thanks!


public <T> MappingIterator<T> readValues(byte[] src, int offset, int length)
    throws IOException, JsonProcessingException
{
    if (_dataFormatReaders != null) {
        return _detectBindAndReadValues(_dataFormatReaders.findFormat(src, offset, length), false);
    }
    return _bindAndReadValues(_considerFilter(_parserFactory.createParser(src),
            true));
}

public <T> MappingIterator<T> readValues(byte[] src, int offset, int length)
    throws IOException, JsonProcessingException
{
    if (_dataFormatReaders != null) {
        return _detectBindAndReadValues(_dataFormatReaders.findFormat(src, offset, length), false);
    }
    return _bindAndReadValues(_considerFilter(_parserFactory.createParser(src, offset, length),
            true));
}

ObjectReader.readValues() ignores offset and length when reading an array

ObjectReader.readValues ignores offset and length when reading an array. If _dataFormatReaders it will always use the full array:
https://github.com/FasterXML/jackson-databind/blob/2.7/src/main/java/com/fasterxml/jackson/databind/ObjectReader.java#L1435


public String getDescription() {
            if (_desc == null) {
                StringBuilder sb = new StringBuilder();

                if (_from == null) { // can this ever occur?
                    sb.append("UNKNOWN");
                } else {
                    Class<?> cls = (_from instanceof Class<?>) ? (Class<?>)_from : _from.getClass();
                    // Hmmh. Although Class.getName() is mostly ok, it does look
                    // butt-ugly for arrays.
                    // 06-Oct-2016, tatu: as per [databind#1403], `getSimpleName()` not so good
                    //   as it drops enclosing class. So let's try bit different approach
                    String pkgName = ClassUtil.getPackageName(cls);
                    if (pkgName != null) {
                        sb.append(pkgName);
                        sb.append('.');
                    }
                    sb.append(cls.getSimpleName());
                }
                sb.append('[');
                if (_fieldName != null) {
                    sb.append('"');
                    sb.append(_fieldName);
                    sb.append('"');
                } else if (_index >= 0) {
                    sb.append(_index);
                } else {
                    sb.append('?');
                }
                sb.append(']');
                _desc = sb.toString();
            }
            return _desc;
        }

public String getDescription() {
            if (_desc == null) {
                StringBuilder sb = new StringBuilder();

                if (_from == null) { // can this ever occur?
                    sb.append("UNKNOWN");
                } else {
                    Class<?> cls = (_from instanceof Class<?>) ? (Class<?>)_from : _from.getClass();
                    // Hmmh. Although Class.getName() is mostly ok, it does look
                    // butt-ugly for arrays.
                    // 06-Oct-2016, tatu: as per [databind#1403], `getSimpleName()` not so good
                    //   as it drops enclosing class. So let's try bit different approach
                    int arrays = 0;
                    while (cls.isArray()) {
                        cls = cls.getComponentType();
                        ++arrays;
                    }
                    sb.append(cls.getName());
                    while (--arrays >= 0) {
                        sb.append("[]");
                    }
                    /* was:
                    String pkgName = ClassUtil.getPackageName(cls);
                    if (pkgName != null) {
                        sb.append(pkgName);
                        sb.append('.');
                    }
                    */
                }
                sb.append('[');
                if (_fieldName != null) {
                    sb.append('"');
                    sb.append(_fieldName);
                    sb.append('"');
                } else if (_index >= 0) {
                    sb.append(_index);
                } else {
                    sb.append('?');
                }
                sb.append(']');
                _desc = sb.toString();
            }
            return _desc;
        }

Reference-chain hints use incorrect class-name for inner classes

``` java
import com.fasterxml.jackson.annotation.JsonCreator;
import com.fasterxml.jackson.annotation.JsonProperty;
import com.fasterxml.jackson.databind.JsonMappingException;
import com.fasterxml.jackson.databind.ObjectMapper;
import org.junit.jupiter.api.Test;

import java.io.IOException;

import static com.google.common.truth.Truth.assertThat;
import static org.junit.jupiter.api.Assertions.expectThrows;

public class ReferenceChainTest {
    // illustrates that jackson's "reference chain" help-text uses incorrect class-names for inner classes
    @Test public void incorrectReferenceChain() throws IOException {
        JsonMappingException jsonMappingException = expectThrows(JsonMappingException.class, () -> {
            ObjectMapper objectMapper = new ObjectMapper();
            objectMapper.readValue(objectMapper.writeValueAsBytes(new Outer()), Outer.class);
        });
        JsonMappingException.Reference reference = jsonMappingException.getPath().get(0);
        assertThat(reference.toString()).isEqualTo("ReferenceChainTest$Outer[\"inner\"]");
    }

    static class Outer {
        public Inner inner = new Inner();
    }

    static class Inner {
        public int x;

        @JsonCreator public static Inner create(@JsonProperty("x") int x) {
            throw new RuntimeException("test-exception");
        }
    }
}

```


public TokenBuffer deserialize(JsonParser jp, DeserializationContext ctxt) throws IOException
{
        copyCurrentStructure(jp);
    /* 28-Oct-2014, tatu: As per #592, need to support a special case of starting from
     *    FIELD_NAME, which is taken to mean that we are missing START_OBJECT, but need
     *    to assume one did exist.
     */
    return this;
}

public TokenBuffer deserialize(JsonParser jp, DeserializationContext ctxt) throws IOException
{
    if (jp.getCurrentTokenId() != JsonToken.FIELD_NAME.id()) {
        copyCurrentStructure(jp);
        return this;
    }
    /* 28-Oct-2014, tatu: As per #592, need to support a special case of starting from
     *    FIELD_NAME, which is taken to mean that we are missing START_OBJECT, but need
     *    to assume one did exist.
     */
    JsonToken t;
    writeStartObject();
    do {
        copyCurrentStructure(jp);
    } while ((t = jp.nextToken()) == JsonToken.FIELD_NAME);
    if (t != JsonToken.END_OBJECT) {
        throw ctxt.mappingException("Expected END_OBJECT after copying contents of a JsonParser into TokenBuffer, got "+t);
    }
    writeEndObject();
    return this;
}

Possibly wrong TokenBuffer delegate deserialization using @JsonCreator

class Value {
@JsonCreator
public static Value from(TokenBuffer buffer) {
...
}
Given JSON string is  { "a":1, "b":null }, it is expected that while deserializing using delegate buffer,
current token will be start object {, and rest of the tokens will be available in buffer:
[START_OBJECT, FIELD_NAME, VALUE_NUMBER_INT, FIELD_NAME, VALUE_NULL, END_OBJECT]

But, buffers ends up being started with field name and then contains single attribute value
[FIELD_NAME, VALUE_NUMBER_INT]

It's due to how TokenBuffer#copyCurrentStructure works when we have current token as a FIELD_NAME, rather than START_OBJECT, because it's forced to move to next token BeanDeserializer.java:120
Hope this helps to nail it down. Is it an intended behavior, or it's regression/bug?


public void remove(SettableBeanProperty propToRm)
{
    ArrayList<SettableBeanProperty> props = new ArrayList<SettableBeanProperty>(_size);
    String key = getPropertyName(propToRm);
    boolean found = false;

    for (int i = 1, end = _hashArea.length; i < end; i += 2) {
        SettableBeanProperty prop = (SettableBeanProperty) _hashArea[i];
        if (prop == null) {
            continue;
        }
        if (!found) {
            // 09-Jan-2017, tatu: Important: must check name slot and NOT property name,
            //   as only former is lower-case in case-insensitive case
            found = key.equals(prop.getName());
            if (found) {
                // need to leave a hole here
                _propsInOrder[_findFromOrdered(prop)] = null;
                continue;
            }
        }
        props.add(prop);
    }
    if (!found) {
        throw new NoSuchElementException("No entry '"+propToRm.getName()+"' found, can't remove");
    }
    init(props);
}

public void remove(SettableBeanProperty propToRm)
{
    ArrayList<SettableBeanProperty> props = new ArrayList<SettableBeanProperty>(_size);
    String key = getPropertyName(propToRm);
    boolean found = false;

    for (int i = 1, end = _hashArea.length; i < end; i += 2) {
        SettableBeanProperty prop = (SettableBeanProperty) _hashArea[i];
        if (prop == null) {
            continue;
        }
        if (!found) {
            // 09-Jan-2017, tatu: Important: must check name slot and NOT property name,
            //   as only former is lower-case in case-insensitive case
            found = key.equals(_hashArea[i-1]);
            if (found) {
                // need to leave a hole here
                _propsInOrder[_findFromOrdered(prop)] = null;
                continue;
            }
        }
        props.add(prop);
    }
    if (!found) {
        throw new NoSuchElementException("No entry '"+propToRm.getName()+"' found, can't remove");
    }
    init(props);
}

ACCEPT_CASE_INSENSITIVE_PROPERTIES fails with @JsonUnwrapped

(note: moved from FasterXML/jackson-dataformat-csv#133)
When trying to deserialize type like:
public class Person {
  @JsonUnwrapped(prefix = "businessAddress.")
  public Address businessAddress;
}

public class Address {
  public String street;
  public String addon;
  public String zip = "";
  public String town;
  public String country;
}
with case-insensitive mapper (mapper.enable(MapperFeature.ACCEPT_CASE_INSENSITIVE_PROPERTIES);) I get exception:
java.util.NoSuchElementException: No entry 'businessAddress' found, can't remove
	at com.fasterxml.jackson.databind.deser.impl.BeanPropertyMap.remove(BeanPropertyMap.java:447)
	at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.resolve(BeanDeserializerBase.java:534)
	at com.fasterxml.jackson.databind.deser.DeserializerCache._createAndCache2(DeserializerCache.java:293)
   ...


public static StdKeyDeserializer forType(Class<?> raw)
{
    int kind;

    // first common types:
    if (raw == String.class || raw == Object.class) {
        return StringKD.forType(raw);
    } else if (raw == UUID.class) {
        kind = TYPE_UUID;
    } else if (raw == Integer.class) {
        kind = TYPE_INT;
    } else if (raw == Long.class) {
        kind = TYPE_LONG;
    } else if (raw == Date.class) {
        kind = TYPE_DATE;
    } else if (raw == Calendar.class) {
        kind = TYPE_CALENDAR;
    // then less common ones...
    } else if (raw == Boolean.class) {
        kind = TYPE_BOOLEAN;
    } else if (raw == Byte.class) {
        kind = TYPE_BYTE;
    } else if (raw == Character.class) {
        kind = TYPE_CHAR;
    } else if (raw == Short.class) {
        kind = TYPE_SHORT;
    } else if (raw == Float.class) {
        kind = TYPE_FLOAT;
    } else if (raw == Double.class) {
        kind = TYPE_DOUBLE;
    } else if (raw == URI.class) {
        kind = TYPE_URI;
    } else if (raw == URL.class) {
        kind = TYPE_URL;
    } else if (raw == Class.class) {
        kind = TYPE_CLASS;
    } else if (raw == Locale.class) {
        FromStringDeserializer<?> deser = FromStringDeserializer.findDeserializer(Locale.class);
        return new StdKeyDeserializer(TYPE_LOCALE, raw, deser);
    } else if (raw == Currency.class) {
        FromStringDeserializer<?> deser = FromStringDeserializer.findDeserializer(Currency.class);
        return new StdKeyDeserializer(TYPE_CURRENCY, raw, deser);
    } else {
        return null;
    }
    return new StdKeyDeserializer(kind, raw);
}

public static StdKeyDeserializer forType(Class<?> raw)
{
    int kind;

    // first common types:
    if (raw == String.class || raw == Object.class || raw == CharSequence.class) {
        return StringKD.forType(raw);
    } else if (raw == UUID.class) {
        kind = TYPE_UUID;
    } else if (raw == Integer.class) {
        kind = TYPE_INT;
    } else if (raw == Long.class) {
        kind = TYPE_LONG;
    } else if (raw == Date.class) {
        kind = TYPE_DATE;
    } else if (raw == Calendar.class) {
        kind = TYPE_CALENDAR;
    // then less common ones...
    } else if (raw == Boolean.class) {
        kind = TYPE_BOOLEAN;
    } else if (raw == Byte.class) {
        kind = TYPE_BYTE;
    } else if (raw == Character.class) {
        kind = TYPE_CHAR;
    } else if (raw == Short.class) {
        kind = TYPE_SHORT;
    } else if (raw == Float.class) {
        kind = TYPE_FLOAT;
    } else if (raw == Double.class) {
        kind = TYPE_DOUBLE;
    } else if (raw == URI.class) {
        kind = TYPE_URI;
    } else if (raw == URL.class) {
        kind = TYPE_URL;
    } else if (raw == Class.class) {
        kind = TYPE_CLASS;
    } else if (raw == Locale.class) {
        FromStringDeserializer<?> deser = FromStringDeserializer.findDeserializer(Locale.class);
        return new StdKeyDeserializer(TYPE_LOCALE, raw, deser);
    } else if (raw == Currency.class) {
        FromStringDeserializer<?> deser = FromStringDeserializer.findDeserializer(Currency.class);
        return new StdKeyDeserializer(TYPE_CURRENCY, raw, deser);
    } else {
        return null;
    }
    return new StdKeyDeserializer(kind, raw);
}

Missing KeyDeserializer for CharSequence

Looks like use of nominal Map key type of CharSequence does not work yet (as of 2.7.8 / 2.8.6).
This is something that is needed to work with certain frameworks, such as Avro's generated POJOs.