JDK 8 compatibility of Kotlin

I stumbled upon an issue with JDK 8 parallel streams in Kotlin in this forum recently. Since I read this, I wonder why this is an issue (or special case) in Kotlin. Isn't parallelStream() a usual JDK method that shouldn't be special in Kotlin?

What other issues are known with JDK 8?

Something like the parallelStream issues could be a show stopper for Kotlin! Developers preferring safety would stay with Java, and developers perferring something more modern would go with Scala …

There are couple of things we need to support for JDK8, and the work has already been started. Most important in this case is support for default methods in interfaces, which is a new feature in JRE8. For this we have to support reading metadata from the new bytecode version 52. Another thing is we (sadly) have to rename our own Stream<T> interface, which conflicts with similar interface from JDK8.

It is good to know that you are working on it.

But I still don’t understand, why I am not able to use a usal method of the JDK (parallelStream).

Function parallelStream() is defined on a Java interface Collection<T> as follows:

 
default Stream<E> parallelStream() {
    return StreamSupport.stream(spliterator(), true);
}

So it is exactly the unsupported case I mentioned above – default method on interface.

Thanks for the explanation.

I fear that static extension methods are a major risk in Kotlin. Imagine you would have released Kotlin 1.0 lets say two years ago, and now comes Oracle and releases a new JDK breaking Kotlin APIs like in the case of stream. I think this is a general problem of static extension methods and maybe the reason why scala uses implicit conversions instead.

Static extensions methods are successfull in .NET, but Microsoft develops the language (C#) and the full standard library. This helps to reduce such conflicts.

I'm not sure implicit conversions in Scala are a good alternative to extension methods. Implicits in Scala are a problem for code readability and they are also one of the things that make the Scala compiler that slow. Furthermore, you can define one implicit conversion from a type to another type, but not another time from the same type to another new type. This is because the compiler stops searching for an implicit conversion once it has found one. You don't have this problem with extension methods. This is very well described in this blog article. The article is of very good quality. You can see that the author really has a deep understanding of Scala.

Extension functions in Kotlin are on the contrary very painless and they exist in other languages as well such as Smalltalk, Objective-C (I believe also in Swift), C#, Groovy (and I think also in Ruby). The Kotlin guys might have to adapt their collection extensions to what the Oracle people have done in JDK8. I don’t think this is a big issue. If it were I would still not trade in code readability for anything else. The Groovy people had to merge their iterators and many other extention functions all over the Java class library with the ones in JDK8 as well and that has been done months ago without causing a stir. So I don’t see any big risk here and certainly not a major one. Scala implicits are on the contrary a major problem of the language. They could fix it by adding extension methods to Scala, but that they can’t do without loosing face after all the controversial discussions about implicits …

1 Like

Extensions functions are of low risk, because instance members are preferred, so if someone introduces filter function on the List<T> with a parameter compatible with functional literal, it will be preferred to the extension member. However, the Java 8 conflict is not so much about extension functions but rather about types. Sometime ago we introduced kotlin.Stream<T> type, with the simples possible interface – iterator() function – with the idea that we can make it compatible with Java8 Stream<T> type.

Currently we seem to come to a conclusion it was a mistake as we cannot fine a way to represent Stream<T> as kotlin type in Java 6 & 7, and then use to Java8’s Stream<T> when targeting JDK8. If you have any idea about how to make it work – please share!

Extensions functions are of low risk, because instance members are preferred, so if someone introduces filter function on the List<T> with a parameter compatible with functional literal, it will be preferred to the extension member

I just realised that this means a backwards compatible JDK update might actually redefine the meaning of existing Kotlin source code in a silent and breaking way, because a new method with the same name that overrides an extension function might not have the same behaviour at all.

I wonder how this can be avoided. Perhaps Kotlin source code (as opposed to Maven/Gradle files) should specify the version of the imports they depend on. If those imports then change, the compiler or IDE can then issue warnings in the case where an extension function suddenly stops being called.

That's a good point :O

This should be really solved in a predictable way, otherwise big problems will occur later. I wonder about the solution too, it is really a tricky situation. When a class adds a new method conflicting with an extension, it is a problem if members are preferred. When there are both a member method and an extensions and extensions are preferred, then a problem occurs when the extension is removed. Both problems are basically the same: in both cases the behaviour probably changes a bit and it can break the client code.

But there might be a way. Firstly, I think that any solution should assume version information in the source.  (If there  is no other reason, then the problem is Java/JVM world still struggles  with versioning on that level.) A library implementing an extension should rather declare which versions of the extended library the extension implementation may depend on; the rest can be left to the common dependency resolution tools (like Maven, Ivy etc.).

Secondly, while it looks more natural to prefer member methods over extensions, it might be in the other way. The first reason is that the developer might have means to tell which extensions methods should be available, while it can hardly suppress the visibility of a method of an imported class. The second reason is then the fact that removing a method is much rarer and it implies incompatible version change, while adding a method does not. Hence unexpected removal of an extension method should manifest itself quite early and in a hard way (either the dependecy resolution tool or the compiler complains that something is wrong and does not allow to proceed).

If such an approach should work, then the developer must have means to tell which extensions the compiler may use in a scope. So if it is possible to tell import me extension X for class Y from Z (wildcards on own risk), it is safe: the developer knows that an extension is used if possible (and gets an error if the import is invalid). However, to prevent import polution, it would be really useful to have scoped imports, i.e. imports could be not only at the top of the file, but in any block and visible only in the limited scope (similar to local variables). This is known in Python and I really like it :wink:

OK, maybe it is not as elegant and requires that a developer must write something more (or the IDE must offer and handle it for him/her), but the source code is not ambiguous then and the applicable symbol set is more explicit. And maybe it is not a good idea, but perhaps it helps to find some better one :slight_smile:

Just wondering: what would be the downsides of making this a compiler error? If a member function is added to the extended class, it is not guaranteed to have the same semantics as the extension function in Kotlin. Thus, the code that relied on that extension function is now broken and needs to be fixed by a developer.

One could argue that there should be a way to suppress this compiler error for cases where you want to define an extension function anyway. A reason for this could be that the compiled library should work with older versions of that class where there is no such conflict yet.

I think I would prefer a compiler error (or warning) if extension functions are ambiguous with member functions, yes. It would indeed mean that updating the JDK could break Kotlin source code however, at runtime old binaries would not break. As the compiler would be flagging a potential compatibility problem with my code I would actually prefer to have the broken source compatibility thus forcing me to examine the code for potential semantic incompatibilities .... certainly preferable to a silent behavioural change that would be detected only by unit tests, if you're lucky.

So, when there is no conflict with member functions, the compiler needs no nudging; if there is a conflicting member function, there should be a warning to be resolved? Sounds reasonable and I tend to agree.

The question is what should be the default choice of the compiler: the member function or the extension? Since the compiler must make a choice for a warning, there must be a default. I would tell that the extension would be better as the client code can control (or should be able to control) the availability of the extension in the scope. Or maybe: it should not be a warning at all, it should be an error. It is actually very similar to method overloading: when there is an ambiguity, the developer must resolve it explicitly. Both overloads and extensions are resolved statically and I guess that the resolution algorithms in the compiler deal both cases in a similar way.

If the compiler treats the ambiguity as an error (which seems to me as the best choice now), the remaining part would be just providing the way to the developer to resolve the ambiguity. How? I don’t know. Maybe by importing an extension under a different name (when it is possible import symbols under different names)? Or with some cast-like or annotation-like syntax telling which option should be chosen? Maybe. Using the flexible import is probably simpler and there is no need to repeated ad-hoc ambiguity resolution. Just brainstorming :8}

I like your idea of combining an error with the fix being renaming the extension function at import time. This is the sort of thing that IntelliJ could easily implement as a quickfix/intention. Especially as if you do it once, you could then tell the IDE to fix similar conflicts in the same way everywhere.

Any news on this topic? It seems like my old example still leads to a compiler error.

JDK 8 support is out of scope of 1.0, so we'll get back to it soon, but not working on it now

It is out of scope for 1.0, but any idea when or for which version it might be in scope? Thanks! Big fan.

That particular issue with parallelStream and stream being unavailable for collections is resolved in Kotlin 1.1.

With Kotlin 1.0 you can use kotlinx.support library which provides these methods as extensions.

1 Like

And in what version of Kotlin do you plan full support of java 8?