> It is not a goal to support cross-compilation. Code compiled in a training run must run on the same CPU architecture, with the same set of features, in subsequent production runs.
Your link is about a completely separate feature, called the AOT code cache (targeted for JDK28, not released yet). The post is about the AOT cache, which forms the foundation for this feature, but is separate and already available since JDK24. The JEP for the feature the post is about is https://openjdk.org/jeps/483
For the past several years, Java features are released piecemeal, with each piece released as soon as it can be useful to some significant portion of users or users who can offer useful feedback (and when it's not expected to restrict future pieces). The AOT cache is already used in production despite not being as user-friendly as it could be, because the companies using it find it valuable.
The first use case is the very common one (for Java) where developers control deployment (internal deployment). The last basic building block, caching machine code, will be released next year in JDK 28, and then the feature is likely to evolve to better support other use cases, such as external deployment.
Yes. And this is the right tradeoff vs the ossification you saw previously in Java (and we're starting to see now in Python, see the recent ejection of the CPython JIT), especially now when you can say "GPT, source dive the JDK and figure out which cutting edge is causing the bleeding."
I spent a couple weeks trying to bolt this on to our setup to support a more serverless style approach. OpenJ9 (sadly) really didn't work out well, perhaps we run in a way they don't optimize for. Azul and OpenJDK's approaches helped - but it is quite intrusive (you need a profiling run), and the absolute benefits aren't that great. It's mostly doing classpath stuff at present, and not really AOT work on machine code (though that is in the pipeline).
All in all, I wasn't particularly impressed with the outcome. Obviously GraalVM has more headroom, but that's an even more intrusive change.
When compared with a Rust port of our service, the delta of the efforts is absolutely astronomical. Ridiculous performance, near-instant startup, radically less memory consumption. I mean - duh - but nonetheless, when I contrast the effort vs payback, it's very hard for me to want to invest so much in techniques that have so little payback (graal) when I can put that into an approach that leaves me in a much better place long-term.
If I had mountains of legacy, obviously I couldn't do that, but in my situation it seems like a much more attractive option.
> Ridiculous performance, near-instant startup, radically less memory consumption.
It seems like your use case is rather not a good fit for Java.
Java shines best with large applications, where more abstract source code is desired and when you care more about typical performance, not the worst case performance. And if you can afford the cost of all those benefits, which is higher RAM usage.
> What would be needed is some sort of compatibility switch that disables all optional CPU features. Sure, you will get some reduced performance of the compiled code, but in exchange, a faster startup time.
I think other AOT-compiled languages don't opt in for the lowest set of CPU features -- they just detect them at runtime and use whenever present, all in compiled machine code.
It's normally an opt-in thing I think, since it's not an unconditional win: you have to figure out which code paths are worth it, and then where to trade off binary bloat vs the runtime cost of dynamic dispatch. In rust for example:
What are the most common ways to improve warmup times, actually? Looked a bunch into this, primarily native image, aot and crac, all of them super messy to get to work with a normal JVM webapp. I understand the warmup concept is inherent in JIT compilation, but how are big companies like Netflix handling this to quickly scale microservices horizontally etc?
Native images shouldn't be too messy anymore, Oracle put a lot of effort into DX improvements and all major web frameworks are now aware it's an important and growing use case.
Struggling to build native images should be a sign you're reliant on libraries that are dangerously outdated and abandoned.
They can definitely be pretty messy in some ways; even the standard library doesn't work out of the box perfectly. I have certainly struggled to get image manipulation with java.awt (e.g. thumbnail generation for a fairly standard web use case) to work in a native image.
So it is an antipattern nowadays to manipulate the classpath after building? Iirc native image require static classpath, but it is quite common to inject e.g. environment specific data via the classpath. Should one just read from the filesystem instead, or what’s the strat there?
With the AOT cache, you can actually modify the classpath (or module path as is now used for modern applications) after it has started up. The names and classes for the APIs are different, but you can still load jars at runtime even with the AOT cache enabled and functional.
The AOT cache compatibility issues come when your boot module path differs between training and actual run. As long as that is not the case, you can use it without problem and load additional modules at runtime after it has started up.
With GraalVM, that is not possible. It still has the closed-world assumption.
I mean, how fast do you need it to be? If you're running lambdas with cold starts all day long yes, java is a problem. Otherwise, occasional restarts of java services are quick assuming they're probably stateless. A few seconds? Whats the problem that needs to be solved here?
I’ve experienced deployments under heavy load being quite painful with slow restarts, even when doing e.g. canary releases. Blue/green would work but was in an environment where we couldn’t deploy immutably, we had to mutate the existing deployment in place.
That, and I think the cache busting based on CPU/environment probing logic will need to be significantly improved. Any difference in CPU features should not bust the cache; specific differences should bust the cache.
There also might be a need for "downgrading" cross-compilation profiles a la "don't compile/AOT-cache this app for the current CPU, compile/cache it for <the kind of CPU that I have 10k EC2 instances running>". You'd think that would be easier to achieve by just compiling on a production-like machine, and you'd be disappointed by the number of shops running Java apps hand-built on a Windows laptop across millions of cloud Linux cores.
Yes (or something along those lines), it's just that this isn't the first use case addressed by this capability, which is being delivered gradually (as we normally do in the JDK), and not even all the basic building blocks are out yet.
I saw this on Reddit, and as expected, the replies from the Java community were quite hostile. Sometimes it feels like Java developers have never left their own ecosystem and have never seen how easy some things (like deployment) are on other platforms. Especially Java's way of deploying applications has hindered not only Java but also alt JVM languages like Kotlin and Scala.
Deployment of Java applications continues to be a mess to this day. You can't get anywhere without obscure Maven or Gradle plugins and even then it's a mess, especially if you're dealing with modules and jlink. Actually, using jlink without these broken plugins is the closest thing to hell. I'm not even going to elaborate further because I've explained this many times (https://news.ycombinator.com/item?id=47423383).
Golang's or especially Rust's tooling on the other hand are simply beautiful. Even .NET is fine now.
> Deployment of Java applications continues to be a mess
If it could help you, we're using JIB. We're on Scala and so we're using sbt-jib. Zero issues, works like a charm, deployment (on k8s) is just not a problem for us. But I would expect that it should work well for Java too.
Agreed, but that's not really relevant to the GP, who was comparing Java deployment ease vs. other platforms. NPM hell doesn't make deployment hard (quite the contrary); it makes security and stability/maintenance hard.
Server-Side/Desktop JavaScript, I think, has a much better/more polished deployment experience than Java...at the cost of having a much worse build (ES module hell, bundler hell...) and maintenance (dependency hell, security hell, constantly changing APIs hell...) experience than Java, which excels at both of those.
https://openjdk.org/jeps/544
> Non-Goals
> It is not a goal to support cross-compilation. Code compiled in a training run must run on the same CPU architecture, with the same set of features, in subsequent production runs.
IMHO, that would be much better than running into crashes and having to figure out the cause yourself.
"This is what I would like AOT cache to mean."
The first use case is the very common one (for Java) where developers control deployment (internal deployment). The last basic building block, caching machine code, will be released next year in JDK 28, and then the feature is likely to evolve to better support other use cases, such as external deployment.
There are plenty of other AOT/JIT cache approaches in the Java ecosystem already, with more mature deployment experience of several years, namely:
OpenJ9, Azul, PTC, Aicas, the Android cousin since Android 7
Technology based on process snapshots CRaC for Amazon OpenJDK distro (Correcto), and OpenJ9, is also another approach available for some time.
All in all, I wasn't particularly impressed with the outcome. Obviously GraalVM has more headroom, but that's an even more intrusive change.
When compared with a Rust port of our service, the delta of the efforts is absolutely astronomical. Ridiculous performance, near-instant startup, radically less memory consumption. I mean - duh - but nonetheless, when I contrast the effort vs payback, it's very hard for me to want to invest so much in techniques that have so little payback (graal) when I can put that into an approach that leaves me in a much better place long-term.
If I had mountains of legacy, obviously I couldn't do that, but in my situation it seems like a much more attractive option.
It seems like your use case is rather not a good fit for Java.
Java shines best with large applications, where more abstract source code is desired and when you care more about typical performance, not the worst case performance. And if you can afford the cost of all those benefits, which is higher RAM usage.
At least that's the theory :)
I think other AOT-compiled languages don't opt in for the lowest set of CPU features -- they just detect them at runtime and use whenever present, all in compiled machine code.
https://doc.rust-lang.org/beta/core/arch/#dynamic-cpu-featur...
https://youtu.be/U2j4bKgKr-s?t=3577
Struggling to build native images should be a sign you're reliant on libraries that are dangerously outdated and abandoned.
The AOT cache compatibility issues come when your boot module path differs between training and actual run. As long as that is not the case, you can use it without problem and load additional modules at runtime after it has started up.
With GraalVM, that is not possible. It still has the closed-world assumption.
Instead, this AOT feature is not very ergonomic to use at all, with too many pitfalls to be of broad use IMO.
There also might be a need for "downgrading" cross-compilation profiles a la "don't compile/AOT-cache this app for the current CPU, compile/cache it for <the kind of CPU that I have 10k EC2 instances running>". You'd think that would be easier to achieve by just compiling on a production-like machine, and you'd be disappointed by the number of shops running Java apps hand-built on a Windows laptop across millions of cloud Linux cores.
Deployment of Java applications continues to be a mess to this day. You can't get anywhere without obscure Maven or Gradle plugins and even then it's a mess, especially if you're dealing with modules and jlink. Actually, using jlink without these broken plugins is the closest thing to hell. I'm not even going to elaborate further because I've explained this many times (https://news.ycombinator.com/item?id=47423383).
Golang's or especially Rust's tooling on the other hand are simply beautiful. Even .NET is fine now.
If it could help you, we're using JIB. We're on Scala and so we're using sbt-jib. Zero issues, works like a charm, deployment (on k8s) is just not a problem for us. But I would expect that it should work well for Java too.
Server-Side/Desktop JavaScript, I think, has a much better/more polished deployment experience than Java...at the cost of having a much worse build (ES module hell, bundler hell...) and maintenance (dependency hell, security hell, constantly changing APIs hell...) experience than Java, which excels at both of those.