LLVM is being used as a JIT in Open Shading Language, a DSL for advanced 3D renderers. The JIT + resulting code was so fast, the OSL team dropped their batch rendering API altogether. (The resulting shaders are 25% faster than the hand-coded C language shaders they replaced.)

Currently, the LLVM JIT-powered OSL is now responsible for 100% of the shading duties at Sony Pictures Imageworks. I'd say that's a resounding success for LLVM as a JIT.

I'm not sure how that refutes the point. OSL is a shading language, that's an environment dominated by runtime execution time, not startup latency. So it might be a "JIT" in that there's no stored binary, but it's performance criteria are much (much!) closer to those of gcc or clang than to, say, a JVM or script interpreter.

And, like the post said: LLVM IR is a compiler IR. It's a great fit here.

> I'm not sure how that refutes the point.

>> [LLVM] is designed to be a static compiler framework, it hardly fits other purposes [...]

I presented a ready example where LLVM was being used not as a "static compiler framework", and was being used for "other purposes". If that's not an outright refutation, it's at least a useful data point.

Right, and my point was that that ready example was as close to a "static compiler framework" as you can get without actually being one. It's an exception-that-proves-the-rule case.

OSL shaders are specialized dynamically at runtime and then JIT'd using LLVM. That's about as far from a static compiler framework like GCC or Clang as you can get, from my perspective.

In addition, fast JIT speed was a very high priority/need and the OSL team spent a few months carefully tuning the LLVM passes to give fast code at a low JIT cost. (All of this was done and discussed publicly; feel free to grep the mailing list.)

Would you mind explaining how OSL is as close to something like Clang or GCC as you can get? I just don't see it.

JIT compilers have to make a tradeoff between time spent on analysis and time spent on execution; this is what makes them hard. It makes them hard because it means they need different levels of optimization, depending on how often a piece of code is going to get executed. If the code is only going to get run once or twice, it usually doesn't make sense to translate it from an efficient interpreter encoding (which LLVM does not have); if it's going to run a few hundred times, it can get a little bit more analysis, while if it's going to be the core of a loop, it makes a lot of sense to run analysis that may take multiple milliseconds.

Using a static list of analyses that are always run before execution would be a static compiler approach, not a JIT approach. A JIT will generally profile the code, optimize it when it gets hot, and deoptimize it when assumptions made during optimization no longer hold (e.g. virtual calls in JVM being optimized to static calls because only one definition of a virtual method exists, but subsequently a new class is loaded that overrides that method). All this dynamic runtime modification of the code (not just initial compilation, but modification of existing, executing code) is where JIT technology is distinct from static compiler technology.

> Would you mind explaining how OSL is as close to something like Clang or GCC as you can get? I just don't see it.

It seems that Open Shading Language is not really Just-in-Time (JIT) compiled. It's compiled from LLVM IR to native machine code Ahead-of-Time (AOT) but at runtime and then executed. This is actually not very far from a static compiler, the only difference is that the target architecture is known at runtime and the LLVM IR is compiled to machine code using that information.

What is the difference of JIT and AOT here is that in a JIT situation, there is some form of an interpreter that is executing byte code of some kind (probably not LLVM IR). When this interpreter reaches a loop of some kind, it will attempt to compile it (from bytecode to LLVM IR and finally from LLVM IR to native code). The interpreter then calls the compiled code, which will run for as long as possible and finally return control to the interpreter which will continue interpreting until it finds another opporturnity for JIT'ing. A JIT compiler is typically employed when the original source language cannot be compiled statically to machine code, because of e.g. dynamic typing.

So from what I can tell, OSL is just a static ahead of time compiler, with the final stage of compilation taking place at runtime.

Please tell me if some of my background facts were incorrect (in particular about OSL).

I think this is correct. OSL is using LLVM in an AOT fashion, though a lot of runtime code generation and specialization is being done just prior to running the AOT LLVM JIT.

As opposed to something like, say, Google v8, which is using runtime feedback to make hot code paths fast (and to remove dynamism when it can be shown to be safe).

I guess I wasn't aware that people weren't including dynamic code generation and AOT compilation in the "JIT" category. To me JIT meant generating machine code "at runtime", and both OSL and v8 would be at opposite ends of the JIT "at runtime" code generation/compilation spectrum -- OSL on the AOT side and v8 on the keep-running-the-compiler side.

In src/liboslexec/llvm_instance.cpp, OSL uses a custom list of 13 LLVM optimizer passes (plus implicit passes pulled in as dependencies) -- less than what Clang uses at -O2 for C, but still a fair amount. Then it runs the full LLVM "-O2" backend. Even if the compilation happens "dynamically" from the perspective of the application, OSL is using many compiler features more commonly associated with "static" compilation.

JIT speed is relative. Shader programs are often executed many many many times, so OSL can likely afford to make different tradeoffs than many other JIT-using applications.

Shaders compile once and execute millions of times. Most code on the Web code compiles once and executes zero, one, or a small number of times. He's saying LLVM is less suited to the latter case because it compiles slowly (though the resulting code is fast).

In the case of pNaCl, startup time and code size issues could easily be addressed by providing the architecture specific versions server side. Either the developer provides those versions as well, or a third party web service could do the translation.

The LLVM IR format could just be the developer target so that things are actually portable, and we don't wind up in a situation where 90% of the NaCl plugins are x86-only.

> And, furthermore, the size of the bitcode is still enormous, comparable to the generated machine code or even worse (think about the size of C++ binaries)

Actually I believe the bitcode is much larger than C or C++ binaries. The issue is that LLVM bitcode is basically a statically-typed language, lower than C. So for example to convert a pointer from one type to another you need to do an explicit cast (using an LLVM 'bitcast' operation). However, when you compile all the way down to native code, you have no need for such niceties and you just copy the value. So the bitcode ends up significantly larger than the native code would be, in order to maintain static typing correctness. There are some other issues as well.

LLVM bitcode was not designed for size, it's - exactly as the article says - just a compiler IR. So it isn't designed for size, portability, JITing speed, or anything like that. Trying to morph it into those is problematic and worse, may lead to compromises in LLVM's core goals. Which would be a shame since LLVM is a damn good compiler IR!