I've been using Java for backend/net crawl tasks since about 2001. It definitely improved drastically with the addition of nio, and there were some irritating segfault issues a few years ago, but nothing a rollback to earlier JVM didn't fix (Until sun fixed it).
You can certainly run for months without issue (memory/crash/speed) as long as you don't have any leaks in your own code.
I'm pretty sure Java is still widely used at Google.
If I was writing the google crawler from scratch today, I'd certainly start with Java, then probably use perl/python for less critical scripting glue, and maybe rewrite any CPU intensive stuff in C/asm.
the perl runtime is far more stable than java. I've never encountered perl to crash while it happens that java simply gives up the ghost and goes belly up from time to time (not very often though)
You can certainly run for months without issue (memory/crash/speed) as long as you don't have any leaks in your own code.
I'm pretty sure Java is still widely used at Google.
If I was writing the google crawler from scratch today, I'd certainly start with Java, then probably use perl/python for less critical scripting glue, and maybe rewrite any CPU intensive stuff in C/asm.