(ELSA is a logger that claims to be able to handle 100000 entries/sec (!!))
When to Use Hadoop
This is a description of why Hadoop isn't always the right solution to Big Data problems, but that certainly doesn't mean that it's not a valuable project or that it isn't the best solution for a lot challenges. It's important to use the right tool for the job, and thinking critically about what features each tool provides is paramount to a project's success. In general, you should use Hadoop when:
Data access patterns will be very basic but analytics will be very complicated.
Your data needs absolutely guaranteed availability for both reading and writing.
There are inadequate traditional database-oriented tools which currently exist for your problem.
Do not use Hadoop if:
You're don't know exactly why you're using it.
You want to maximize hardware efficiency.
Your data fits on a single "beefy" server.
You don't have full-time staff to dedicate to it.
The easiest alternative to using Hadoop for Big Data is to use multiple traditional databases and architect your read and write patterns such that the data in one database does not rely on the data in another. Once that is established, it is much easier than you'd think to write basic aggregation routines in languages you're already invested in and familiar with. This means you need to think very critically about your app architecture before you throw more hardware at it.
(ELSA is a logger that claims to be able to handle 100000 entries/sec (!!))
When to Use Hadoop
This is a description of why Hadoop isn't always the right solution to Big Data problems, but that certainly doesn't mean that it's not a valuable project or that it isn't the best solution for a lot challenges. It's important to use the right tool for the job, and thinking critically about what features each tool provides is paramount to a project's success. In general, you should use Hadoop when:
Do not use Hadoop if: The easiest alternative to using Hadoop for Big Data is to use multiple traditional databases and architect your read and write patterns such that the data in one database does not rely on the data in another. Once that is established, it is much easier than you'd think to write basic aggregation routines in languages you're already invested in and familiar with. This means you need to think very critically about your app architecture before you throw more hardware at it.