Looks like the marketing copy is off from the actual implementation, AFAICT.
Sensei (先生) means teacher or professor in Japanese(http://en.wikipedia.org/wiki/Sensei).
It shares the same pronunciation and writing with the Chinese word that has the same meaning. This name indicates that the system can be used in place of Oracle database in many applications.
Okay, so on the name alone I can replace my Oracle database! Great!
But they built the entire thing around "eventual consistency."
And statements like:
"Sensei provides a high-level of durability by maintaining N replicas of each shard to guarantee a level of availability and fault-tolerance"
Don't seem to make sense when talking about ACID given that a write operation will happen at some point. Looks like the data event producers will shard the data across N replicas without quorum... so there's no guarantee that there will be N replicas available... is that right (and that the transaction won't be lost mid-stream either)?
Skimming through the source it doesn't seem to be doing anything terribly revolutionary... and I can see the usefulness of the trade-offs they made in this database for certain scenarios. However I don't think the claims of ACID guarantees and "real time" are particularly representative of what this DB will actually do. They just don't seem to jive with "eventual consistency" models.
I'm not a hardcore database guru though so maybe I'm missing something?
Where does that page claim perfect ACID semantics? It's meant to describe what Sensei gives you for each aspect of ACID.
Sensei needs an event stream to process. We've open-sourced and apache-fied Kafka which is a great candidate for an event stream. For Atomicity and Isolation, the event stream must provide these guarantees.
Consistency is handled with a routing parameter. Requests partitioned around an id will always go to the same searcher, so they won't go backwards in the stream except in failure scenarios. This is eventually consistent, but tries to keep things sane.
Durability: The event stream helps with this. We don't immediately flush while indexing in Lucene, so if there's a crash we can replay the persistent event stream.
Does this make sense? SenseiDB is not intended for purely transactional processing. For some applications, sensei would make a good candidate for replacing your DB. For others, not so much.
Where does that page claim perfect ACID semantics? It's meant to describe what Sensei gives you for each aspect of ACID.
And they're all well and good features! I can tell SenseiDB isn't transactional. Like I said, I skimmed the source and understand at a high level what it's does. I could see it being very useful in certain conditions as LinkedIn currently does and I'm sure others will.
However, I think the copy is confusing (at least it was for me). On the guarantees page there's an "ACID-ity" headline. For each aspect of ACID, as you say, the page describes what Sensei offers. The confusing part was that I was mentally comparing each aspect against what I understand to be the common semantics of ACID. I think it would be more clear if there was some distinction under the main headline that acknowledges this difference.
I think the structure of the page is probably confusing people. The big header on the page is "Data guarantees: Things we promise on how we manage your data," and the first subheader underneath that is "ACID-ity". This strongly (even if unintentionally) suggests that you guarantee ACID-compliance.
I'd suggest rewriting the copy as something along these lines:
# Data guarantees: How we manage your data
## Not quite ACID (and that's not a bad thing)
Although the principles of ACID are important, strict conformance has its costs. Sometimes it's the right tradeoff — but not always. By relaxing those requirements, Sensei can offer superior performance and durability. Here's how we approach the four ACID principles, and why you might prefer to do things Sensei's way:
Oddly enough, I think the linguistic bit is inaccurate as well. I don't believe there's any dialect of Chinese where sensei is pronounced the same as in Japanese — it's pronounced "sehn-say" in Japanese, "shyan-shung" in Mandarin and (IIRC) "seen-son" in Cantonese. (Also, though it kindasorta means "teacher," the word is more closely equivalent to "mister" in modern Chinese. Actual teachers are laoshi.)
My apologies for the confusion! The intent of the page is to provide a transparency of data guarantees in the database language. And the goal is to provide an explanation of trade-offs between respect performance and data distribution and the strong ACID guarantees of a database. Any suggestions of wording changes is greatly appreciated!
Seriously though it goes on...
They claim the database is ACID: http://javasoze.github.com/sensei/data-guarantee.html
But they built the entire thing around "eventual consistency."
And statements like:
Don't seem to make sense when talking about ACID given that a write operation will happen at some point. Looks like the data event producers will shard the data across N replicas without quorum... so there's no guarantee that there will be N replicas available... is that right (and that the transaction won't be lost mid-stream either)?Skimming through the source it doesn't seem to be doing anything terribly revolutionary... and I can see the usefulness of the trade-offs they made in this database for certain scenarios. However I don't think the claims of ACID guarantees and "real time" are particularly representative of what this DB will actually do. They just don't seem to jive with "eventual consistency" models.
I'm not a hardcore database guru though so maybe I'm missing something?