NoSQL: The Love Child of Google, Amazon and … Lotus Notes

CouchDB creator Damien Katz. Image: Damien Katz

Most students of the web trace the NoSQL movement back to Google and Amazon.

As they grew their enormously successful online services, Google and Amazon needed new ways of storing massive amounts of data across an ever-growing number of servers, so each created a new software platform that could do so. Google built BigTable. Amazon built Dynamo. And after these internet giants published research papers describing these sweeping data stores, so many other outfits sought to duplicate them.

The result was an army of “NoSQL” databases specifically designed to run across thousands of servers. These new-age software platforms — including Cassandra, HBase, and Riak — remade the database landscape, helping to run so many other web giants, including Facebook and Twitter, but also more traditional businesses.

“If you look at every NoSQL solution out there, every one goes back to the Amazon Dynamo paper or the Google BigTable paper,” says Jason Hoffman, the chief technology officer at cloud computing outfit Joyent. “What would the world be like if no one at Google or Amazon ever wrote an academic paper?”

Well, the world would still have CouchDB, one of the oldest NoSQL databases. CouchDB creator Damien Katz wasn’t inspired by Google or Amazon or any other web giant. He was inspired by Lotus Notes, an online collaboration platform originally developed in the 1970s and ’80s.

Although Notes is best known as an email system, it was more than that. It was a foundation for building applications that depended on databases — i.e., organized collections of information. Using Notes, businesses built everything from expense-reporting applications to IT help desk tools. Katz was among those who built such applications — he got his start developing Notes apps for Lotus itself, in 1995 — and he says that even then, the platform demonstrated many of the same characteristics that have made today’s NoSQL databases so successful.

Like its NoSQL successors, Notes went outside the scope of relational databases — traditional databases that store information in neat rows and columns. “It was a sophisticated system that made it easy to do things that are hard to do with relational databases,” Katz says.

In many ways, Katz’s story can help explain the NoSQL movement — and why these databases are so different from what came before. Despite the movement’s undoubted success, the notion of a NoSQL database is still so hard to pin down — “NoSQL means so many different things, depending on who you’re talking to,” Google distinguished engineer Andrew Fikes recently told us — and many across the tech industry have yet to grasp the importance of these new database creations.

“NoSQL” is a misnomer. NoSQL databases aren’t designed to abandon SQL, the structured query language used pull information from traditional databases such as Oracle and MySQL. A better name would be “non-relational database.” NoSQL databases don’t use the neat tables of data that underpin relational databases.

These database have two basis characteristics: They can stretch across many servers — letting you expand your operation as need be, even across different geographical locations — and they give you the freedom to structure your data how you like. It’s this second characteristic that so strongly echoes Lotus Notes.

Platonic Ideal

The Notes platform was inspired by PLATO Notes, an online community that ran on the PLATO mainframe at the University of Illinois. PLATO Notes creator David R. Woolley wrote in 1994 that the project began in 1973 as a simple bug reporting system. Originally, users reported bugs simply by editing a text document, but this led to a few problems.

“There was no security at all. It was impossible to know for sure who had written a note,” Woolley wrote. “Most people signed or at least initialed their comments, but there was nothing to enforce this. And occasionally some joker would think it was fun to delete the entire file.”

So Woolley — then just 17 years old — was assigned to create a more structured system for reporting bugs. The tool he developed let users type their bug report into an application which would save the report into a file, along with the user’s name and the date of the submission. The support staff could then display the notes and add responses, which would be added to the same file. Woolley also added two more sections: “System Announcements” and “Public Notes.” General Notes was a message board that enabled users to post and respond to messages on any topic.

Woolley’s method of saving messages to a file, instead of a relational database, was a predecessor to the modern “document database.”

You can think of a relational database as a big spreadsheet. Data is organized into tables, columns, and rows. If you want to add a field, you add a column, and that column appears in every row for that particular table. This keeps your data structured and uniform, but it’s more difficult to manage lots of unstructured data or data that’s structured in multiple ways.

A document database is more like a collection of documents. Each entry is a document, and each one can have its own structure. If you want to add a field to an entry, you can do so without affecting any other entry.

Soon, PLATO developers were adding more applications. By 1974, they had an e-mail application, a chat room, online games, and more.

In 1984, Ray Ozzie — a Lotus developer who had worked on PLATO while attending the University of Illinois — left Lotus to start a company called Iris Associates. Lotus then funded Iris with the agreement that it would have exclusive rights to the company’s flagship product: a system for corporations that was based PLATO.

Today, many see Lotus Notes as a legacy system ready to be relegated to the same dustbins as WordPerfect and Novell Netware. But Notes paved the way for just about every type of corporate communication and collaboration application that came after it, from e-mail clients like Microsoft Outlook to the social network tools like Jive Software to, yes, CouchDB.

Pages: 1 2 View All