NoSQL databases have become increasingly popular over the last decade as web-based applications and large datasets have proliferated. Also known as "not only SQL" or "non-relational" databases, NoSQL provides more flexibility and scalability than traditional relational databases. In this beginner‘s guide, we‘ll cover the key concepts of NoSQL databases and why they are often better suited for modern applications.
What is NoSQL?
NoSQL encompasses a wide variety of database technologies that store data in ways other than the tabular relations used in relational databases like MySQL and PostgreSQL. Some of the key characteristics of NoSQL databases include:
Flexible data models: NoSQL databases do not require fixed table schemas, allowing developers to store unstructured, polymorphic, or nested data. This flexibility is useful for handling unpredictable data types.
Scalability: NoSQL databases are designed to scale out easily across cheap commodity servers or even across data centers. This makes them suitable for cost-effectively handling massive datasets and heavy request loads.
High performance: By giving up some transactional guarantees from relational databases, NoSQL databases can offer extremely fast read and write access. This performance advantage allows them to handle heavy request volumes.
High availability: NoSQL architectures often replicate data across data centers and cloud availability zones. This replication improves availability and protects against failures or outages.
So in summary, NoSQL provides flexibility in data models, scalability to handle big data applications, speed for performance-critical systems, and availability crucial for always-on applications. These advantages make NoSQL databases well-suited for web, mobile, gaming, IoT and big data use cases.
Drawbacks of Relational Databases
For decades, relational databases like MySQL, SQL Server and Oracle DB have dominated as the primary data storage technology for applications. So why has NoSQL become so popular? To understand that, we have to recognize some of the drawbacks of relational databases:
Rigid schemas: Relational databases require developers to define schemas upfront to determine the precise tables, columns, foreign keys, and relationships. This rigid structure makes it hard to store or query polymorphic data.
Scaling difficulties: Adding more servers only provides vertical scaling to a point for relational databases. Handling big spikes in traffic or massive datasets is difficult without major provisioning overhead.
Join complexity for graphs: Querying connected graph data leads to complex, slow join operations in relational databases. The fixed schema also struggles to map graph relationships efficiently.
No built-in caching: Relational databases lack built-in caching for fast data access. Added caching layers adds more complexity for developers.
By giving up some data integrity guarantees from relational models, NoSQL provides more flexibility and scalability critical for modern applications with unpredictable usage patterns and data structures.
NoSQL Database Usage in Industry
While NoSQL encompasses many database categories, some of the most widely used NoSQL databases in industry include MongoDB, Apache Cassandra, Redis, Apache HBase, Neo4j, Couchbase and DynamoDB. Large internet companies pioneered early NoSQL adoption, but modern cloud services have dramatically expanded mainstream usage.
According to DB-Engines rankings, MongoDB is the most popular NoSQL database managing over 13 million databases across industries like technology, finance, media, healthcare and government. Cassandra and Redis usage has also grown substantially in recent years. Major companies across many verticals now use MongoDB or Cassandra as their operational database powering customer-facing applications and services at massive scale – including over 80% of Fortune 100 companies. For example, leading brands powered by NoSQL databases include:
Retail & Commerce – Walmart, eBay, BestBuy, Target, IKEA
Finance & Fintech – Goldman Sachs, HSBC, Robinhood, Paypal, Venmo
Technology – Cisco, IBM, HP, Dell, AMD
Gaming & Media – Sony, EA Games, Disney, Hulu, Tencent
Travel – AirBnB, SkyScanner, Tripadvisor
NoSQL usage in industry spans from popular SaaS applications that need flexible schemas for multi-tenant support, to global banks that rely on high uptime and absolute data integrity guarantees even at extreme scale and throughput levels.
While relational and mainframe-based systems still play a major role across many traditional enterprises, most growing digital applications have rapidly transitioned to modern NoSQL databases as their systems-of-record due to the aforementioned benefits like horizontal scale and schemaless flexibility. Even conservative enterprises in finance, healthcare and government are now adopting document and graph oriented NoSQL both for internal line-of-business systems as well as customer-facing services.
Types of NoSQL Databases
There are four main types of NoSQL databases, each using a different underlying data model:
Key-Value Stores
Key-value stores function like giant hashmaps, where you associate key strings with value data blobs. They act much like get/set operations in programming languages. Popular examples include Redis, Aerospike and Amazon DynamoDB.
Use cases: High-performance caching, session management, user profiles
Example platforms: Redis, Memcached, Aerospike, DynamoDB
Document Databases
Document stores organize data storage in documents similar to JSON or XML objects. Related data is stored together in these variable-schema documents for easy distribution and scaling. Leading examples are MongoDB and Couchbase.
Use cases: Content management, catalogs, user profiles
Example platforms: MongoDB, Couchbase, Cosmos DB
Column-Family Stores
Designed to store very large volumes of data across distributed commodity servers, column stores organize data by columns instead of rows. Columns can be logically grouped into column families. Cassandra pioneered this model.
Use cases: Logging & metrics, ecommerce transactions, IoT data
Example platforms: Cassandra, HBase, Google BigTable
Graph Databases
As the name suggests, graph databases structure storage around nodes in a graph and the relationships between those nodes. By storing connections, deep graph traversals are fast. Examples include Neo4j and Amazon Neptune.
Use cases: Social networking, recommendations, geospatial
Example platforms: Neo4j, TigerGraph, JanusGraph, Amazon Neptune
This taxonomy covers the most popular categories of NoSQL databases available today across both commercial and open source options. There are additional emerging database types as well such as time-series databases optimized for handling timestamped IoT and metrics data.
NoSQL Database Performance
A key motivation behind NoSQL adoption has been substantial performance gains over traditional relational database systems. By relaxing some ACID constraints and introducing optimistic concurrency control, NoSQL databases can offer order-of-magnitude improvements in throughput and latency.
For example, benchmark tests using the Yahoo Cloud Serving Benchmark (YCSB) framework have shown 10-100x higher throughput in MongoDB compared to leading relational databases:
MongoDB throughput compared to MySQL and Postgres (Source: MongoDB)
These massive performance gains allow NoSQL databases like MongoDB to handle enormous request volumes and data sizes even on commodity infrastructure. Netflix, for example, handles over 2 billion requests per day across a 500+ TB cluster on MongoDB.
By using a scale-out architecture optimized for modern hardware and cloud infrastructure, NoSQL databases outperform relational databases significantly in real-world production environments. Their high throughput unlocks use cases with demanding performance needs like fraud analysis, real-time analytics, and IoT data processing.
Scaling Databases with NoSQL
A key driver behind NoSQL adoption has been the difficulty of scaling relational database infrastructure. As datasets grow exponentially, NoSQL databases make it easier to scale out horizontally across cheap commodity servers. Their flexible data models and native replication also remove a lot of manual database administration overhead.
Here are some of the typical ways NoSQL databases help scale applications more easily:
Sharding – Sharding refers splitting up databases across multiple servers, making it possible to scale out across hundreds of commodity servers. MongoDB, Cassandra, Redis and others make sharding simple and transparent.
Rebalancing – For evenly distributing data and requests across shards, databases need to rebalance and migrate data automatically. This helps maintain performance as datasets grow.
Replication – By replicating data across multiple servers, NoSQL databases can ensure high availability and improve read speeds for distributed users.
Eventual Consistency – Instead of guaranteed ACID transactions, NoSQL systems typically use optimistic replication approaches to ensure fast writes. Data is eventually consistent across replicas.
See AlsoData Modelling with MongoDB
Together these capabilities allow NoSQL databases to painlessly scale out on cheap commodity infrastructure instead of hugely expensive high-end relational database servers.
For example, leading media company Sony operates over 2000 game titles running on MongoDB across multiple data centers, storing 1.3+ trillion datasets supporting 400+ million monthly users. This massive scale would be impossible economically with traditional relational databases.
By natively sharding and replicating data automatically, Sony can painlessly scale capacity horizontally without any schema migrations. They can also take advantage of cloud bursting capabilities to handle seasonal spikes in gaming traffic. Such limitless scale-out with zero downtime exemplifies the advantages NoSQL provides for write-heavy media applications.
Cloud-Native Database Advantages
The rise of cloud platforms like AWS, Google Cloud and Azure has also accelerated NoSQL adoption. Cloud-native databases designed specifically for cloud infrastructure offer many advantages:
Automatic scalability – Services like DynamoDB and Cosmos DB autoscaling instantly to handle demand spikes or data growth.
Serverless operation – Fully managed services that do not require managing database servers yourself.
Global distribution – Data is replicated across regions to keep latency low for users worldwide.
Pay-per-use pricing – Usage-based billing instead of heavy upfront licensing and infrastructure costs.
By reducing operational overhead and manual database administration, cloud-based NoSQL services have made it even easier for companies to transition from legacy relational databases. Leading examples include AWS DynamoDB for key-value data, Atlas cloud service for MongoDB and Azure CosmosDB providing turnkey globally distributed operational databases. Such fully managed services accelerate time-to-market by letting development teams focus on applications without any backend database DevOps.
According to 2018 research by IDC:
By 2022, worldwide enterprise IT spending for cloud-native databases on public cloud platforms will reach $20 billion – growing 5x faster than spending for traditional databases.
This rapid growth will be driven by enterprises migrating existing applications as well as building new cloud-native applications leveraging NoSQL directly.
Examples of NoSQL Usage
Here are some examples of popular modern applications powered by NoSQL databases:
Netflix – Uses Cassandra for all customer streaming data, recommendations and analytics
Uber – MongoDB stores trip data and analytics as well as geographical data and user profiles
Pinterest – Redis manages the home feeds for 125+ million users
Airbnb – Cassandra runs all online payments, search and messaging
Twitch – Cassandra handles 120,000 writes per second for live streaming
Almost all leading web and mobile applications have transitioned to NoSQL databases due to their flexibility and scalability. Even data-driven giants like Google, Facebook, Amazon and Twitter rely extensively on customized NoSQL infrastructure.
While relational databases still dominate across traditional enterprises currently, analysts project over 50% of new enterprise applications will use NoSQL databases like MongoDB and Cassandra by 2022. This shift is being driven by digitally transforming companies who need to innovate faster leveraging modern data architectures.
NoSQL Data Modeling
The flexible schemas of NoSQL provide developers more latitude when modeling data for applications. But some guidelines help ensure optimal querying and storage efficiency long term:
Denormalization – Unlike strict relational modeling, duplicate joined data across documents and collections instead of foreign keys for faster access.
Application-Driven – Model documents and column families around application entities and access patterns rather than strict normalization.
Polymorphism – Take advantage of dynamic schemas to store derived, polymorphic or variable attribute sets in a single collection.
Scoped Relations – Some explicit one-to-many and one-to-one relationships can minimize need for client-side joins.
Blending some relational modeling experience while allowing flexibility is an effective approach for experienced developers getting started with NoSQL.
Document Database Modeling
For document databases like MongoDB, data for a given entity like customer or product is typically stored together in a single variable-length JSON-style document. Related data that is always accessed together avoids expensive application-side joins. Polymorphic data can also be stored without adding sparse columns common in relational modeling.
Referencing between documents should leverage manual linking rather than foreign keys. Document sizes can vary drastically within collections based on complexity of data being stored.
Column Family Database Modeling
For wide column stores like Cassandra, data model revolves around column families which function as containers for rows of related data. Primary key design is crucial to ensure write scaling and query efficiency. Within column families, groups of related attributes can be stored together in dynamic columns.
Secondary indexes need special consideration to avoid performance pitfalls. Column Cardinality ratios determine which attributes become column names versus cell values for optimal storage and caching.
Graph Database Modeling
For graph databases like Neo4j, data modeling focuses on identifying the nodes, relationship types, and access patterns to store connection-centric data. Nodes typically represent entities like customers or devices. Relationships describe how nodes connect or associate along context-specific semantic types like “FRIEND_OF” or “INSTALLED_IN” etc.
Unlike relational schemas, the connections are first class citizens, allowing graph algorithms to rapidly traverse millions of these relationships across the entire dataset for fast recommendations and AI/ML model training.
Integrating SQL and NoSQL
While NoSQL alternatives offer clear advantages for operational systems and analytics, decades of existing data and BI infrastructure revolve around SQL interfaces. Modern data architectures are increasingly using hybrid approaches:
- Maintain relational systems-of-record while offloading analytics to NoSQL data lakes
- Implement Lambda architectures with batch + real-time analysis on NoSQL, served back to SQL for reporting
- HTAP databases that combine OLTP and OLAP to eliminate need for ETL and duplication
Using NoSQL as the ingestion layer and persistent storage foundation enables building clean abstraction layers or microservices for SQL-based consumption:
NoSQL data lake ingestion with integrated SQL analytics access pattern (Source: AWS)
By leveraging NoSQL where it provides the most benefit while still giving SQL interfaces access to the raw data, enterprises get the best of both worlds – extreme scale, performance and flexibility while retaining compatibility with existing tooling and user skills.
Key Takeaways
Here are the most important things to know about NoSQL databases:
NoSQL encompasses document, key-value, column-family, graph and other non-relational databases.
They provide more scale-out capability, performance and flexibility compared to relational databases.
Leading options include MongoDB, Cassandra, Neo4j and Amazon’s cloud databases.
Major advantages are flexible schemas, easy scaling, fast reads/writes and high availability.
NoSQL is well suited for web, mobile, IoT and big data applications.
Usage spans major industries like retail, finance, media, healthcare and transportation.
Hybrid SQL + NoSQL approaches help leverage data lakes while retaining BI stack integration.
For modern applications with unpredictable workloads and data models, NoSQL provides better performance, scalability and developer productivity than sticking to rigid relational databases. Companies from startups to tech giants have embraced NoSQL to scale their web and mobile applications cost-effectively. While early adoption was concentrated at internet companies, modern cloud services have dramatically expanded mainstream enterprise usage of document, column-family and graph databases. By combining NoSQL flexibility with integrated SQL analytics access, organizations can build agile data architectures ready for the next decade of digital transformation.
