Drizzle - A Microkernel DBMS for High Performance Scale-Out Applications
Blog | Bugs | Build Farm | Events | Feature Requests | IRC | LCOV | Mailing List | Planet Drizzle | Source Code | Source Docs | Wiki

Contents 1 Usage 2 Installation 3 MicroKernel Architecture 4 Benchmarks 5 DTrace and Drizzle 6 Table Raw Reader 7 Drizzle Storage Engine API 8 Drizzle Plugins 9 Drizzle Executioner 10 Drizzle Parser 11 Drizzle Lex 12 Drizzle Scheduler 13 Event Scheduler 14 Drizzle Connection Management 15 Drizzle Client & Protocol 16 Drivers 17 Drizzle Console 18 Stored Procedures 19 Triggers 20 Views 21 Prepared statements. 22 Variables with Functions for Defaults 23 Partitioning & Sharding 24 Information Schema 25 Performance Schema 26 Replication 27 Rabbit Replication 28 Transaction Log 29 Logging 30 Backup 31 Authentication, Authorization, Access 32 Parameter Changes 33 Memcached and Drizzle 34 BlitzDB and Drizzle 35 Gearman and Drizzle 36 Storage Engine Changes 37 Other 38 FRM Removed 39 Objects Removed 40 Commands Removed 41 Operators Removed 42 Procedures Removed 43 Keywords Removed 44 Alternatives to Removed Commands 44.1 SHOW ENGINES

Usage

• Drizzle is not designed for embedded use.
• Drizzle is optimized for massive concurrency

Installation

• There is no Windows version. Drizzle runs under any Unix system (currently tested on Linux, Mac OS X, and Solaris only).
• There is no 'scripts/mysql_install_db' or related command.
• There is no 'mysql' or related default system database.
• There is no UNIX socket, the only protocol is TCP/IP.
• drizzled defaults to the port 3306 (the official MySQL port) and speaks the MySQL protocol. There is also an officially assigned Drizzle port, and the protocol behind that port is currently being developed. At some point drizzled will default to this new port/protocol when it is stable.
• There is no timezone support. Use UTC.

MicroKernel Architecture

Drizzle is designed around the concept of being a micro-kernel. Many operations, and more coming, have been pushed out of the kernel and into pluggable components. The goal is a small, light weight, kernel which is both easy to maintain and easy to extend. Drizzle is written in C++ and makes heavy use of Standard Template Libraries (STL).

Benchmarks

As part of the Drizzle automated build infrastructure there are several benchmarks which are run in our staging environment after each code update

• Crashme - Announcement
• Dbt2
• Drizzleslap
• Random Query Generator - Documentation
• Sqlbench - Documentation - Announcement
• Sysbench - Documentation - Results

DTrace and Drizzle

Drizzle can be built with dtrace support as it contains the various probes necessary. You will need to add -–enable-dtrace during the configuration step. See http://posulliv.github.com/2009/09/14/using-dtrace-with-drizzle.html for more details.

Table Raw Reader

table_raw_reader is a tool which reads table proto information from disk and provides a very thorough analysis of the data. See  http://www.flamingspork.com/blog/2009/10/01/table_raw_reader-reading-the-table-proto-from-disk-and-examining-everything for more details

Drizzle Storage Engine API

• Storage Engine API Proposed Refactoring

Drizzle Plugins

• Loading dynamic plugins at start-up

Drizzle Executioner

• Redesign of Drizzle Execution Flow

Drizzle Parser

Drizzle's parser no longer knows or confirms engines. It simply parses the statement and lets the execution system validate the state or existence of the engine See Brian's blog for more details.

Drizzle Lex

• Refactoring Lex Structure

Drizzle Scheduler

• Pool of Threads Scheduler Design
• Discussion on Pool of Threads performance
• Discussion on design

Event Scheduler

This has not been ported to Drizzle at this time as a plugin.

Drizzle Connection Management

• Connection Handling in Drizzle

Drizzle Client & Protocol

libdrizzle is the client library for Drizzle which provides the protocol and client interface and has been completely rewritten. It supports both Drizzle and MySQL protocols, so you can use it with MySQL servers as well. This new library provides complete non-blocking I/O support, a concurrent query interface, improved buffering support and also provides a complete server protocol implementation - not just the client.

• libdrizzle

Drivers

• C/libdrizzle
• C++ - in development
• JDBC
• Lua - in development
• Perl
• PHP
• Python
• Ruby - in development

Drizzle Console

The drizzle server can be started with a debug console by specifying the –console-enable flag. This will start up drizzled and provide a command line prompt from where you can enter sql commands. For more details see http://oddments.org/?p=96  and  http://oddments.org/?p=96 and http://oddments.org/?p=147

Stored Procedures

Drizzle is being re-architected to use languages such as Python, Perl, Ruby, PHP, and Java as server side languages.

Triggers

Work is being done to create per column callback functions.

Views

Blueprint for rewrite: https://blueprints.launchpad.net/drizzle/+spec/query-rewrite

Prepared statements.

The new protocol work is addressing the previous historical issues around memory allocation that made the previous system unusable in most cases. Full support for client side is still available.

See https://lists.launchpad.net/drizzle-discuss/msg05127.html   and  https://lists.launchpad.net/drizzle-discuss/msg05124.html for the latest discussion

Variables with Functions for Defaults

See  https://lists.launchpad.net/drizzle-discuss/msg05094.html  and  https://lists.launchpad.net/drizzle-discuss/msg05117.html for the latest discussion

Partitioning & Sharding

Sharding is being added to the new Drizzle protocol. This will allow the clients to specify a key with a session or query, and this key can be acted upon at a few different levels. The client side can act on this key and choose the correct server from a list of servers, treating your server list as buckets in a distributed hash table. The key could simply be passed along as well, and an intermediate proxy could act on the key by doing a lookup for you and redirect to the appropriate server. Third, the server could be the component to act on the key and somehow handle sharding internally, such as using redirection or possibly in the storage engine layer for network-based storage engines. Keeping the sharding key opaque and in the core protocol allows for any of these options, and others in the future.

Information Schema

• Removed references to event fields information, {user, schema, table, column, table constraints, paramaters} privileges fields info
• Use Yes/No instead of True/False for status to follow the SQL standard

Performance Schema

coming soon...... will be looking at prototyping an example for the Bell release. A proposal blueprint is in launchpad

Replication

Replication has been redesigned to use Google's ProtoBuffer library as a message format. This makes it simple to write new replication clients. Currently replication plans include plugins to write to a file and to send messages to a Gearman server. The internals of replication now work on tuple changes, thus removing any deterministic behavior requirements.

• Replication Architecture
• The Command Message
• Developing a Replication Plugin
• Group Commit
• Transaction Log
• Gearman replicator for Drizzle
• Replicating from MySQL to Drizzle and beyond
• Replicating from MySQL to Drizzle using Tungsten Replicator

Rabbit Replication

• Launchpad site
• Introduction
• Drizzle Replication using RabbitMQ as a transport
• Replication from drizzle to memcached / project voldemort
• Drizzle persistence in Project Voldemort
• Better replication from drizzle to cassandra
• Cassandra support in rabbitreplication
• Hadoop  (coming soon)
• Drizzle replication to WebSockets
• Multi-threaded Replication Appliers
• Drizzle to Infinispan replication

Transaction Log

• Transaction Log and Information Schema

Logging

The "general" and "slow" query log files have been dropped in favor of a new pluggable interface. Currently support for logging to a file, Gearman, and syslog are supported. A regular expression can also now be applied to log files so that filtering can be accomplished. See Ronald's blog for detailed examples.

Backup

This is in planning. Currently file system snapshots and "dumps" are supported. Online backup support if available for the Archive engine, and there are plans to incorporate one of the third party systems for other engines.

Authentication, Authorization, Access

Support for PAM and HTTP AUTH are currently provided (and via PAM LDAP).

Parameter Changes

• -p means port and -P means password. This is the opposite way around to MySQL but more like other *nix applications.
• TO_DAYS returns a day number based on the number of days since year 4713 B.C. not the year 0 as MySQL uses. This follows the convention of what most other major databases use as well.

Memcached and Drizzle

A plugin provided by Padraig allows you to query memcached statistics from an INFORMATION_SCHEMA table. See http://posulliv.github.com/2009/09/29/viewing-memcached-statistics-from-drizzle.html  and  http://posulliv.github.com/2009/09/19/using-memcached-with-c.html for more details

BlitzDB and Drizzle

BlitzDB is a storage engine that aims to be highly efficient for general purpose use. The current goal of BlitzDB is to become the modern choice of a non-transactional engine and bring healthy competition to MyISAM and MEMORY engines. BlitzDB is currently developed with Drizzle.

• Homepage
• BlitzDB and Keyless Tables
• BlitzDB and Tokyo Cabinet Concurrency Model
• 2009 summary

Gearman and Drizzle

A plugin is available to use Gearman for logging Drizzle query information.

• Gearman logging plugin
• Gearman Replication plugin

Storage Engine Changes

The default storage engine is InnoDB. Drizzle expects that engines have a higher degree of underlying architecture (be re-entrant/etc)

• The MERGE storage engine has been removed.
• The FEDERATED storage engine has been removed (work has been moved to FederatedX)
• The BLACKHOLE storage engine is not installed by default. You may install it by using ./configure --with-plugins=blackhole
• CSV is now a TMP only engine.
• MyISAM is now a TMP only engine.
• Archive is fully supported (and has been updated to Asynchronous IO)
• PBXT is supported in its own branch.
• XtraDB is supported in its own branch.

Other vendors are currently porting their Engines to Drizzle. It is best to ask each maintainer on the current state of their engine.

Other

Drizzle has removed the FRM system and now delegates to engines for data dictionary. "THD" has been removed in favor of a Session object which is now separated from a "thread" thus allowing new scheduler models that allow for better use of modern SMP hardware. The scheduler itself is now pluggable, and several alternatives are under design.

FRM Removed

• Drizzle, The Death of FRM
• table_raw_reader
• Drizzle FRM replacement: the table proto
• The Schema protobuf message: Drizzle’s metadata on a schema
• The Table protobuf message format

Objects Removed

• There is not longer a requirement for a "mysql" schema.
• There is no SET datatype, use enum.
• There is no SET NAMES command, everything defaults to UTF8
• There is no CHARSET or CHARACTER SET commands, everything defaults to UTF8
• There is no TIME type, use DATETIME or INT.
• There is no TINYINT, SMALLINT or MEDIUMINT. Integer operations have been optimized around 32 and 64 bit integers.
• There are no TINYBLOB, MEDIUMBLOB and LONGBLOB datatypes. We have optimized a single BLOB container.
• There are no TINYTEXT, MEDIUMTEXT and LONGTEXT datatypes. Use TEXT, same as BLOB.
• There is no UNSIGNED (as per the standard).
• There are no spatial data types GEOMETRY, POINT, LINESTRING & POLYGON (go use Postgres).
• No YEAR field type.
• There are no FULLTEXT indexes for the MyISAM storage engine (the only engine FULLTEXT was supported in). Look at either Lucene, Sphinx, or Solr.
• No "dual" table.
• The "LOCAL" keyword in "LOAD DATA LOCAL INFILE" is not supported

Commands Removed

• Most SHOW commands.
• drizzleadmin has been removed (the drizzle CLI now includes these operations)

• ALTER TABLE UPGRADE
• REPAIR TABLE
• CREATE FUNCTION (UDF's must be provided as shared objects)
• CONVERT (expr USING transcoding_name) is no longer valid with the USING clause since everything is now UTF8

Operators Removed

• BIT: &&,>>,<<,~,^,|,&

Procedures Removed

• Removed crypt(), bit_length(), bit_count()

Keywords Removed

CIPHER, CLIENT, CODE, CONTRIBUTORS, CPU, DEFINER, DES_KEY_FILE, ENGINES, EVERY, IO, IPC, ISSUER, BIT_AND, BIT_XOR, BIT_OR

Alternatives to Removed Commands

SHOW ENGINES

drizzle>> SELECT * FROM information_schema.plugins WHERE LOWER(plugin_description) like '%engine%';