Configurator for MySQL Cluster - version 2.7

If you only want to run localhost, please use the Sandboxes!

If you haven't used the tool before, the check the readme for requirements etc!

If you want to upgrade to these scripts from version 2.3, then use 'sh download-and-compile.sh' and then 'sh gen-hostfiles.sh.

If you don't run 'download-and-compile.sh' you need to copy the binary tgz of mysql cluster into mysqlcluster-xx/repo and in create a file called scripts/install/.s9s/latest

The content of scripts/install/.s9s/latest should be the filename of the binary filed you just copied into 'repo'.

But you must run './gen-hostfiles.sh' if you upgrade from an earlier package from Configurator.

Running the scripts

Run the scripts as ./scriptname or bash scriptname. sh scriptname does not work on all distros

Configurator - build your Clustered Datastore

Cluster version:		We recommend MySQL Cluster 6.3 for general purpose. MySQL Cluster 7.0 has lot of new cool features (e.g multithreading, online add node). 6.2 is no longer recommended, as 6.3 is so much better, see here for more information about the differences. There are prebuilt binaries for both versions, but the source releases are made more frequently.
Number of management servers		Two management servers are recommended in a HA setup.
Number of Data Nodes:		Each data node requires two cores. You should have atleast two computers (for two data nodes). Two or four computers (depending on the number of cores in each computer) for four data nodes. One data node is only for testing purpose only.
Number of APIs (sql nodes, slapds):		You should have atleast two MySQLD for a HA setup. The sum of nodes (management servers, data nodes and apis) must be less than 255 (max number of supported nodes in Cluster 6.2 and Cluster 6.3).
DataMemory:	MB	The OS will require about 1GB of RAM. Approximately 900MB will be used by internal buffers in the Data node. The IndexMemoryis derived from the DataMemory and will be 12.5% of the DataMemory. This is enough for most applications since only the hash primary key is stored there (20B per record in the database). However, if you have a lot of UNIQUE indexes then you might have to increase this later (just edit the config.ini file). The minimum value for DataMemory is 80MB. Thus, if you have 8GB of RAM, then you could set DataMemory= 0.875*(8172MB-1024MB-900MB) = 5400MB.
DiskPageBufferMemory:	MB	By default this is 64MB. You can get this value from sizer. This should be as big as possible if you rely heavily on disk data. Also read the blog disk data - a summary, if you want to know more. Leave this as it is if you don't use disk data.
Number of tables:		The default setup will support 4096 tables objects. Each INDEX (and UNIQUE) and BLOB/TEXT attribute also counts towards a table object. If you plan to run more, then you should think about this and set this to an approximate. The maximum number of table objects supported in MySQL Cluster is 20320. Each table object takes about 20KB of RAM.
Replication role:		None - the cluster will not be used in replication Master - the cluster will act as the replication master (one or more mysqlds will produce binary logs) Slave - the cluster will act as a standby cluster
Multi-connection: (ndb_cluster_connection_pool)		Each Mysql Server will make X connections to cluster (it will look like X mysql servers has connected). Gives better performance - lower response times and higher throughput. More info in my blog
PortNumber		This is the port the management server listens on. 1186 is the default port.
Config directory		This is where the config.ini and my.cnf files will be stored. The 'user' specified below must have write access to this directory. The 'user' must exist on all machines. The plan is to make this dynamic, i.e, at run time of the scripts - but this was the quickest fix.
User		user='root': management servers and data nodes will be running as root, mysql servers as 'mysql' user='user': management servers, data nodes and mysql serves will be running as 'user'

Documentation

Go to the documention for more info and how to use the scripts.

Read this first

The Configurator lets you define a production quality MySQL Cluster and:

propose configuration files (config.ini and my.cnf

generate scripts for administration of MySQL Cluster from a single point

email the scripts in a tarball to you

It is important that you follow the instructions and don't make typos.

The configuration files are proposal and you might have to make additions to them depending on the type of load and queries you are running. In particular, the scripts are intended to be very simple so that you can modify them as you wish.

A set of scripts are also generated to let you:

download and compile the latest version of MySQL Cluster

distribute the Cluster software to the hosts that you have specified

bootstrap the cluster in order to create necessary directories and install the mysql server(s)

start and stop the cluster, as well as starting and stopping individual nodes

Please note that this has only been tested on Linux-based systems and you are strongly adviced to setup shared SSH keys in order to avoid typing the password a lot of times. You also need to have root credentials to take advantage of some features (lock memory to avoid swapping etc).

The scripts are intended to be executed on one of the nodes in the cluster, or another node connected to the cluster. This node is denoted as the front-end node.

Read the documention for more info about the scripts.

The download and compile scripts (used if you want to build from source - MySQL makes source releases are more frequent than binary builds) requires that the following is installed on front-end node (thus, this will likely not work on Solaris, but feedback would be appreciated):

gcc and g++

bash

wget

ncurses

same OS (kernel + CPU arch) since the scripts requires a homogenous cluster

At the end of the Configurator you will get instructions how to install the tools above on Ubuntu/Debian and Fedora/Redhat/CentOS

Current Limiations and Important notes

Max one mysql server per computer is allowed

DiskPageBufferMemory and SharedGlobalMemory should be increased if using disk data. Please read my blog about disk data and this one (more to come) and use the dimensioning toolkit to help you scope out the config.

I have not tried this on localhost. If you want to run on localhost only, please use the eval/development guide for this.

You must have root priviliges to run this. The scripts will create a "mysql" user account. The mysql servers will run as the mysql user. The data nodes as user "root".

Changelog

2009-May-20 - Version 2.6 - the scripts are now testes with big configurations of Cluster. A detailed description will be available soon of the new features. A lot of more error checking in the scripts has been added, and timeouts on every process so that it cannot get stuck. Also pid checks to check if a process is really there, and a lot of other things. Checkout tools/install-initd-linux.sh (completely reworked) or tools/execute-all-mysql.sh which executes a command on all mysql servers in the Cluster. Quite handy for GRANTs and such.

2009-May-15 - Version 2.5 - more info to come soon!

2009-May-15 - Version 2.4 - and now the d/l link works...

2009-May-15 - Version 2.4, Almost all have been updated..will write more detailed soon.

2009-May-5 - Version 2.3, improved error checking of user specified data. Move 'enable multi-connect' to first page (this page). Improved 'start-ndb_mgmd.sh' script. All scripts in the scripts are now started with ./scriptname instead of bash ./scriptname or sh ./scriptname which caused some problems on different platforms. Documentation was also updated.

2009-Apr-28 - Version 2.2, now with a "query-param" script to check the values of various parameters

2009-Apr-28 - Fixed issue with an path on the webserver (config2->config).. sigh.

2009-Apr-25 - Rewritten almost all scripts with better error handling and progress reports. I have also moved the install scripts to scripts/install and the monit scripts to scipts/extra. The install_initd_linux.sh uses chkconfig and it does not work on all linux distros. Going to fix that. When you have entered the email address, then you will also get a link where to download the package, in case you experience problems with receiving the email (I have raised an issue to the hosting company about this)

2009-Apr-07 - Fixed stupid bug I introduced during the weekend (4th of April), which made it impossible to generate the scripts.

Disclaimer: Please note that the scripts themselves are not Supported by MySQL nor Sun Microsystems!! The configuration files (config.ini/my.cnf) are Supported!

License

The scripts, configurations, and any other files generated by the Configurator is licensed according to GPL v2.

severalnines