Control Your Database Infrastructure
Severalnines
 

Announcing ClusterControl Support for MariaDB 10

 
July 22, 2014
By Severalnines

We just wanted to make it official: Severalnines ClusterControl now supports MariaDB 10!

 

As most of you know will know by now, MariaDB 10 is the newest and most advanced version of the popular MariaDB relational database system. Whilst remaining application-compatible with the MySQL database, it adds many new capabilities to address the most challenging web and enterprise application use cases. Cluster deployments would be based on MariaDB Galera Cluster 10, which is a complete merge of MariaDB 10.0.12 and Galera Cluster. 

 

ClusterControl for MariaDB Clusters

 

ClusterControl gives you the power to deploy, manage, monitor and scale entire MariaDB Galera clusters efficiently and reliably. ClusterControl acts as a virtual DBA assistant and frees system administrators and developers from the complexity and learning curves associated with database clusters. 

 

 

MariaDB European Roadshow - Summer 2014

 

We had the opportunity to showcase ClusterControl for MariaDB 10 recently during the European MariaDB Roadshow, which took us to Helsinki, Amsterdam, Paris, Hamburg, Berlin and Frankfurt. We were delighted of course to be invited by SkySQL as guest speakers and to have the opportunity to talk to existing and future MariaDB users. 

 

There is one more date scheduled at the moment for this roadshow - London in September.

 

Joint live webinar: Management & Automation of MariaDB Galera Clusters

 

If you’ve missed the European Roadshow and/or would like a more in-depth presentation and demo of ClusterControl, please do register for our joint webinar with SkySQL on 30th September 2014.

 


How to Configure ClusterControl to run on nginx

 
July 10, 2014
By Severalnines

ClusterControl uses the Apache HTTP Server to serve its web interface, but it is also possible to use nginx. nginx + PHP fastcgi is well-known for its capabilities to run on a small memory footprint compared to standard Apache + PHP DSO.

 

In this post, we will show you how to run ClusterControl on nginx web server by swapping out the default Apache web server installed during the initial deployment. This blog post does not mean that we officially support nginx, it just an alternative way that a portion of our users have been interested in. For instance, Phil Bayfield wrote a blog on the same topic a while back.

 

Apache Configuration

 

Before we jump into nginx configurations, let’s look at how the ClusterControl web application is configured with Apache web server. ClusterControl consists of two web components, a web UI and CMONAPI. These components are located in the Apache’s document root which might vary depending on the operating system.

 

1. Make sure ClusterControl UI and CMONAPI exist in the document root. Document root for RedHat/CentOS and Ubuntu 14.04 LTS (Apache 2.4) is located at /var/www/html while Debian and Ubuntu 12.04 and lower is located at /var/www. ClusterControl will be installed under this root directory and you should see something like this:

$ ls -l /var/www/html
drwxr-xr-x. 6 apache apache 4096 Jul  9 16:31 cc-cmonapi-1.2.6
drwxr-xr-x. 5 apache apache 4096 Jul  9 16:31 cc-ui-1.2.6
lrwxrwxrwx. 1 root   root     26 Jul  9 16:31 clustercontrol -> /var/www/html//cc-ui-1.2.6
drwxr-xr-x. 4 apache apache 4096 Jul  8 18:34 cmon
lrwxrwxrwx. 1 root   root     31 Jul  9 16:31 cmonapi -> /var/www/html//cc-cmonapi-1.2.6

 

2. Apache must be able to read custom configuration file (.htaccess) under the document root directory, by setting AllowOverride options to All. Example in /etc/httpd/conf/httpd.conf:

<Directory "/var/www/html">
    Options Indexes FollowSymLinks
    AllowOverride All
    Order allow,deny
    Allow from all
</Directory>

 

3. Require the following PHP modules to be installed and enabled:

  • common
  • mysql
  • ldap
  • gd
  • curl

The standard Apache installation via package manager will install PHP to run as dynamic shared object (DSO). Running on this mode will require you to restart Apache in case of php configuration changes.

 


New Webinar on July 9th: How To Set Up SQL Load Balancing with HAProxy

 
June 17, 2014
By Severalnines

 

We continuously see great interest in MySQL load balancing and HAProxy, so we thought it was about time we organised a live webinar on the topic!

 

As most of your will know, database clusters and load balancing go hand in hand. 

 

Once your data is distributed and replicated across multiple database nodes, a load balancing mechanism helps distribute database requests, and gives applications a single database endpoint to connect to. 

 

Instance failures or maintenance operations like node additions/removals, reconfigurations or version upgrades can be masked behind a load balancer. This provides an efficient way of isolating changes in the database layer from the rest of the infrastructure. 

 

Webinar: How To Set Up Load Balancing with HAProxy

 

Wednesday, July 9th 2014

Register now - Europe/MEA/APAC

Register now - North America/LatAm

 

In this webinar, we will cover the concepts around the popular open-source HAProxy load balancer, and show you how to use it with your SQL-based database clusters. We will also discuss HA strategies for HAProxy with Keepalived and Virtual IP.

 


Big Data Integration & ETL - Moving Live Clickstream Data from MongoDB to Hadoop for Analytics

 
June 16, 2014
By Severalnines

MongoDB is great at storing clickstream data, but using it to analyze millions of documents can be challenging. Hadoop provides a way of processing and analyzing data at large scale. Since it is a parallel system, workloads can be split on multiple nodes and computations on large datasets can be done in relatively short timeframes. MongoDB data can be moved into Hadoop using ETL tools like Talend or Pentaho Data Integration (Kettle).

 

In this blog, we’ll show you how to integrate your MongoDB and Hadoop datastores using Talend. We have a MongoDB database collecting clickstream data from several websites. We’ll create a job in Talend to extract the documents from MongoDB, transform and then load them into HDFS. We will also show you how to schedule this job to be executed every 5 minutes.

 

Test Case

 

We have an application collecting clickstream data from several websites. Incoming data is mostly inserts generated from user actions against HTML Document Object Model (DOM) and stored in a MongoDB collection called domstream. We are going to bulk load our data in batch from the MongoDB collection into Hadoop (as an HDFS output file). Hadoop can then be used as a data warehouse archive on which we can perform our analytics. 

 

For step by step instructions on how to set up your Hadoop cluster, please read this blog post. Our architecture can be illustrated as below:

 

Our goal is to bulk load the MongoDB data to an HDFS output file every 5 minutes. The steps are:

  1. Install Talend
  2. Design the job and workflow
  3. Test
  4. Build the job
  5. Transfer the job to MongoDB server (ETL server)
  6. Schedule it to run in production via cron

 

Install Talend Open Studio

 

We’ll be using Talend Open Studio for Big Data as our ETL tool. Download and install the application on your local workstation. We'll use it to design and deploy the process workflow for our data integration project.

 

Extract the downloaded package and open the application. Accept the license and create a new project called Mongo2Hadoop. Choose the corresponding project and click Open. You can skip the TalendForge sign-in page and directly access the Talend Open Studio dashboard. Click on Job under Create a new section and give the job a name. We are going to use the same name with project name.

 


Join Us at the European MariaDB Roadshow this Summer!

 
June 12, 2014
By Severalnines

We’re delighted to be joining Team MariaDB at SkySQL with a talk on ‘Automation & Management of Database Clusters’ as they tour across several European regions in the coming weeks.

 

Whether you’re a MariaDB enthusiast or newbie, a MySQL veteran or newbie, or even a MongoDB user who also happens to run MySQL or MariaDB databases: this roadshow is a good place to find out about the latest developments on the MariaDB database as well as related tools and technologies. 

 

From our own perspective, we’ll be updating participants on how to best automate & manage their database clusters, and demo some of the capabilities of ClusterControl.

 

The theme of this year’s MariaDB roadshow is ‘Reaching New Heights - Proven Features and Cool New Technologies’. 

 

You will hear about the latest MariaDB innovations including how the MariaDB 10 GA release significantly increases the performance of MariaDB. You will also learn how the new MaxScale - an open source proxy platform for MariaDB and MySQL - provides you with a new tool for even better scalability.


 

There are six dates/locations to chose from: 

 


How to Deploy Percona XtraDB Cluster without Internet Connection

 
May 23, 2014
By Severalnines

If you need to deploy a database cluster behind your corporate firewall, on servers with no internet access, then this blog is for you. Most of the installation tutorials out there require servers to have an internet connection, and that would not work very well with companies under strict policies and compliance guidelines. 


Our online Galera Configurator supports deployment without Internet connection for Percona XtraDB Cluster. You would need to prepare a minimum of four hosts (three DB nodes and one management node), generate a Deployment Package from the Configurator and copy it to the ClusterControl host. In this post, we will show you how to deploy Percona XtraDB Cluster 5.5 in an environment without internet connection on CentOS 6.5 (installed with minimal installation). Our setup is illustrated below:

 

Prerequisites

 

Prior to performing the offline installation:

  • You need to have a full image of the OS distribution ready on DVD (CentOS-6.5-x86_64-bin-DVD1.iso is sufficient)
  • You may need to have an external DVD drive to mount the ISO on the ClusterControl host.
  • All hosts are able to connect to each other via LAN.

 

Set Up Local Repository

 

Mount the CentOS installation DVD using the external DVD drive on the ClusterControl host. On this host, we will configure Apache web server to host the local repository via HTTP to be accessed by the database hosts.

 


Webinar Replay, Slides & Q&A: Introducing ClusterControl 1.2.6 - Managing your MySQL, MariaDB & MongoDB Clusters

 
May 19, 2014
By Severalnines

 

Thanks to everyone who attended and participated last week’s joint webinar on ClusterControl 1.2.6! We had great questions from participants (thank you), most of which are transcribed below with our answers to them.

 

If you missed the sessions or would like to watch the webinar again & browse through the slides, they are now available online.

 

Webinar topics discussed: 

  • Database Infrastructure Lifecycle
  • Deploy, Monitor, Manage, Scale
  • MySQL, MariaDB & MongoDB Clusters
  • ClusterControl Overview & Demo
  • ClusterControl New Features in 1.2.6 & Demo
  • Centralized Authentication using LDAP or Active Directory
  • Role-Based Access Control
  • OpenStack: Galera Deployment Automation
  • Hybrid setups with Galera and Asynchronous MySQL Replication
  • Manage single instance MySQL Servers or existing MySQL Replication setups

 

Watch the replay

 

Read the slides

 

Read the Questions & Answers:

 


Archival and Analytics - Importing MySQL data into Hadoop Cluster using Sqoop

 
May 16, 2014
By Severalnines

We won’t bore you with buzzwords like volume, velocity and variety. This post is for MySQL users who want to get their hands dirty with Hadoop, so roll up your sleeves and prepare for work. Why would you ever want to move MySQL data into Hadoop? One good reason is archival and analytics. You might not want to delete old data, but rather move it into Hadoop and make it available for further analysis at a later stage. 

 

In this post, we are going to deploy a Hadoop Cluster and export data in bulk from a Galera Cluster using Apache Sqoop. Sqoop is a well-proven approach for bulk data loading from a relational database into Hadoop File System. There is also Hadoop Applier available from MySQL labs, which works by retrieving INSERT queries from MySQL master binlog and writing them into a file in HDFS in real-time (yes, it applies INSERTs only).

 

We will use Apache Ambari to deploy Hadoop (HDP 2.1) on three servers. We have a clustered Wordpress site running on Galera, and for the purpose of this blog, we will export some user data to Hadoop for archiving purposes. The database name is wordpress, we will use Sqoop to import the data to a Hive table running on HDFS. The following diagram illustrates our setup:

The ClusterControl node has been installed with an HAproxy instance to load balance Galera connections and listen on port 33306.

 

Prerequisites

 

All hosts are running CentOS 6.5 with firewall and SElinux turned off. All servers’ time are using NTP server and synced with each other. Hostname must be FQDN or define your hosts across all nodes in /etc/hosts file. Each host has been configured with the following host definitions:

192.168.0.100		clustercontrol haproxy mysql
192.168.0.101		mysql1 galera1
192.168.0.102		mysql2 galera2
192.168.0.103		mysql3 galera3
192.168.0.111		hadoop1 hadoop1.cluster.com
192.168.0.112		hadoop2 hadoop2.cluster.com
192.168.0.113		hadoop3 hadoop3.cluster.com

 

Create an SSH key and configure passwordless SSH on hadoop1 to other Hadoop nodes to automate the deployment by Ambari Server. In hadoop1, run following commands as root:

$ ssh-keygen -t rsa # press Enter for all prompts
$ ssh-copy-id -i ~/.ssh/id_rsa hadoop1.cluster.com
$ ssh-copy-id -i ~/.ssh/id_rsa hadoop2.cluster.com
$ ssh-copy-id -i ~/.ssh/id_rsa hadoop3.cluster.com

 


New Release Webinar on May 13th: Introducing ClusterControl 1.2.6 - Live Demo

 
May 7, 2014
By Severalnines

 

Following the release of ClusterControl 1.2.6 a couple of weeks ago, we are now looking forward to demonstrating this latest version of the product on Tuesday next week, May 13th.

 

This release contains key new features (along with performance improvements and bug fixes), which we will be demonstrating live during the webinar. 

 

Highlights include:

  • Centralized Authentication using LDAP or Active Directory
  • Role-Based Access Control
  • OpenStack: Galera Deployment Automation
  • Hybrid setups with Galera and Asynchronous MySQL Replication
  • Manage single instance MySQL Servers or existing MySQL Replication setups

 

New Features Webinar on ClusterControl 1.2.6 - May 13th 2014

 

DATE & TIME

Europe/MEA/APAC

Tuesday, May 13th at 09:00 BST (UK) / 10:00 CEST (Germany, France, Sweden)

Register Now

 

North America/LatAm

Tuesday, May 13th at 9:00 Pacific Time (US) / 12:00 Eastern Time (US)

Register Now

 

 

SPEAKER

Vinay Joosery, CEO, Severalnines

 


How to Setup Centralized Authentication of ClusterControl Users with LDAP

 
April 24, 2014
By Severalnines

ClusterControl 1.2.6 introduces integration with Active Directory and LDAP authentication. This allows users to log into ClusterControl by using their corporate credentials instead of a separate password. LDAP groups can be mapped onto ClusterControl user groups to apply roles to the entire group, so it is very convenient for larger organizations who have a centralized LDAP-compliant authentication system. This blog shows you how to configure LDAP authentication in ClusterControl, and allow users to use their Active Directory or LDAP username and password to log in to ClusterControl. 

 

LDAP authentication can be configured from ClusterControl, in the Admin dashboard (ClusterControl > Admin > LDAP Settings). If you are running ClusterControl v1.2.5 or older, please upgrade to v1.2.6. It requires PHP LDAP package to be installed and enabled. To install, you can just simply use following command:


For Redhat-based:

$ yum install php-ldap


For Debian-based:

$ apt-get install php5-ldap


You may need to restart Apache web server to ensure that PHP LDAP is loaded properly. If you are upgrading your ClusterControl installation using the s9s_upgrade_cmon script, PHP LDAP will be installed automatically.

 

Organization Structure

 

The following is a simple organization structure running on OpenLDAP:

dc=severalnines,dc=org (2)

  • ou=group (4)
    • cn=Database Administrator
    • cn=Manager
    • cn=System Administrator
    • cn=Programmer
  • ou=people (4)
    • cn=Eric Steven
    • cn=Matthew Carey
    • cn=Michael Joshua
    • cn=Ronnie Harp



Search
Keep in touch
Signup for our newsletter:
Name:
*Email:

Alex's Blog

Quickstart $ git clone https://github.com/alyu/docker.git $ cd docker/percona-xtradb-5.6/centos $ ./build.sh $ ./start-servers.sh 3 $ ./bootstrap-cluster.sh $ ssh root@172.17.0.2 (root123) $ mysql -...
21 weeks 2 days ago

Johan's Blog

FOREIGN KEYs in MySQL Cluster is a big step forward. It is now possible to run enterprise software with NDB Cluster as the storage backend. Over the years, the lack of FOREIGN KEYs have been one of...
1 year 5 weeks ago