Creating an Aerospike Vagrant Cluster for Development

logo-aerospike

After a lot of looking around and what not trying to set up a local Vagrant Aerospike Cluster for testing, I decided to just crack on and do it myself – so I thought I’d share the methodology here in case anyone else is looking for the same set up…

All this was done on OS X (El Capitan) – I’m pretty sure most of it would transfer to another OS as there’s nothing Mac specific involved but as I haven’t tested it, you may need to tweak a few things.


 

NB: This post assumes you are familiar with Vagrant already and how to create and set up boxes and that you’ve used and done all this stuff before – it’s not within the scope of this post to show you how Vagrant works and it’s set up for first time etc – there’s loads of stuff on the ‘net which will help you get set up

 

Firstly I’d recommend you get and install Vagrant Manager – it’s an OS level app that monitors your vagrant boxes – great for bringing them up and down and really handy when working with groups etc as you’ll see later.

Next we need to create a Vagrant config file to set up our Aerospike boxes. This config will set up 3 identical boxes which can be brought up together, or one at a time etc to mimic a cluster and allow you to perform failures, failovers etc and see Aerospike’s amazing replication system working.

Navigate to the folder you want your vagrant boxes in and create a file named 'Vagrantfile'. You can then edit this file ($ vi Vagrantfile) and write in the following:

Vagrant.configure(2) do |config|
  #### DEFINE BOX 1 ####
  config.vm.define "aerospike_vm_1" do |aerospike_vm_1|
    aerospike_vm_1.vm.box = "aerospike/centos-6.5"
    aerospike_vm_1.vm.provider :virtualbox do |vb|
      vb.name = "aerospike_vm_1"
    end

  # Network settings for this box:
  aerospike_vm_1.vm.network "private_network", ip: "33.33.33.91"
  end
  #### END OF BOX 1 ####

  #### DEFINE BOX 2 ####
  config.vm.define "aerospike_vm_2" do |aerospike_vm_2|
    aerospike_vm_2.vm.box = "aerospike/centos-6.5"
    aerospike_vm_2.vm.provider :virtualbox do |vb|
      vb.name = "aerospike_vm_2"
    end

  # Network settings for this box:
  aerospike_vm_2.vm.network "private_network", ip: "33.33.33.92"
  end
  #### END OF BOX 2 ####

  #### DEFINE BOX 3 ####
  config.vm.define "aerospike_vm_3" do |aerospike_vm_3|
    aerospike_vm_3.vm.box = "aerospike/centos-6.5"
    aerospike_vm_3.vm.provider :virtualbox do |vb|
      vb.name = "aerospike_vm_3"
    end

  # Network settings for this box:
  aerospike_vm_3.vm.network "private_network", ip: "33.33.33.93"
  end
  #### END OF BOX 3 ####
end

This will create basically 3 identical boxes with individual network settings and allow you to set up Aerospike servers on each one (it’ll load as default as the base box is the Aerospike one – which will also install AMC).

From there you should now be able to bring up the boxes by issuing the following command in the same folder:

To bring up all 3 boxes at once:

$ vagrant up

To bring up just one box:

$ vagrant up aerospike_vm_1 (or 2 / 3 etc)

Once you have brought them up (individually or all at once) if you refresh your Vagrant Manager, you should see the number of VM boxes currently running:

Screen Shot 2016-02-18 at 12.39.57

You’ll also see that you can now control these boxes from Vagrant Manager too…

Screen Shot 2016-02-18 at 12.40.54

From here you now need to set up your configs for Aerospike to create your cluster and get them talking to each other. If you SSH into one of the servers (vm_1 for example) and we can set the config for the first server:

$ vagrant ssh aerospike_vm_1

You can also use Vagrant Manager to SSH in, either way is just as effective (VM opens a new terminal window for you). You should see a load of information about settings and the AMC (Aerospike Monitoring Console) etc and how to get a free tee shirt! but in the end you should see the following:

*******************************************************************
**********Aerospike Database Server is running!
**********Aerospike Monitoring Console is running!

If you don’t see the above, check for any errors and if there are no error messages then simply run the following commands:

$ sudo service aerospike start
$ sudo service amc start

This should bring up both the Aerospike service and the AMC console. You can check your server is up and running by navigating to http://localhost:8081 which should bring up the AMC console – type localhost in the box that appears and you should be connected to your local running Aerospike instance:

Screen Shot 2016-02-18 at 12.51.01

Screen Shot 2016-02-18 at 12.51.16

If something has gone wrong during this process you’ll need to try and trouble shoot why Aerospike is not running on the Vagrant box you have provisioned and if necessary Google any issues you come across (unfortunately I can’t give resolutions for every possible issue which may occur – but there shouldn’t really be any)

If all is well so far, then you need to edit the config for Aerospike. Navigate to the config directory ($ cd /etc/aerospike) and check the config files in there – you should have either 2 or 3 config files… named something like aerospike.conf / aerospike_mesh.conf or aerospike_ssd.conf. The best thing to do is if you have an aerospike.conf file, move this to create a backup and then create a blank config file which you will use by issuing the following commands. If you don’t have an aerospike.conf file, skip the first bit:

$ sudo mv aerospike.conf aerospike.conf.bak
$ sudo vi aerospike.conf

Once you have your blank file, you can create your Aerospike config file (note, this is a default and quick running config file for testing – feel free to read the Aerospike docs on customising the config file elements and change to suit your needs):

# Aerospike database configuration file.
# This stanza must come first.
service {
    user root
    group root
    paxos-single-replica-limit 1
    pidfile /var/run/aerospike/asd.pid
    service-threads 4
    transaction-queues 4
    transaction-threads-per-queue 4
    proto-fd-max 15000
}
# Disable caching on our LUA Modules for development
mod-lua {
    cache-enabled false
}
logging {
    # Log file must be an absolute path.
    file /var/log/aerospike/aerospike.log {
        context any info
    }
}
network {
    service {
        address any
        port 3000
        access-address 33.33.33.91 # Set to 92 for vm_2 and 93 for vm_3 etc
        network-interface-name eth2 # Needed for Node ID
    }
    heartbeat {
        mode multicast
        address 239.1.99.2
        port 9918
        interface-address 33.33.33.91 # Set to 92 for vm_2 and 93 for vm_3 etc
        interval 150
        timeout 10
    }
    fabric {
        port 3001
    }
    info {
        port 3003
    }
}
# Set up our Namespace for testing / development (like a table in SQL DB)
namespace test {
    replication-factor 2
    memory-size 1G
    default-ttl 0 # 0 = no TTL (lasts forever)
    storage-engine device {
        file /opt/aerospike/data/test.dat # Path to your data file
        filesize 2G
        data-in-memory false
    }
}

 

Once you have edited this file, you can save it – as mentioned feel free to customise any of the values such as file size, memory, replication etc etc. The only important things to note are the interface-address and access-address just making sure the addresses match the server they are running on.

To test your changes, make sure you are only running the aerospike_vm you have edited the config on and then simply restart the service by running the following:

$ sudo service aerospike restart

You should see the updated changes in your AMC by refreshing the window (or waiting a few seconds). If not, or the service failed to start – check any error messages in the console or the log to ensure that your config file is valid.

You then need to update the same config files on each of your other aerospike vagrant boxes – ensuring that you set the IP’s for interface-address and access-address to match the ones for the server (so for aerospike_vm_2 that would be 33.33.33.92 etc).

Screen Shot 2016-02-18 at 13.19.27

That’s pretty much it – you now have a fully working local cluster of Aerospike servers which you can access from your DEV env and bring up / down as you see fit to see how replication works etc. Aerospike really is an amazing piece of kit and you should look at writing UDF files in LUA to really unlock it’s potential (blog post on that coming soon).

 

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s