Tutorial: Creating and Using a Cluster Configuration

Tutorial: Creating and Using a Cluster Configuration

A short walk-through that demonstrates how to use gfsh to create a cluster configuration for a Pivotal GemFire cluster.

Note: This tutorial requires that you install and configure Pivotal GemFire for your environment. See Installing Pivotal GemFire. This tutorial can be performed on a single computer.

The gfsh command-line tool allows you to configure and start a Pivotal GemFire cluster. The cluster configuration service uses Pivotal GemFire locators to store the configuration at the group and cluster levels and serves these configurations to new members as they are started. The locators store the configurations in a hidden region that is available to all locators and also write the configuration data to disk as XML files. Configuration data is updated as gfsh commands are executed.

This section provides a walk-through example of configuring a simple Pivotal GemFire cluster and then re-using that configuration in a new context.

  1. Create a working directory (For example:/home/username/my_gemfire) and switch to the new directory. This directory will contain the configurations for your cluster.
  2. Start the gfsh command-line tool. For example: 
    $ gfsh
    The gfsh command prompt displays.
        _________________________     __
   / _____/ ______/ ______/ /____/ /
  / /  __/ /___  /_____  / _____  / 
 / /__/ / ____/  _____/ / /    / /  
/______/_/      /______/_/    /_/    v8.1.0

Monitor and Manage GemFire
gfsh>
  3. Start a locator using the command in the following example: 
    gfsh>start locator --name=locator1
Starting a GemFire Locator in /Users/username/my_gemfire/locator1...
.............................
Locator in /Users/username/my_gemfire/locator1 on 10.0.0.13[10334] as locator1 is currently online.
Process ID: 5203
Uptime: 15 seconds
GemFire Version: 8.1.0
Java Version: 1.7.0_71
Log File: /Users/username/my_gemfire/locator1/locator1.log
JVM Arguments: -Dgemfire.enable-cluster-configuration=true 
-Dgemfire.load-cluster-configuration-from-dir=false 
-Dgemfire.launcher.registerSignalHandlers=true -Djava.awt.headless=true 
-Dsun.rmi.dgc.server.gcInterval=9223372036854775806
Class-Path: /Users/username/Pivotal_GemFire_810_b50582_Linux/lib/gemfire.jar
:/Users/username/Pivotal_GemFire_810_b50582_Linux/lib/locator-dependencies.jar

Successfully connected to: [host=10.0.0.13, port=1099]

Cluster configuration service is up and running.
    Note that gfsh responds with a message indicating that the cluster configuration service is up and running. If you see a message indicating a problem, review the locator log file for possible errors. The path to the log file is displayed in the output from gfsh.
  4. Start Pivotal GemFire servers using the commands in the following example: 
    gfsh>start server --name=server1 --group=group1
Starting a GemFire Server in /Users/username/my_gemfire/server1...
.....
Server in /Users/username/my_gemfire/server1 on 10.0.0.13[40404] as server1 is currently online.
Process ID: 5627
Uptime: 2 seconds
GemFire Version: 8.1.0
Java Version: 1.7.0_71
Log File: /Users/username/my_gemfire/server1/server1.log
JVM Arguments: -Dgemfire.default.locators=10.0.0.13[10334] -Dgemfire.groups=group1 
-Dgemfire.use-cluster-configuration=true -XX:OnOutOfMemoryError=kill -KILL %p 
-Dgemfire.launcher.registerSignalHandlers=true -Djava.awt.headless=true 
-Dsun.rmi.dgc.server.gcInterval=9223372036854775806
Class-Path: /Users/username/Pivotal_GemFire_810_b50582_Linux/lib/gemfire.jar
:/Users/username/Pivotal_GemFire_810_b50582_Linux/lib/server-dependencies.jar

gfsh>start server --name=server2 --group=group1 --server-port=40405
Starting a GemFire Server in /Users/username/my_gemfire/server2...
.....
Server in /Users/username/my_gemfire/server2 on 10.0.0.13[40405] as server2 is currently online.
Process ID: 5634
Uptime: 2 seconds
GemFire Version: 8.1.0
Java Version: 1.7.0_71
Log File: /Users/username/my_gemfire/server2/server2.log
JVM Arguments: -Dgemfire.default.locators=10.0.0.13[10334] -Dgemfire.groups=group1 
-Dgemfire.use-cluster-configuration=true -XX:OnOutOfMemoryError=kill -KILL %p 
-Dgemfire.launcher.registerSignalHandlers=true -Djava.awt.headless=true 
-Dsun.rmi.dgc.server.gcInterval=9223372036854775806
Class-Path: /Users/username/Pivotal_GemFire_810_b50582_Linux/lib/gemfire.jar
:/Users/username/Pivotal_GemFire_810_b50582_Linux/lib/server-dependencies.jar

gfsh>start server --name=server3 --server-port=40406
Starting a GemFire Server in /Users/username/my_gemfire/server3...
.....
Server in /Users/username/my_gemfire/server3 on 10.0.0.13[40406] as server3 is currently online.
Process ID: 5637
Uptime: 2 seconds
GemFire Version: 8.1.0
Java Version: 1.7.0_71
Log File: /Users/username/my_gemfire/server3/server3.log
JVM Arguments: -Dgemfire.default.locators=10.0.0.13[10334] 
-Dgemfire.use-cluster-configuration=true -XX:OnOutOfMemoryError=kill -KILL %p 
-Dgemfire.launcher.registerSignalHandlers=true -Djava.awt.headless=true 
-Dsun.rmi.dgc.server.gcInterval=9223372036854775806
Class-Path: /Users/username/Pivotal_GemFire_810_b50582_Linux/lib/gemfire.jar
:/Users/username/Pivotal_GemFire_810_b50582_Linux/lib/server-dependencies.jar
    Note that the gfsh commands you used to start server1 and server2 specify a group named group1 while the command for server3 did not specify a group name.
  5. Create some regions using the commands in the following example: 
    gfsh>create region --name=region1 --group=group1 --type=REPLICATE
Member  | Status
------- | --------------------------------------
server2 | Region "/region1" created on "server2"
server1 | Region "/region1" created on "server1"

gfsh>create region --name=region2 --type=REPLICATE
Member  | Status
------- | --------------------------------------
server1 | Region "/region2" created on "server1"
server2 | Region "/region2" created on "server2"
server3 | Region "/region2" created on "server3"
    Note that region1 is created on all cache servers that specified the group named group1 when starting the cache server (server1 and server2, in this example). region2 is created on all members because no group was specified.
  6. Deploy jar files. Use the gfsh deploy command to deploy application jar files to all members or to a specified group of members. The following example deploys the mail.jar and mx4j.jar files from the Pivotal GemFire distribution. (Note: This is only an example, you do not need to deploy these files to use the Cluster Configuration Service. Alternately, you can use any two jar files for this demonstration.) 
    gfsh>deploy --group=group1 --jar=${SYS_GEMFIRE_DIR}/lib/mail.jar
Post substitution: deploy --group=group1 --jar=/Users/username/Pivotal_GemFire_810_b50582_Linux/lib/mail.jar
Member  | Deployed JAR | Deployed JAR Location
------- | ------------ | -------------------------------------------------
server1 | mail.jar     | /Users/username/my_gemfire/server1/vf.gf#mail.jar#1
server2 | mail.jar     | /Users/username/my_gemfire/server2/vf.gf#mail.jar#1

gfsh>deploy --jar=${SYS_GEMFIRE_DIR}/lib/mx4j.jar
Post substitution: deploy --jar=/Users/username/Pivotal_GemFire_810_b50582_Linux/lib/mx4j.jar
Member  | Deployed JAR | Deployed JAR Location
------- | ------------ | -------------------------------------------------
server1 | mx4j.jar     | /Users/username/my_gemfire/server1/vf.gf#mx4j.jar#1
server2 | mx4j.jar     | /Users/username/my_gemfire/server2/vf.gf#mx4j.jar#1
server3 | mx4j.jar     | /Users/username/my_gemfire/server3/vf.gf#mx4j.jar#1
    Note that the mail.jar file was deployed only to the members of group1 and the mx4j.jar was deployed to all members.
  7. Export the cluster configuration.
    You can use the gfsh export cluster-configuration command to create a zip file that contains the cluster's persisted configuration. The zip file contains a copy of the contents of the cluster_config directory. For example:
    gfsh>export cluster-configuration --zip-file-name=myClusterConfig.zip --dir=/Users/username
    Pivotal GemFire writes the cluster configuration to the specified zip file. 
    Downloading cluster configuration : /Users/username/myClusterConfig.zip

    The remaining steps demonstrate how to use the cluster configuration you just created.

  8. Shutdown the cluster using the following commands: 
    gfsh>shutdown --include-locators=true
As a lot of data in memory will be lost, including possibly events in queues, do you 
really want to shutdown the entire distributed system? (Y/n): Y
Shutdown is triggered

gfsh>
No longer connected to 10.0.0.13[1099].
gfsh>
  9. Exit the gfsh command shell: 
    gfsh>quit
Exiting...
  10. Create a new working directory (for example: new_gemfire) and switch to the new directory.
  11. Start the gfsh command shell: 
    $ gfsh
  12. Start a new locator. For example: 
    gfsh>start locator --name=locator2 --port=10335
Starting a GemFire Locator in /Users/username/new_gemfire/locator2...
.............................
Locator in /Users/username/new_gemfire/locator2 on 10.0.0.13[10335] as locator2 is currently online.
Process ID: 5749
Uptime: 15 seconds
GemFire Version: 8.1.0
Java Version: 1.7.0_71
Log File: /Users/username/new_gemfire/locator2/locator2.log
JVM Arguments: -Dgemfire.enable-cluster-configuration=true 
-Dgemfire.load-cluster-configuration-from-dir=false 
-Dgemfire.launcher.registerSignalHandlers=true -Djava.awt.headless=true 
-Dsun.rmi.dgc.server.gcInterval=9223372036854775806
Class-Path: /Users/username/Pivotal_GemFire_810_b50582_Linux/lib/gemfire.jar
:/Users/username/Pivotal_GemFire_810_b50582_Linux/lib/locator-dependencies.jar

Successfully connected to: [host=10.0.0.13, port=1099]

Cluster configuration service is up and running.
  13. Import the cluster configuration using the import cluster-configuration command. For example: 
    gfsh>import cluster-configuration --zip-file-name=/Users/username/myClusterConfig.zip
Cluster configuration successfully imported
    Note that the locator2 directory now contains a cluster_config subdirectory.
  14. Start a server that does not reference a group: 
    gfsh>start server --name=server4 --server-port=40414
Starting a GemFire Server in /Users/username/new_gemfire/server4...
........
Server in /Users/username/new_gemfire/server4 on 10.0.0.13[40414] as server4 is currently online.
Process ID: 5813
Uptime: 4 seconds
GemFire Version: 8.1.0
Java Version: 1.7.0_71
Log File: /Users/username/new_gemfire/server4/server4.log
JVM Arguments: -Dgemfire.default.locators=10.0.0.13[10335]
-Dgemfire.use-cluster-configuration=true -XX:OnOutOfMemoryError=kill -KILL %p
-Dgemfire.launcher.registerSignalHandlers=true -Djava.awt.headless=true
-Dsun.rmi.dgc.server.gcInterval=9223372036854775806
Class-Path: /Users/username/Pivotal_GemFire_810_b50582_Linux/lib/gemfire.jar
:/Users/username/Pivotal_GemFire_810_b50582_Linux/lib/server-dependencies.jar
  15. Start another server that references group1: 
    gfsh>start server --name=server5 --group=group1 --server-port=40415
Starting a GemFire Server in /Users/username/new_gemfire/server5...
.....
Server in /Users/username/new_gemfire/server2 on 10.0.0.13[40415] as server5 is currently online.
Process ID: 5954
Uptime: 2 seconds
GemFire Version: 8.1.0
Java Version: 1.7.0_71
Log File: /Users/username/new_gemfire/server5/server5.log
JVM Arguments: -Dgemfire.default.locators=10.0.0.13[10335] -Dgemfire.groups=group1 
-Dgemfire.use-cluster-configuration=true -XX:OnOutOfMemoryError=kill -KILL %p 
-Dgemfire.launcher.registerSignalHandlers=true -Djava.awt.headless=true 
-Dsun.rmi.dgc.server.gcInterval=9223372036854775806
Class-Path: /Users/username/Pivotal_GemFire_810_b50582_Linux/lib/gemfire.jar
:/Users/username/Pivotal_GemFire_810_b50582_Linux/lib/server-dependencies.jar
  16. Use the list regions command to display the configured regions. Note that region1 and region2, which were configured in the original cluster level are available. 
    gfsh>list regions
List of regions
---------------
region1
region2
  17. Use the describe region command to see which members host each region. Note that region1 is only hosted by server5 because server5 was started using the group1 configuration. region2 is hosted on both server4 and server5 because region2 was created without a group specified. 
    gfsh>describe region --name=region1
..........................................................
Name            : region1
Data Policy     : replicate
Hosting Members : server5

Non-Default Attributes Shared By Hosting Members  

 Type  | Name | Value
------ | ---- | -----
Region | size | 0


gfsh>describe region --name=region2
..........................................................
Name            : region2
Data Policy     : replicate
Hosting Members : server5
                  server4

Non-Default Attributes Shared By Hosting Members  

 Type  | Name | Value
------ | ---- | -----
Region | size | 0
    This new cluster uses the same configuration as the original system. You can start any number of servers using this cluster configuration. All servers will receive the cluster-level configuration. Servers that specify group1 also receive the group1 configuration.
  18. Shutdown your cluster using the following commands: 
    gfsh>shutdown --include-locators=true
As a lot of data in memory will be lost, including possibly events in queues,
  do you really want to shutdown the entire distributed system? (Y/n): Y
Shutdown is triggered

gfsh>
No longer connected to 10.0.0.13[1099].
Parent topic: Configuring and Running a Cluster