Deploying ViPR Services Docker Container on AWS

Update 7/2/15: EMC Software Defined Solutions team launched ECS 2.0 at DockerCon and announced the availability of a free community supported version of ECS available at emc.com/getECS.  This essentially replaced the project I refer to in this post.  Head over to my post that walks you through deploying ECS 2.0 in AWS or directly on CentOS.

EMC is all-in on Open Source, code sharing, and APIs to cater to developers. The @EMCCode team has many fantastic projects in the works.  Go check it out at http://emccode.github.io.  Some of them like this project I am writing about will be public very soon but for now it is available internal to EMC only.  Consequently, some details are not available here until the final public release.  This post walks you through everything to get you setup with an instance running Docker in AWS and then deploying the <secret_codename> Docker container that functions as a standalone ViPR Services instance.  You can use this to test S3, Swift, and Atmos object functionality.  Note that some ViPR Services capabilities are not available in this standalone instance.

If you have not played with Docker, this is a good introduction and will get you using your first container.

Deploy Docker in AWS

  1. Login to AWS Instance Wizard
  2. Select the Amazon Linux AMI or your preferred distribution.
  3. Choose the instance type.  You need 12-16GB of RAM for the instance.  Go with the  r3.large with 2 CPUs and 15GB of RAM.
  4. Go with the defaults on the Configure Instance Details page unless you have specific networking requirements.
  5. Review and click Launch. We will configure the security group later.
  6. Create a new key pair or use an existing one.  This is what you will use to connect to the new instance.  NOTE: I recommend setting up billing alerts at this point to prevent you from getting a surprise bill the size of your teenage daughter’s texting usage charges.
  7. Launch the instance.
  8. Browse to your instance(s), select the one you created, and click connect for instructions for connecting.
  9. SSH into your instance
    ssh -i key.pem ec2-user@<ip_address>
  10. Install Docker
    sudo yum install -y docker ; sudo service docker start

Deploy <Secret_Codename>

Now that Docker is deployed in your AWS linux instance you are ready to deploy the <secret_codename> Docker container.

  1. Download the container
    sudo docker pull emccode/<secret_codename>
  2. Next, run the container and bind the ports from the container to the host using “-p”
    docker run -tid -p 10101:10101 -p 10301:10301 -p 10501:10501 emccode/<secret_codename>:latest
  3. The container ID is displayed.  Replace <id> with the value.  This connects to the persistent bash session.  Use <ctrl-p> and <ctrl-q> to leave this session but keep the container running.
     docker attach <id>
  4. You just unleashed the ViPR.
  5. Open the above ports.
    1. Browse to your AWS instance and select the ViPR instance.
    2. Click on the Security Group name below to view the settings.
    3. Click Edit.
    4. Click Add Rule and then specify the ports we used earlier: 10101, 10301, 10501.  Configure the Source to a specific IP, range, or Anywhere.
    5. Save
  6. Get your secret key
    cat /StandaloneDeploymentOutput

Client Access

The below s3cmd examples are copied from https://github.com/emccode/<secret_codename>/blob/master/clients.md

  1. Install s3cmd
  2. Configure s3cmd
    • s3cmd --configure
  3. Configure s3cmd to use:
    • Access key: wuser1@sanity.local
    • Secrect key: YourSecretKeyHere
    • Encryption password:
    • Path to GPG program:
    • Use HTTPS protocol: no (in simulator version)
  4. Add HTTP Proxy:
    • HTTP Proxy server name: <ip_of_vipr>
    • HTTP Proxy server port: 10101

List buckets

  • s3cmd ls

Create bucket

  • s3cmd mb s3://new-bucket

Copy file to bucket

  • s3cmd put README.md s3://new-bucket

Copy file from bucket

  • s3cmd get s3://new-bucket/README.md test.md

Command help

  • s3cmd --help