Driving Business Results with Big DataA personal journey into the world of big data in the cloud – part 1

1 INTRODUCTION

As an SAP HCM solution architect and general technophile I have a rapidly growing interest in the possibilities that HANA, Big Data and cloud based solutions can create for organisations. In order to grow my own skillset, with a view to becoming much more engaged professionally in Big Data and cloud technologies I’ve spent some downtime recently, continuing own journey into this brave new world.

I’ve had an interest in the potential of AWS solutions for some time as being both a cost saver to organisations (and for my own private ventures) and for their potential to provide enormous, temporary, demand driven computational power. Going through the catalogue of MOOC’s available on openSAP the course titled Driving Business Results with Big Data naturally sparked my interest both for its content and hands on approach.

As part of this course, participants are invited to try out a number of exercises in a real system that they provision for themselves via Amazon Web Services and SAP CAL, the cloud appliance library. Some personal effort and of course personal cost is involved here, which brings me to the purpose of my writing. Going through the presentations and reviewing the comments and posts in the course discussions I struggled to find detailed information on expected costs that individuals would incur, so I decided to just dive in, try out the exercises and openly post the costs incurred. I also felt that it might be interesting to see how easy it is to navigate my way through setting up the required systems and services, documenting my steps along the way as I complete the exercises.

I’ll be writing about my experiences completing the openSAP course in a short series of posts and I hope you’ll join me on my journey. Please feel free to comment and add your own experiences and if you’d like to get in contact with me directly, please feel free to connect with me via LinkedIn, I’ll be happy to hear from you.

© Jason Greene 2015

2 GETTING STARTED

At the beginning of week 3, the course material directs participants to some links with information on how to get started:

System Access – General Documentation

This is the most important link initially.

The second link, providing access to CAL is a direct link to the cloud appliance library for setup of the required systems within AWS.

Being a little nervous of the procedure, this being my first time setting up on AWS and using CAL, I was glad I decided to take a look at the CAL link before committing anything on AWS.

The reason for this becomes clear when you see the screen to the right.

I understood from the information on SCN that the default region is us-east-1 but the dropdown only contains us-east-1 and cannot be overwritten.

© Jason Greene 2015

Being aware that SAP content doesn’t always work brilliantly on IE, I tried again using Firefox just to be sure it wasn’t a browser issue but this didn’t help. For the majority of people, i.e. everyone in the US and Europe, this probably isn’t a big deal but if you’re not in the US and already have AWS free tier services you might not be too happy. AWS does at least make it easy to operate services in different regions. The documentation on SCN does describe how to request availability in other regions but unfortunately the instructions only seem to be relevant to paying customers. A shame but let’s continue and see where we get to.

2.1 CREATING AN AWS ACCOUNTFrom this point forwards, I’ll be including some additional information in the form of a small boxed area as shown below:

Start End Duration Accrued cost

11:00 12:00 1:00 $ 2.00

For each step, I’ll show the time I started and finished, the actual duration to complete the step and the cost I incurred. At the very end I’ll wrap up with the total time and cost. It looks like I’ll incur costs in US dollars although I’ll obviously be actually billed in AUD on my credit card. If I’m able to report costs in AUD then I’ll make that clear.

So, ignoring the CAL information regarding regions for now, since I definitely want to use the current AWS free tier services once I’m finished with this set of exercises, I’m going to start by creating a free tier account. The documentation on SCN indicates that it’s possible to (and recommended) create a separate account for CAL:

I’m back to using IE at this point (please don’t shoot me) as I just find it suits me better.

In case you were wondering, you can set up your AWS account here.

© Jason Greene 2015

Very conveniently, AWS allows me to sign up using my existing Amazon account. Entry of my contact details and verification of my credit card is required but the whole process is very slick. I had to smile when the signup procedure lead to an automated call to my phone within a couple of minutes so I could enter a 4 digit pin that Amazon provides during the signup process. Very cool.

Being cautious to minimize costs, I selected the basic option for the support plan. Clicking continue brings me to a final page where I can launch the management console. Very nicely done and in the background I can hear confirmation emails pinging into my inbox.

Start End Duration Accrued cost

13:30 13:34 0:04 $ 0.00

Yes, that’s right. Total time to create the account was 4 minutes at zero cost where the total time includes the time it took for Amazon to phone me and send me confirmation mails that my account was ready to use.

At this point, I took a long break for lunch but also to play around in AWS, looking at the various services available via the AWS management console.

© Jason Greene 2015

2.2 PROVISION SOLUTION INSTANCES WITH SAP CALGoing back to SAP CAL, it’s time to start the provisioning process. Firstly, it’s necessary to obtain access credentials from Amazon. Since SAP only gives me the option to provision within us-east-1 I changed my AWS region to N. Virginia. Rather than use root access, I’ll follow the instructions from SCN below:

I’ve seen comments from some folks on the course where they had difficulty with this approach and ended up resorting to using the root account information but I’d be very concerned about doing that, particularly in a business environment where you might be using AWS for other solutions and services too. It’s a risk that can be avoided.

Having followed the instructions, I downloaded the access keys for use later and tried out the provisioning process in CAL. Unfortunately it didn’t work and I got only a partial instance creation. In the CAL appliance library console I received an error indicating that AWS was verifying my account. A pity that no information on this was provided in the course or CAL instructions. Since I could see that there were two EC2 instances up and running I terminated the installation to contain my rising panic at the thought of unchecked costs accruing on my credit card.

Start End Duration Accrued cost

15:00 16:00 1:00 $ 0.00

Thankfully still no costs associated at this stage, at least not visibly but also no great success. Time to try again.

Going back to SAP CAL, I restarted the process. I’m glad that I chose not to use my root keys as SAP have remembered the keys I used last time. Not entirely sure I’m happy about that as I expected that terminating my only instance would mean wiping out my setup information entirely for the sake of my

© Jason Greene 2015

privacy and security. I’m probably missing something but least I can delete the user in AWS once I’m finished with the course!

Rerunning the provisioning process again having waited a while seems to be more successful:

Although the console instructs me that I should allow 1hr for completion, everything looks good so far.

1x r3.8xlarge, 2x r3.2xlarge and an m3.medium. The CAL console still reports that the services are activating, although the individual systems all show up as green and AWS reports all systems go. I’ll give it another couple of hours before I suspend them…

No costs showing up yet. Although the initial provisioning failed, the second time appears to have gone well. I’ve not done anything else with the systems yet so of course I can’t be sure but the procedure was pretty painless. As I was at home I even managed to start preparing my family evening meal while the systems were being created, which is surely a huge benefit of the RDS approach.

Checking CAL again at 5:25, I can see my systems are now all active so I suspended them for the evening. Brilliant.

Start End Duration Accrued cost

16:00 17:25 1:25 $ 0.00

With this final step, my total potentially billable time so far, from opening my AWS account to suspending the new instances is 2hrs and 29 minutes. Cumulatively I’ve spent less than half a day on this so far even including my long lunch break.

In the next instalment I’ll move on to the first of the course exercises and I hope you’ll continue reading, thanks!

© Jason Greene 2015