Enterprise Cloud Architectures PT 1: What is a cloud

I think its important to first point out that there is no official document or standard that defines the concept of the cloud... but generally cloud architectures tend to have some specific  characteristics.

This series basically aims to lay out my opinions on the subject.

The problem...

In high traffic enterprise applications, where traditionally you'd have a few real heavy duty  application servers, and a couple database instances... you'd quickly reach an area of diminishing returns. As disk IO and connection counts rose, the capability of the machine to handle the load diminished rapidly.

Traditional architectures rely heavily on tightly coupled, highly localized data and content, and very little distribution, etc. Costs of SAN disk arrays go up dramatically as storage requirements increase, and still you're talking costs of a centralized (though often scalable) disk arrays

What is a cloud architecture?

Cloud architectures are designed to be massively scalable, highly available, and robust. The components or services are generally designed to be decoupled hopefully adding a great deal of flexibility to the architecture itself.

This diagram has nothing to do with this post (as this is not about specific implementations of cloud architectures), but actually I thought this was a perfect representation of a cloud. Traditionally in this sort of diagram, you'd expect to assume that items represented as a cloud clipart, were components of the app that weren't hard defined, nor were they specific about the implementation or number of machines involved in serving that component. In the windows azure diagram above you can see that the storage, queueing, and blob storage components are represented as a cloud. 

The symbology here denotes that these components are kind of service layers... that are designed to be decoupled from one another. Their particular implementation or hardware requirements are purposely abstracted away from these diagrams. The "service" is responsible for distributing commands to specific underlying servers etc. This is a very common trend in the cloud architectures I've come across thus far.

In the diagram itself above, the user story represents the process of uploading an image, storing binary data into cloud blob storage, and perhaps storing meta data about the binary, in the table storage.  The underlying implementation of the data store could be a number of database servers with replication, a database pool with sharding or partitioning, or several servers just attaching to same SAN lun's. The point here is that the underlying implementation, and the fetching and saving of that data is the responsibility of the blob storage layer, not the business logic layer.

Key characteristics in my opinion are...

1. Components or services are decoupled
1. Underlying implementation is never touched directly, and ultimately shouldn't matter from the point of view of the consumer.
2. Components or services shouldbe responsible for broadcasting messages, fetching data they need. After all you basically send a command, or a business object to the cloud... the cloud figures out who needs to handle it, and how its to be handled.
2. Implementations of components or services generally tend to be made up many many smaller nodes.. which ultimately generally are less powerful.
1. Webservice component of a cloud app should be machine in-specific. Web app's should be as stateless as possible. (requests may migrate from one machine to another at any point of time)
2. Sometimes the sheer number of connections on a single machine can hamper performance, and 4 lighter  weight machines vs 1 heavy duty machine may be more suitable.
3. Depending on application requirements, this may help alleviate storage woes, in that the $150k storage array could potentially be replaced by cheaper hardware and still be better off.

What a cloud is not...

I've seen all too many times, people mixing up typical client server applications, with the idea of a cloud architecture...when in fact that analog only really works in one direction.

1. Cloud architectures are not highly coupled
1. Layers should be clearly defined (business logic layer should not need to know how to fetch data out of database b on machine a which ultimately houses the data, but rather know the data layer knows how to delegate requests to appropriate places.)
2. Business logic layer should never know the physical breakdown of service implementation. Your business logic layer should never be sending data to specific machines in those components at all. In my opinion... any of those component participants should be able to either handle your request, or delegate it to the machine that can. Likewise for ultimate scalability... the ability to drop in and out of the pool is a huge plus.
2. Management of cloud architectures shouldn't usually refer to specific machines participating in the service components individually, but rather have something in place to allow bulk updates, and deploys.
1. If I have 20 webapp servers participating in my web services component, I really shouldn't need to push new builds to each of those machines individually, manage them individually etc. (after all the idea is that the web server component is a cloud of one or more machines, and the underlying implementation doesn't really matter.)

Keeping in tradition with many of my other blog posts.. this ended up being a sort of slightly structured mind dump. Sorry in advance if any of it seemed confusing or like parts of it ran away.

In next article, I intend to show how some products lend themselves to cloud architectures better than others, and hopefully layout some problems I've encountered in my development efforts (to perhaps prevent you guys from making the same mistakes I have).

Eclipse slow on OSX?

Just as a quick note...

I always noticed using eclipse for android development on my macbook pro was torturously slow, but I didn't realized there was a solution (a pretty odd one at that).

Download the 32bit carbon build. Seems kind of weird considering carbon is the legacy technology and 32bit seems like an unnecessary restriction.

Eclipse is considerably faster on osx in this configuration.

Hope it helps someone and their sanity!

Designing and building an android application pt 1

For me, getting my processes down is perhaps the hardest part when learning new platforms, new languages, etc. I dont Expect everyone to follow how I design and build applications from the ground up, but usually I just see how other people do things and adapt what I like, toss out what I dont.

For the purposes of this demonstration my intent is to build a package tracking application for my countless shipped packages.

A million dollar app from a 2c Idea

While I've historically had a pretty big opposition to super detailed rigid specifications, building even an informal requirements document and some use cases can help you consolidate ideas, prioritize features, and get an idea of what you MIGHT like the app to look like.

I usually start out by building a simple list of requirements, and end up with something like the following...

1. Track Packages
2. Client Server Architecture
3. Notification of status changes

Now we need to start drilling into each item better....

1. Track Packages 
1. Enter in a Package tracking number to add it to list of tracked packages
2. Display list of tracked packages (respected shipping company icons as well), and status. We'll also need the ability to manage the packages, so perhaps simple inline delete button
3. Details view with all package details. 
2. Client Server Architecture
1. Most tracking services have limits on calls to their API, so we want to buffer these through a server and control the actual hits against ups.com etc.
2. Packages to track, and their statuses will be stored on the phone as well. 
3. Notification of status changes
1. Periodically poll server for updates and send notification update
2. Eventually implement cloud to device messaging to handle push notifications

For now I'll leave it at that, and brainstorm the UI a bit. I highly recommend Balsamiq Mockups (http://www.balsamiq.com/) for mocking it up. Some of the UI bits are iphone specific, but translate pretty evenly even for android apps. About 10 minutes and I came up with the following.







Anyway, now I've got a pretty good idea of where I am going without going to nuts with the prototyping, the rigid specification, or overly presumptive UI.

Next time, we'll start looking at the structure of the projects.

UNTIL NEXT TIME!

google protocol buffers in android

Lets face it... there is a good chance your android app will need some sort of mobile to server communication of some sort. You could go the route of raw sockets, with "packets" of your own design. You could post json, you could post xml, or you could build a web form and post to a url.

All of these are very valid options, however I've kind of decided I'm a fan of protocol buffers. What are protocol buffers you might ask? A language / platform independent framework for serializing / deserializing binary data that generally produces lightweight messages quickly.(read more about it here: http://code.google.com/p/protobuf/)

Protocol buffers

The general idea is you have a .proto file that defines your "protocol" in plain human readable text. It supports many data types, ability to not provide a property, and even send raw bytes.

I've tried both the usual protocol buffers implementation from url above with and without the -lite flag, and the protobuf implementation for j2me and frankly seems like so far the j2me implementation is faster and lighter weight, and hasn't yet been problematic for missing anything.  (http://code.google.com/p/protobuf-javame/)

I am going to create a very simple proto file that might represent a message you may be sending in your application.

message LocationSearch{
       required string searchterms = 1; // the text body of the search
       optional double latitude  = 2; //optional lat/lng coords... if its provided we'll use it.
       optional double longitude = 3;
}

I can run proto generator (more on this in a moment) on it and it will create LocationSearch.java on the android side and then proto generator on the .net side (my server will be .net in this case) and get LocationSearch.cs.

In android we need to add the reference, so right click project and then go to Build Path -> Add External Archives. Choose the location you downloaded protobuf-javame-1.0.0.jar. Copy the generated .java file to some place in your packages and we'll get started momentarily.

How you decide to delineate messages could be anything, but I've just implemented handlers so if I post to /LocationSearch on a specific server, I should get a response back. To do this, we'll need to download and reference (the same way we referenced protobuf) apache httpclient library v 4.02.

I've created a utility method that posts a simple message from byte[] and returns a byte[] representing the response. I've not read up on optimizing for android recently... so there are undoubtedly places that need to be final etc. First the method...

public static byte[] postMessage(String url,byte[] data, String filename) throws Exception
    {
  byte[] rData = new byte[0];
  HttpClient httpclient = new DefaultHttpClient();
  HttpPost httppost = new HttpPost(url);

  MultipartEntity reqEntity = new MultipartEntity(
   HttpMultipartMode.BROWSER_COMPATIBLE);

  reqEntity.addPart("message",
    new InputStreamKnownSizeBody(
      new ByteArrayInputStream(data),
      data.length, "application/octet-stream", "message"));
  Log.i("Sending:",new String(data));
  httppost.setEntity(reqEntity);

  System.out.println("executing request " + httppost.getRequestLine());
  HttpResponse response = httpclient.execute(httppost);
  HttpEntity resEntity = response.getEntity();

  if (resEntity != null) {
   
   ByteArrayOutputStream bos = new ByteArrayOutputStream();
   resEntity.writeTo(bos);
   rData = bos.toByteArray();
   Log.i("Response",new String(rData));

  }
  return rData;
 }

and a prerequisite class to handle being able to upload from a byte[] vs disk.

public class InputStreamKnownSizeBody extends InputStreamBody{
 private int lenght;

 public InputStreamKnownSizeBody(
   final InputStream in, final int lenght,
   final String mimeType, final String filename) {
  super(in, mimeType, filename);
  this.lenght = lenght;
 }

 @Override
 public long getContentLength() {
  return this.lenght;
 }
}

Ok... so now we have the facility to post a message and get a response back. Lets see what a round trip might look like...

//lets create a protocol buff object, and send it on its way.
 LocationSearch search = LocationSearch.newBuilder().setsearchterms("rob john*");  

 // remember lat/lng are optional
 byte[] response = postMessage("127.0.0.1/LocationSearch",search.toByteArray(),"dummy");

 //now if we had created a "LocationSearchResponse class with applicable data, we could do...
 LocationSearchResponse responseObj = LocationSearchResponse.parseFrom(response);

A couple notes about my examples

• I assume the server portion of this you guys can set up, and handle parsing/sending a response. The general scenario is pretty much the same everywhere
• You'll want to make sure all your basics are covered as far as exception handling and failing gracefully (some of these things get in the way of conveying concepts =)
• Obviously if you aren't running a server on your local machine, the URL in the example above will simply not work.
• Filename in the postmessage method above is currently not doing a thing (yet)

Homework

1. Create the corresponding return message with whatever you think might be appropriate to receive from server. 
2. Implement server side logic. Once you parse the form and get the "message" element which from the server appears as a posted file, .parseFrom into your message format. Create the response, and write the resulting bytes to the response.

I hope this example proved helpful if nothing else, than to know what another option is as far as device to server (or even device to device) communications.

Until next time!

Android reusable button wiring

Prior to android 1.6, button wiring in android was a process of building out your event listeners and wiring programmatically.  1.6 introduced declarative button wiring in the xml for the ui.

Earlier on, I was trying to build something that loosely resembled the phone dialer but of course NOT the phone dialer. About 15 buttons each having the differentiation from one another nothing more than having a value of a diff number.

I quickly got frustrated writing switch cases and if's for something that was simply taking the value and passing it directly on. I instead I realized I could do the following...

...

And for my click handler...

public void dialerButtonClick(View target) {
doSomethingWithButtonClick(Integer.parseInt(target.getTag().toString()));
}

Is this going to make sense in every case? No... probably not. getTag almost definitely is more of a hit, than simply comparing R.id's, but it seemed to work pretty good for what I wanted it for.

Until next time.

Understanding the Activities and Intents in android

When I first started tinkering in android I had a problem finding the equivalent to views and activities  in languages I've written for before. It just took a while to understand when I should be building an activity etc. Not a difficult concept to understand, just realizing that this isnt exactly a 1:1 map to other architectures. An activity is generally a process, perhaps create account, perhaps create blog post.

Imagine for example that you have a twitter app (I honestly dont use twitter, so my example may be bad, but I think its something enough people use to understand where I am going with it.). This twitter app has a few key "core" concepts. Posting, Change Follow status, Change settings etc. It may be obvious to some, but these would be ideal candidates to be activities. Activities which may consist of one or more Views being displayed. How exactly does the application entry points have anything to do with this? Using intent filters etc, I can allow any of these activities to be launched from other apps, or directly from launcher itself as exactly that... a well defined process. Multiple activities dont have to mean multiple entry points, but when you think of exposing New Tweet functionality to other apps, you expose the activity in your activity filter.

Intent intent = new Intent();
intent.setClassName("com.mycompany.someapp", "com.mycompany.someapp.Tweet");
intent.putExtra("com.mycompany.someapp.TweetMessage", "Hello world!"); 
startActivity(intent);

Which loosely translates to the pseudo code:
Open Tweet activity setting TweetMessage to "Hello World" (which I can retrieve and use later

A brief post, but hopefully may ease the transition from development in other languages

Android view containers PT. 1

Ok, so what do you do if you need to build something like the app drawer from scratch, having freedom to add Views to it at will?

To build a layout that allows for the display and management of child views itself, you need to inherit from ViewGroup, or a subclass of view group (technically you COULD build out your own implementation inheriting from a straight view, but we wont get into that now).

As a sort of experiment in software development, I'll be building a ViewGroup subclass that will effectively work like a LinearLayout, with the exception that the content will actually wrap. Its important to note that I intent to start with very simple minimal code, and build it into something I'd consider ready for primetime.

First things first we need a new class. I personally created a new WrappingLinearLayout class in my com.mycompany.UI package. This class should obviously extend "ViewGroup", and implement the abstract methods etc.

Drawing in android works in two major phases.. measuring (helping plan ahead as far as how to draw controls accounting or not for things like margins, and weights), and laying out.

Measuring

The concept is more or less loop through children views, and decide measurements for individual view items to be drawn. Depending on the complexity of your layout, you may just do nothing more than measure, or you might go to the other opposite end of the spectrum where you account for orientation, dpi independent rendering, LayoutParams like fill_parent and wrap_contents, and weighting. To start with the simplest possible solution.. we'll provide bare minimum implementation of Overridden onMeasure method.

@Override
    protected void onMeasure(int widthMeasureSpec, int heightMeasureSpec) {
 //get count of children view elements
 final int childCount = this.getChildCount();
  
 //iterate through children and measure each accordingly.
 for (int i = 0; i < childCount; i++) {
     View child = getChildAt(i);
      
      //call measure on each child, so that we can call getMeasuredHeight() later.
      //UNSPECIFIED basically tells the measure to use whatever was originally supplied for size and layoutparam information
      child.measure(MeasureSpec.UNSPECIFIED, MeasureSpec.UNSPECIFIED);
     
  }
  
  //now we need to measure the actual controls dimensions... we dont have to call base here, but for now
  //let android do its thing.
  super.onMeasure(widthMeasureSpec, heightMeasureSpec);
 }

Pretty simple, and commented if there is any confusion.

Now that we've got everything measured out in there most basic forms, lets get to the layout.

Layout

The gist of this step is to handle the actual layout... (in the end, the measure step will have determined the size of all the views including margins weights etc, and now its time to perform layout of those children)

So currently my goal is to simulate the most basic of functionality of LinearLayout, which is that I can take the childviews and tile em (note: at this point, views will still eventually overflow off this custom view). So with that said, we need to effectively iterate through the child views drawing the first, and subsequently offsetting the next by offset+width of the first. Lets override onLayout to achieve this basic effect.

@Override
 protected void onLayout(boolean changed, int l, int t, int r, int b) {
  //get count for iteration through child views
  int childCount = this.getChildCount();
  
  //initialize the x offset of the view. (currently only handling horizontal layout, and doesnt wrap yet!)
  int lOffset = 0;
  
  //iterate through the child views laying them out as specified in the initial instances.
  for (int i = 0; i < childCount; i++) {
      
   View child = getChildAt(i);
   
   //if this view is not "Visible" or "Invisible" just skip
   if (child == null || child.getVisibility() == GONE)
    continue;
   
   //i'm giving children a 5px margin top and left here, which should be done by theming.
   //note: for the left parameter its the lOffset and that the right parameter needs to include
   //      offset+width. Also, we call getMeasured(Width | Height) because getHeight wont work here.
      child.layout(lOffset+5,5,lOffset+child.getMeasuredWidth(), child.getMeasuredHeight());
      
      //finally lets make sure offset gets incremented the width of just-processed view.
      lOffset+=child.getMeasuredWidth();
  }
  
 }

Ok, so as far as basic implementation goes, we've got the bare minimums there. We didn't build out the constructor for use with declarative instantiation, so we'll slap together a quick bunch of code to instantiate the WrappingLinearLayout, and add a couple items to it.

//instantiate control
        WrappingLinearLayout wrl = new WrappingLinearLayout(this.getApplicationContext());
        
        //set bg color of control to white for better troubleshooting
        wrl.setBackgroundColor(Color.WHITE);
        
        //create a button for testing
        Button button = new Button(this.getApplicationContext());
        button.setText("DoStuff()");
        button.setLayoutParams(new          LayoutParams(LayoutParams.WRAP_CONTENT,LayoutParams.WRAP_CONTENT));
        button.setWidth(100);
        button.setHeight(50);
        
        //gotta make sure it actually flows... need a second control
        Button button2 = new Button(this.getApplicationContext());
        button2.setText("DoStuff2()");
        button2.setLayoutParams(new LayoutParams(LayoutParams.WRAP_CONTENT,LayoutParams.WRAP_CONTENT));
        button2.setWidth(100);
        button2.setHeight(50);
        
        //add two buttons to WrappingLinearLayout
        wrl.addView(button);
        wrl.addView(button2);

        //finally set the content view to the custom layout view we created...
        setContentView(wrl);

Conclusion

So there you have it... a very simple and rudimentary example of the beginnings of a flow layout container. Lots of things missing that I hope to add in the near future with the rest of this series. This code should by all means NOT be used, for it has a lot of hardcoded values... but should be a little easier to explain HOW it all works.

Until next time!