Thursday, September 27, 2012

Fuctional Programming Principles in Scala - Getting Started

Sometime back I registered for the Functional Programming Principles in Scala, on Coursera. I have been meaning to learn Scala from a while, but have been putting it on the back burner because of other commitments. But  when I saw this course being offered by Martin Odersky, on Coursera, I just had to enroll in it.

This course is a 7 week course. I will blog my learning experience and notes here for the next seven weeks (well actually six, since the course started on Sept 18th).

The first step was to install the required tools:
  • JDK - Since this is my work machine, I already have a couple of JDK's installed
  • SBT - SBT is the Scala Build Tool. Even though I have not looked into it in detail, it seems like a replacement for Maven. I am sure we will use it for several things, however upto now I only know about two uses for it - to submit assignments (which must be a feature added by the course team), and to start the Scala console. Installed sbt from here, and added the path to it's bin directory in my .bashrc file. When I first started sbt, I did not realize that sbt would need to connect to the Internet to download it's dependencies. My computer was disconnected from the Internet, which caused sbt to give up. This is the first time I realized that sbt seems to be doing something that I would expect Maven to do.
  •  Next, I installed the Scala IDE, which seems to be Eclipse with a Scala plugin. Since I already have Eclipse, I wonder if I cold have simply installed a plugin.
Once I had the tools setup ready, I created a workspace for all my DIYLearning courses (since I plan to always be enrolled on at least one course, from Coursera, Udacity, edX, or any other open courseware offering that is good).

Then I downloaded the example project, made a few modifications to it, ran unit tests and submitted it. I learned quite a few things by wandering around this example project.

Directory structure is similar to Maven:
The directory structure of the projects is similar to what it might have been had I used Maven. The sources were in 'src/main/scala', and tests in 'src/test/scala'. sbt seems to have a directory structure similar to Maven. I like that. It does not make sense to relearn everything from scratch for every single tool.

Scala imports are slightly different
This is how Scala imports everything from a namespace. Notice how they have deviated from Java imports by using the '_' character for the wildcard. I asked a question on the forum, and was told that functional programming languages by convention use the '_' character for wildcards. So ok, that too makes sense.

import Lists._

Unit tests in Scala
Unit tests in Scala look similar to unit tests in Java, but there are a few things which stand out as different. Scala uses ScalaTest, as the unit testing tool in Scala. To create a ScalaTest test case, we have to extend FunSuite. Their website also mentions something about traits. I have heard this term before, but do not really know what it is. Need to look into what it really means. I also noticed that ScalaTest's have only one type of assert, which is equivalent to assertTrue. This also makes sense to me, because everything can be reduced to an assertTrue assertion.

ScalaTest introduces an operator called 'test'. This operator takes two arguments, the first one being the name of the test, and the second one being the body of the test. The body usually consists on an assert. Here is how a test operator looks

test("testing equality")(assert(1==1))

The body of the test can also be given as a block, which makes it more readable.

test("testing equality") {assert(1==1)}

Thursday, April 05, 2012

Creating an Eclipse web project using Maven and Struts 1.x

I am doing some Struts work after a long time. Interestingly it is for a Test Driven Development training. A client wants me to do a hands on session, using Struts 1.x, and EJB's. They also want to do the hands on sessions with a small but production project. It seems their developers are tired of Calculator and Shape projects :-) Well I can't really blame them.

I have been planning to create some micro applications around . I hope to create each application using a different technology, so I can have several reference points for teaching.

For this session I am going to make a web application which will store and display my slides. I am sure you are thinking ... but why not just use SlideShare ? Well besides the fact that I think this makes a great application that is small, but also production quality. Ideal for using to teach. But there are other reasons as well. It is very hard to embed code snippets on Slideshare. Slideshare also does not support comments per slide.

Since I started working with Maven sometime back, I thought, why not use Maven for dependency and build management for this project.

While creating an Eclipse web project for Struts 1.x using Maven led me to some issues, and I feel it might help others if I documented them.

The first thing I needed to do was to find a Maven archetype for creating a Struts project. Interestingly the Struts 1.x archetype has been removed from the Maven 2 repository. So, I downloaded the archetype from this repository. Then I did a 'mvn install' to install the archetype in my local Maven repository.

svn co
cd struts-archetype-blank
mvn install

Then I created my Struts project, and used maven's eclipse plugin to generate Eclipse proejct files.

mvn archetype:generate -DarchetypeGroupId=org.apache.struts -DarchetypeArtifactId=struts-archetype-blank -DarchetypeVersion=1.3.5-SNAPSHOT -DgroupId=com.diycomputerscience -DpackageName=com.diycomputerscience.stslides -DartifactId=struts-slides

mvn eclipse:eclipse

However, when I tried running the project, I realied that this project was not an Eclipse dynamic web project. That's not good because I wanted to run it from within Eclipse. A bit of searching led me to this post which explained that I needed to run

mvn eclipse:eclipse -Dwtpversion=2.0

Now all seems to be fine. I have a Struts 1.x dynamic web project which is managed by Maven and can be run within Eclipse.

Monday, June 06, 2011

Running your own one person company

Recently there was a post on PuneTech on mom's re-entering the IT work force after a break. Two of the biggest concerns mentioned were :
  • Coping with vast advances (changes) in the IT landscape
  • Balancing work and family responsibilities
Since I have been running a one person company for a good amount of time, I suggested that as an option. In this post I will discuss various aspects of running a one person company.

  • You have full control of your time. You can choose to spend as much or as little time as you would like. There is also a good chance that you will be able to decide when you want to spend that time.
  • You get to work on something that you enjoy doing. Tremendous work satisfaction.
  • You have the option of working from home.
  • It can take a little while for the work to get set, so you may not be able to see revenues for some time.
  • It takes a huge amount of discipline to work without a boss, and without deadlines.
  • You will not get the benefits (insurance, etc) that a company would give you.
  • Your finances may or may not show a constant graph.
  • You may have a hard time getting home loans, since most banks (especially in India) feel that employed people are more stable (whatever that means :-) )
  • People may think you are not doing real work (most people have a very skewed notion of what real work means)
  • You will have to deal with distractions, if you are working from home.
Even though the disadvantages outnumber the advantages, qualitatively I think the advantages are vast. I may be a bit biased, but I really think that a 9 - 5 job in a large organization, totally sucks the soul out of a person.

I will restrict this advise to India, whose laws I am accustomed with. You do not need to register any legal entity to run a one person business. You can run it on your own name, you can raise invoices on your own name, and you can file your taxes on your personal PAN number.

Some people may create a sole proprietorship, but that is purely to have a business name and to be able to invoice clients on a business name, rather than your personal name. Besides this, having a sole proprietorship has the advantage of being able to easily separate personal and business accounts. The latter can be achieved simply by keeping a separate savings account for work related transactions.

Some people may also choose to create an LLP. Besides the points mentioned for sole proprietorship,an LLP will shield your personal finances against business losses, lawsuits, etc.

I will not get into the legalities of creating a sole proprietorship, or LLP. My suggestion is to start work on your personal name, and create a legal entity later, if required. You should consult your CA for details of creating a business entity.

Types of work
There are many types of work that can be done as a one person business. I will focus on IT related fields in this post.

Many people make a living by doing freelance projects. You can either get projects directly from clients, or you can work on one of many freelancing websites. These websites publish projects that people want completed. If you feel that you can do the project, then you can bid for it. If you get the project, you will have to complete it in the stipulated time, and with the required features. The payment usually comes once the client is satisfied with the work. Most of these websites allow the client as well as the developer to rate each other. These ratings go a long way in determining whether a client will give you projects. Typically a freelancing website have work related to coding, documentation, website design, etc. Some such websites are: Elance,, FreelanceSwitch, ODesk, and (Disclosure: I will get a small commission if you register on some of these websites, following the provided link)

Regardless, of whether you get projects from a freelancing website, or on you own, you should always have your own website, and a blog. I cannot stress how important it is for a freelancer to have a blog. Your blog is the place where potential clients and friends get a glimpse of the real you. It is also your showcase for projects completed, and an online portfolio.

Contract programming:
If you have been in the software industry for a long while, then you probably already have a lot of contacts who would like to give you some contract programming work. This can also be lucrative and fun. In contract programming you typically make a commitment (such as 20 hours per week), and charge on an hourly rate.

If have passion for sharing your knowledge with others, then teaching is a wonderful way to do satisfying work and earn a livelihood. There are various avenues for teaching. You can teach in a local college, you can teach at companies, or you can teach on the Internet. You can also do all of these if you desire.

Some places you can create courses and teach on the web are,, and You can also create your own website for teaching using Moodle, or a similar open source software. I have created my own website for teaching computer science, but I am not yet making any significant revenue from it, so beware before you take it as a role model :-) However, you can check out this website of a database expert who maintains a learning community on his website.

You can also create and sell screencasts like the people at Peepcode, and Destroy all software.

I think there is a lot to be said about teaching, and it probably warrants a separate blog post.

Creating mobile applications:
Many people make a living by creating mobile applications, for Android, iPhone, etc. The amounts vary, but it is very much do-able. Several people give helpful tips and also post there monthly incomes on their blogs. You might find them useful. This is one example website of an independent developer who posts monthly incomes and also gives helpful tips.

Creating stock photos, and designs:
If you are the arty type, you can create stock photos, images, or icons, and put them up for sale on stock photo websites. There are many such websites, and some people are able to make a good continuous stream of income from them. iStockPhoto is one such website. You can also create designs for clients on sites like

Selling websites:
If you enjoy making and selling websites, you can make and auction your websites on

Niche websites:
If you are really passionate about something, and also have a lot of knowledge in that field, then you can share your knowledge on a niche website, and earn revenues from advertisements, affiliate sales, subscriptions, etc. See this blog post on information about building niche websites.

If you have something to sell, you can also set up shop using a hosted e-commerce solution. However, keep in mind that managing logistics, shipping, handling, and customer care may be more than what one person can handle.

I hope you found this post useful. This is a brief list of things that can be done as a one person entity, and some pointers on how to go about doing it. I think plenty more can be said on this topic.

I will try and write further blog posts talking about each of these points in greater detail.

Wednesday, April 27, 2011

My HSQLDB schema inspection story

This is a simple story of my need to inspect the schema of an HSQLDB database for a participar FOREIGN KEY, and the interesting things I had to do to actually inspect it.

I am using an HSQLDB 1.8 database in one of my web applications. The application has been developed using the Play framework, which by default uses JPA and Hibernate.

A few days back, I wanted to inspect the schema which Hibernate had created for one of my model objects. I started the HSQLDB database on my local machine, and then started the database manager with the following command

java -cp ./hsqldb- org.hsqldb.util.DatabaseManagerSwing

When I tried the view the schema of my table, it showed me the columns and column types on that table, but it did not show me columns were FOREIGN KEYs.
Image and video hosting by TinyPic

Image 1: Table schema as shown by HSQLDB's database manager

I decided to search on StackOverflow and find out how I could view the full schema of the table in question. I got a few hints, and they all pointed to the system tables, so I decided to turn on the "show system tables" option from HSQLDB's view menu.

The first table that caught my eye was information_schema.system_tables, so I fired the query:

select * from information_schema.system_tables;

This gave me all the system as well as application tables, but did not give me the detail I was looking for, which was the FOREIGN KEY contstraints on the 'USERREGISTRATIONDATE' table.

The next table that caught my eye was information_schema.system_table_constraints, so I fired the following query:

select * from information_schema.system_table_constraints where table_name = 'USERREGISTRATIONDATE';

This helped a bit further. It told me that the USERREGISTRATIONDATE table had a FOREIGN KEY and the constraint name was 'FK98DCB61247140EFE'.
Image and video hosting by TinyPic

Nice, but it still did not tell me which column the constraint refered to.

The next table I came across was information_schema.system_crossreference. This seemed to have a column called 'FK_NAME'. Good, I fired the query:

select * from information_schema.system_crossreference where FK_NAME='FK98DCB61247140EFE';

Here is what this table showed me...

Image and video hosting by TinyPic


Awesome, this time it showed me the name of the the FOREIGN KEY 'FK98DCB61247140EFE' references the 'ID' column of table 'USER'

I finally had my answer, but before ending this post, let me mention just one more thing. I realized that I did not need the information_schema.system_table_constraints table at all. I only needed the information_schema.crossreference table with the following query:

from information_schema.system_crossreference

Image and video hosting by TinyPic
Image 4: Finally the query that clearly showed me the FOREIGN KEY

Now all this should not have been so time consuimng. The GUI tool should have given me this information right away.

Friday, October 15, 2010

Double Checked Locking And Java Singletons

I read this article by Bill Pugh on why the double checked locking idiom does not guarantee thread safety in Java Singletons. That article taught me a lot of new things, and to be honest, I had to re-read that article at least a couple of times to partially understand it :-)

I recently created a presentation to make at DevCamp on this topic. What follows are my slides and an explanation of each slide. I hope you enjoy this presentation and find it useful.

slide 1:
In this presentation I will discuss the double checked locking idiom, and explain why it does not work to provide thread safety to Java Singletons. I will also talk about how using volatile fields will fix the problem in JDK 1.5 onwards.

slide 2 (Singleton):
Many of you might already have used the Singleton design pattern. In case you have not, here is a brief description of what Singletons are. The Singleton pattern is used when we want to ensure that there is only one instance of a class per something. In most cases, the something is 'JVM - Classloader' combination. So, in such cases we want to ensure that there is only one instance of the Singleton class in the JVM-Classloader. But we are not restricted to this. We may need only one instance of the Singleton per Session, or per Request, or per anything else that makes sense in the software you are making.

It is clear that we want to control the instantiation process of the Singleton class. To do this, the first thing we do as shown in the example is to make the constructor private. With the constructor private, no other class can create an instance of this class. OK, so how will we create any instance of this class? We provide a public static method called getInstance() which will return an instance of this class. We will also use a private static field called instance to hold the one and only instance of this class. Since we control the getInstance() method, every time it is invoked we will return instance. The first time however, instance will not be initialized, so we put a null check in getInstance() and instantiate instance if it has not already been instantiated. Hence forth we will always return a reference to instance. This way we ensure that only one instance of our Singleton will be ever created (the one we save in the field called instance).

slide 3 (Is This A Safe Singleton?):
Look at the class carefully and ask yourself "Is this Singleton safe?"

slide 4 (The Singleton is not Threadsafe):
If you have looked at the class carefully, you will realize that the Singleton is not thread safe. This slide shows why it is not thread safe. Imagine that a thread T1 invokes getInstance(). If an instance of the Singleton has not been created till now, then instance will be null, and T1 will enter the if block. Now imagine that T1 gets pre-empted, and thread T2 also calls getInstance(). Since T1 got pre-empted before it got a chance to create instance, the reference in instance is still null. So T2 will also enter the if block. Now if T2 does not get pre-empted then it will go ahead all the way where it will create an instance of Singleton, and probably even use it. Now when T1 is resumed, unfortunately it does not know that the Singleton has already been created, since it has already gone past the null check. T1 will also happily create an instance of our Singleton, thus breaking the Singleton pattern. We have 2 instances of the Singleton which is simply not allowed.

slide 5 (Let;s Make It Threadsafe):
Alright so we need to make the getInstance() method thread safe. The simplest way to do this is by making it synchronized. Once it is synchronzied, only one thread will be allowed to enter the method at a time, and we will never get into an issue where we end up having multiple instances of the Singleton.

slide 6 (That Was Expensive):
Making a method synchronized is expensive. A thread executing a synchronized method must acquire a monitor before it can invoke the method, and release the monitor after executing the method. This takes time and CPU cycles. In early JVM's (I have read somewhere), the time to invoke (not execute) a synchronized method was as much as 100x the time taken to invoke a non synchronized method. I think this was improved in later JVM's where it was about 15x. I recently did a quick test on JDK 1.6, and the number I got was 2x. Regardless of how much slower it is, it is definitely slower, and programmers always want to make their code fast.

slide 7 (Let's Use Double Checked Locking):
Some very smart programmers came up with the "double checked locking" idiom to relieve the code of this expensive operation. What we really should have done, is only synchronize the code which instantiates the Singleton, and not the entire method. That way we incur the expense of the synchronized block only once, when the Singleton needs to be created, and all subsequent times (instance == null) will always be false, and the instance will be returned to the calling code. We put a second check inside the synchronized block, because it is possible that a thread could get pre-empted just after the first null check, but before it enters the synchronized block. In such a case, if another thread enters this method, and goes all the way to creating the instance then the first thread will not know of it and will create a second instance. To prevent this, we put another check in the synchronized block.

So this is a pretty smart solution. We eliminate the synchronization on the method, thus eliminating synchronization on the main path.

slide 8 (Will It Work?):
The solution described in the previous slide (double checked locking) is a pretty smart solution. But it does not work.

slide 9 (Why?):
There are two reasons why it does not work. The compiler as well as the processor (as well as the JVM) may reorder instructions if they feel it might be more optimal. Off course they cannot randomly reorder instructions but they could if they can prove that the reordering will maintain as-if-serial semantics. However, this is not the only reason. On a multi-processor system, each processor has it's own memory cache. The cache is not always synchronized with main memory immediately. This can cause a write to a memory location to happen in the local cache, which will not be visible to other threads which could be running on other processors. Even if the writing thread does flush it's local cache, it is still possible that another thread which reads that value, may not have pulled in most recent values from the main memory into it's local cache. This can cause a situation very similar to reordering where the effect of a write is not visible to another thread that is reading that variable's value.

slide 10 (instance = new LoneRanger(””) != Atomic operation):
If you look at the example in slide 7, you might think that the statement below is an atomic statement.
instance = new LoneRanger();
But that is not the case. For the sake of simplicity this slide shows the statement above broken into 2 operations. In the first part, an instance of the class is constructed, and in the second part a reference to that instance is assigned to the variable instance.

slide 11 (Compiler Reordering):
Now what if the compiler reorders the instructions in such a way that the field instance is assigned an uninitialized block of memory (created for the class LoneRanger), and then the constructor of LoneRanger is invoked in the second step.

slide 12 (Compiler Reordering Pseudocode):
This slides shows pseudocode to understand the effect of compiler reordering on the double checked locking code. The code in the synchronzied block which instantiates our Singleton is broken into two steps: assignment, and initialization, where the assignment happens before the initialization.

Now imagine thread T1 enters the getInstance() method, enters the synchronized block and executes the statement which assigns the block of memory allocated for LoneRanger to the variable called instance. If T1 gets preempted and thread T2 enters getInstance() then T2 may see a non null value for instance. Thus T2 will actually be returned an initialized object of LoneRanger. This could cause all sorts of bugs in the software.

So in the first problem described a few slides back, we ran into the issue of having multiple instances of our Singleton, which we attempted to fix with the double checked locking idiom. However, this introduced the issue where a thread might be given a reference to an uninitialized instance of the Singleton.

slide 13 (Can We Prevent reordering):
Programmers never give up :-) When an even smarter programmer was explained this problem, he remarked "so let us prevent reordering and our problem will be solved !". Ok let's try that, but how do we prevent reordering? Well there is something called a memory barrier, which may help us. Instructions cannot be reordered across memory barriers (although they may be reordered within a memory barrier). Let's see if we can introduce memory barriers to prevent the reordering that's been giving us such a hard time till now.

slide 14 (Memory Barrier):
A memory barrier is a low level (at the level of the processor) construct which is used to create a fence around instructions. Instructions which are fenced inside a memory barrier cannot be moved out of the fence, and memory caches are also synched with main memory when a memory barrier is encountered.

slide 15 (Memory Barrier):
This slide explains memory barriers with an illustration. In this slide notice that instructions (instr2, instr3, & instr3) are fenced with the memory barrier. Because of the semantics of a memory barrier these instructions can never be moved out of the barrier, but they may be re-ordered within the memory barrier. We can also see when the memory barrier is entered the local processor cache is invalidated and latest values are read from the main memory, and when the memory barrier is exited then the local cached is flushed and the main memory is updated with the latest values.

slide 16 (Memory Barrier):
Because memory barrier is a low level construct, there is no way to explicitly create one in a high level language like Java. However, the synchronized keyword in Java implicitly creates a memory barrier. Before I read this, I had no clue that synchronization in Java is anything more than a mutex. But I read somewhere that when a monitor is obtained an memory barrier is also created and when a monitor is released the memory barrier ends (this is my understanding, please correct me if this is wrong).

slide 17 (Double Checked Locking With Memory Barrier):
The fact that assignment and construction of our Singleton could be reordered created the issue of potentially having an uninitialized Singleton. We want to ensure that the reordering does not happen. For that let us separate the construction and assignment with a memory barrier and put the assignment after the construction. We do this with 2 synchronized blocks and a temporary variable. In the inner synchronized block we will initialize the Singleton and assign it to a temporary variable. We don't care if this is reordered, because the class variable instance will still be null which will prevent another thread from getting an initialized instance of the Singleton. Then we assign the reference to the temporary variable to the class variable instance. This happens outside the memory barrier. So by employing a memory barrier we are hoping to maintain program order in the execution of instructions. Again a very smart solution, but unfortunately this does not work either.

slide 18 (Monitor Exit Semantics):
The semantics of monitor exit specify that everything that happened before the monitor exit should happen before it, which means that nothing from the inner synchronized block (where we instantiate the Singleton) will be moved out of the inner synchronized block, however, it does not mean that something will not be moved from outside of the block to within it.

slide 19 (Monitor Exit Semantics):
This slide explains monitor exit semantics with an illustration. As we can see inst2 and inst3 are within the memory barrier. Neither of them will be moved out of the block, but inst4 which is out of the memory barrier may be moved in the barrier.

slide 20 (Double Checked Locking With Memory Barrier):
This slide shows the code where we tried to use the memory barrier. But this time we show the code with a potential reordering. With the statement instance = tempInstance inside the memory barrier it could be further reordered to the point before the actual construction of the Singleton, bringing us back to the same problem.

slide 21 (OK So What The Hell Will Work ?):
Alright enough of playing around... not I am getting a bit edgy.... just tell me what the hell is going to work. Is this what you are saying to yourself? Java has a special modifier called volatile. We can use these fields to help us with the Singleton.

slide 22 (Semantics of volatile):
The volatile modifier is used in Java to communicate state changes between threads. This slide explains the semantics of volatile with an illustration.

The code on the left shows a class which has two methods: writer() and reader(). The writer() method writes values of variables x and v to memory, and the reader() method reads values of x and v from memory. Imagine that both these method will be called by different threads T1 and T2, running on different processors P1, and P2. Each processor has a local memory cache and off course there is also the main memory. The variable v is volatile and x is not volatile.

When thread T1 executes writer() we are guaranteed that the instructions will not be reordered because v is volatile. We are also guaranteed that when thread T1 exits the method, processor P1's local cache will be flushed to main memory, which means that the value of x as well as v will be visible to other threads. When thread T2 executes the method reader(), it first reads the value of v, this is going to invalidate the local cache of P1, and fetch the latest values from main memory. This ensures that it sees the latest values (which in this case were the ones written by T1 when it called writer()) of v and x.

So this slide explains the semantics of volatile and how volatile may be used to ensure that instructions are executed in program order and also ensure the visibility of writes by one thread by another thread.

slide 23 (Double Checked Locking With Volatile):
This slide shows our old and well known example which uses double checked locking, but this time with a little difference. This time we have made the variable instance a volatile field. Making it volatile will ensure that the write to initialize the LoneRanger object and the assignment of that instance to the field instance will not be reordered. This elimitaes the problem of seeing an uninitialized Singleton object.

slide 24 (Singleton With Static Initializer):
After doing all these coding gymnastics, in this slide we show a solution which is probably the simplest solution for creating a thread safe Singleton. Instead of instantiating the Singleton in getInstance(), we use a static initializer, which will instantiate the Singleton when the class is loaded. Java semantics ensure us that all static fields of a class will be completely initialized before the class is available for use. This means that the field instance will be properly initialized before anyone makes use of the class to get an instance of the Singleton.

slide 25 (Singleton With Static Initializer):
In this slide we understand the pros and cons of using a static initializer. Even though using a static initializer is the simplest solution, it is possible that everything the Singleton needs to initialize itself may not be available when the class is loaded (which causes the static initializer to be invoked). It is also possible that the Singleton may be eagerly loaded, which may result in greater loading time for our application, something that may be undesirable if instantiating the Singleton is an expensive operation.

slides 26, 27, 28 (Summary):
In these slides we summarize the main points covered in the presentation till now.

slide 29 (Resources):
Links to some very good and relevant articles, including my favorite article by Bill Pugh.

slide 30 (Thank You):
Thank you for your patience. I hope you found this presentation useful. One more thing before signing of - many people (here and here, but someone also disagrees) consider Singletons to be evil...

Saturday, July 24, 2010

DIY Masters in Computer Science

Since my formal education, a lot of advances have taken place in software development. I have been able to keep up with a few with regular reading and practice. But a lot of this learning has been a bit random, and as a result a bit dissipated as well. I feel like I want to engage in continuous learning, in a more organized manner.

Over the years there are several core concepts which I have forgotten because I have not been able to use them in my regular work. I feel like relearning those concepts.

I think the volume and content of both these can constitute a masters course in Computer Science. But I do not want to go back to school. Not because there is anything wrong with school - I had a great time in grad school. But here's why...

I don't want to go back to school because I want to define the courses I want to learn, and not pick up from what's offered.
I don't want to go back to school because I want to be able to learn at my own pace, which at times may be slower than 1 course per semester.
I don't want to go back to school because I do not want to spend a fortune learning stuff which I can learn myself using free resources.
I don't want to go back to school because I would rather create online/social credentials than get a school certificate.
I don't want to go back to school because I want to demonstrate that a person can not only get knowledge but also credentials if they engage in disciplined self-study and leave learning trails on the Internet.

So this time I am doing a DIY (Do-It-Yourself) masters in Computer Science. so I can refresh things I have forgotten and learn new technologies and concepts which have gained importance in recent times, in an organized way.

I did a Masters in Computer Science more than a decade back. Since then, Internet, communication technologies, and social networking, have made it possible for someone to do a similar program all by themselves, using open courseware, and social learning.

By doing this program, I am not only planning to enhance my own knowledge, but am also hoping to show how one can get a Master's education worth of knowledge, and credentials, by self learning, and without spending a fortune. Here's a very brief statement of purpose.

My DIY Learning Process:

I have created a learning plan which outlines at a high level the topics I want to learn. I will study one or two topics at a time, and at a manageable pace, given other work commitments.

Once the topics to learn have been identified, I will identify learning resources, forums, and mentors for that topic.

A very basic study plan is to study the material, and make notes of my understanding, as well as questions and thoughts, I get in my mind as I am learning. I will make these notes available on a special blog. This blog will serve as a personal knowledge base (I can refer to it in the future), as well as a learning trail (for proof of study and understanding).

All the homework I do while I am taking a course will be made available in the public domain. I will either post it on my blog, or if the homework involves coding, on a public open source repository such as Github. I will also do one or more projects to practice the entire body of knowledge as a whole and publish that too in the public domain.

I will also create presentations of what I learn, and make them available in the public domain.

I will connect with mentors who are experts in the topic I am learning. Depending on their time availability I will request them to help me identify gaps in my understanding, and validate my knowledge.

When I have doubts, I will ask questions on Internet forums. If my questions are not answered satisfactorily on the forums, I will refer them to my mentors.

Please visit my learning plan for further details.

Establishing Credentials:

A person can have several reasons for learning. One is for the knowledge (either for the joy of knowing something, or for more practical application of the knowledge), and another for establishing credentials, so someone else may entrust us with work which requires such knowledge. A self learned person may have the knowledge, but may lack credentials to prove it. It is also possible for a person studying in a silo to think he has grokked what he just learned, when in reality he may not have understood the matter properly. To be able to provide proof of knowledge as well as to validate my learning with other practitioners, I plan to engage in what can be loosely classified as social learning.

I will do the following to document my learning and to engage with the community of practitioners, in the hope of validating my knowledge and establishing credentials for what I learn:
  • Answer questions on forums
  • Blog my study notes, and clearly articulate my takeaway from all the lectures I view, or text I read
  • Create presentations and post them on YOUTube, or other video sharing services
  • Publish homework on open source code repositories such as GitHub, etc
  • Request my mentors to quiz me to help me find gaps in my understanding of a topic. I will publish the quiz as audio/video and request the mentor to post their feedback in the public domain
  • Take quizzes and tests wherever possible and economical
So I am leaving these learning crumbs on various places on the Internet, but I need something to bring everything together. Something which can serve as the focal point, or a lens into all my learning. I created a wiki site to serve as the focal point.

Have you been meaning to learn something yourself? Perhaps you can do your own DIY course in whatever interests you. Here is a lens into my DIY experiment.

Wednesday, May 12, 2010

Android apps can run in parrallel (video)

This is the seventh in a series of videos published by Google on Android programming.

In this video the speaker starts an app to track photos his buddy publishes on Flickr. He can keep this application running in the background, so while he is browsing or checking his email, this application will constantly track the photos and notify him if a new photo has been uploaded.

Tuesday, May 11, 2010

Adndroid apps don't have any borders (video)

This is the sixth in a series of videos provided by Google on Android programming.

00:00 Introduction
00:32 Zebra crossing application
01:35 Library app
02:14 A GeoSocial app which combines user GPS, user created photos, and Compass

My Takeaway:
Android applications do not have any borders. They can use data from other applicatiosns, they can use hardware available on the device such as camera, compass, GPS, accelerometer, etc, and they can also use content from the web API's.

The first application called Zebra Crossing is an application which can pull information from bar codes and QR codes. So if you photograph a product's bar code and give it to the application, it will pull up information about the book from the web. If on the other hand you give it a QR code from the back of a business card, it can extract contact information about that person which you can use to call or send email. The Zebra Crossing app also publishes intents, to allow other applications to communicate with it.

The second application is a library application which uses the "Zebra Crossing" app to add books to your personal library.

The third application, which I found to e really cool is an application which mashes together GPS, photographs, and the Compass to create a GeoSocial application. So here's what it does. Imagine you are on a vacation and want to locate cool things around where you are. You can use this application to find out all the photographs people have taken around that locality. If you find something you like, you can actually locate that thing (which was photographed) on a map and also use the Compass to get there. Cool isn't it?

Monday, May 10, 2010

How to embed the web in Android apps (video)

This is the fifth in a series of videos provided by Google on Android programming.

This video shows how we can use the WebView to create applications that render HTML pages. Along with being able to embed HTML and Javascript web pages in apps, the most exciting thing I found in this view was the fact that the Javascript in a webpage can communicate with Java code and Java code can communicate with javascript. This is really cool.

Sunday, May 09, 2010

Android programming - All applications are created equal (video)

This is the fourth video in the series of videos provided by Google on Android programming.

00:25 Dianes ringtone app
01:08 App to create a shortcut to anything on the system
01:36 Replacing the home screen (End at 02:17)

My Takeaway:
This video shows that all applications created on the Android platform are equal. Even though I have not done any iPhone development, I believe applications written for iPhone do not have access to everything the way Android applications do. So this is a positive for Android development.

In this video we see how to a custom application can change the ringtone on an Android phone, how another application can create shortcuts to anything on the phone, and how we can also change the 'Home' application.