Sunday, October 04, 2009

Some thoughts on redesigning education

Some time back I read a blog post on redesigning education. It asked some very good questions. Stuff which I had been thinking of myself. I left my thoughts on the blog, but I would also like to start a conversation around these ideas with those who read this blog as well.

I would like to know what other people think of the issue of redesigning (college) education.

I have often thought about how college education can be improved. To answer this question, we first have to ask a very basic question. What is the purpose of education?

To me, we need education for 3 things:

  1. To learn more about the world around us
  2. To lead positive constructive lives
  3. To earn a good living / fulfill our ambitions

I think education has to a large extent evolved to fulfill #3 (with a bias towards earning a comfortable living). The semester system, along with multiple choice tests, and grading, has made our education system into an assembly line. Students are pushed into the assembly line, given classes, administered tests, branded with a grade and pushed out of the assembly line, into the workforce.

Even though this is not the best way to teach, this system has for the most part worked till now. I think there are several reasons.

1. For many of students, getting an education is simply a means to gain employable skills (I do not say this in a negative way).
2. A university has to provide these skills in a time and cost efficient way to the students. It has to provide some branding which will be valuable to students, and make it easy for employers to spot the talent they want to recruit.

In the last few years I have taught programming classes at a local college. In one semester, I tried to move away from the regular grading mechanism. I wanted to focus on deep learning of programming skills, sharing and brainstorming ideas, code reviews, collaborating with the community of practice, etc. However, students still needed to be graded, so I had to come up with a way to fulfill that requirement also. Without going into details I will simply say, in that semester, I put in an effort which was orders of magnitude greater than the regular effort (because I had to fulfill the grading requirement). Students learned a lot in that semester, and I too enjoyed running that course, but towards the end of the semester, I was tremendously stressed out managing the course, and my regular programming work.

So even though the current education system is sub-optimal, I do not think it is because of evil intentions. It is possibly the most practical mechanism that has emerged over time.

However, many things have changed in the past few decades. The shelf life of knowledge has greatly reduced. The Internet has made it possible to share information, and collaborate with people at a distance. Many businesses have started valuing knowledge over degrees (this is at least true in software, and probably in media, advertising, etc as well). The read-write web has made it possible for students to easily create a digital portfolio, and to establish alternate credentials. So all the things a student needs for learning (information, interactions, guidance, review and feedback, and credentials) can be put together in a non traditional way. I stress so much on non traditional because the traditional education system excludes many learners, and this I feel can be changed.

So here is the ideal education system according to me:

A student enters the education system with some goals, and connects with mentors who will help her in fulfilling those goals. These mentors may be traditional teachers who work for a university, or may be employees of an organization, senior practitioners who choose to help on the Internet, friends, family, or people who have retired from the workforce, but would like to share their wisdom and help others.

With the help of mentors, the student defines learning goals, and identifies resources. These resources could be traditional classes, books, or digital material (videos, text, audio...).

With these materials and a micro mentor network, the student begins her learning process. When they have questions, there are a several resources from their mentor network they can turn to. These students may be part of a traditional classroom in some cases, whereas in other cases they may be part of a local or virtual study group. Think of it as being part of an appropriate group for every course they want to take. There are several tools, both real world as well as digital to enable this.

As the student learns, they leave a digital learning trail. One possible way is using blogs, audio recordings, wikis, contributions on forums, and other digital artifacts. So students blog their assignments and problem sets. They participate on Internet forums asking and answering questions. They may create a podcast (or screencasts) of their assignments and presentations. Maybe some students will be able to do practical work which is similar in nature to the assignments and problem sets.

Senior practitioners, traditional teachers, mentors from the community, help the student understand the strengths and weaknesses in their knowledge. The student subsequently fills in the holes by seeking help from their mentor network, and by revisiting those concepts. Community members endorse a students' understanding of their topic of study. Maybe tests still have a place... I don't know...

When a student knows enough, they can enter the workforce. Proof of their knowledge already exists on the Internet. Some organizations may accept them purely based on their digital portfolio and an interview, while others may expect them to take some tests. Students can prepare for these tests, if they wish to work for that organization.

Students however, do not stop their education after getting employed. The process outlined above continues, but perhaps at a slower pace. Thus even after starting work a person continues to accumulate (non credit) credentials. These credentials may either be continuing education certificates from a university or Internet endorsements from the community of practice.

This to me is the ideal educational scenario. But it will not be without problems.

There will always be the issue of credibility of an online portfolio, and endorsements from random mentors. Is there a process using which we can streamline online credentials and validate the credibility???

There will be issues with self-discipline. College gives a certain structure. Doing it by oneself needs a lot of will-power and discipline. One can easily while away time, thinking they are learning something. Maybe the Pomodoro technique can help here.

But I think this scenario is workable. I like it because it allows students to learn at their own pace (without excluding students who do not have resources to attend traditional colleges), from many mentors, thus gaining knowledge and wisdom from many sources. But most importantly, it allows students to take control of their education, and seek out the best, albeit disparate sources for knowledge.

Here's my own humble effort towards contributing towards this goal -

Monday, August 24, 2009

Fixing a mistake after commiting in Git

So, I ran into an interesting issue while coding today. I made a lot of changes and committed code to my Git repository. Just after committing, I realized that I still had to make some changes to a file and those changes should also have gone in with the previous commit.

Fixing a commit mistake can be done in two ways. Revert the commit, make the changes and then recommit them. This is the preferred way if we have already made the changes public (and thus someone may already have pulled them). However, in my case I am the only one working on this project right now and I had not pushed the changes to GitHub, so I could try the other method.

So here's what I did. I made changes to the file, added it to the index, and committed it thus:

$git commit --amend

Committing code with the --amend switch will cause the index to be committed as part of the previous commit. Git also brings up the editor window, in case we want to change the commit message. This method can also be used to change the commit message without adding any files to the commit.


Sunday, August 23, 2009

Lack of design patterns in Python

While searching for PyCon videos, I came across Joe Gregorio's very good video on (lack of) design patterns in Python. I have also added the video timeline along with some notes I made for myself and my takeaway. Enjoy the video.

[00:00] - Start
[00:15] - People pick tools based on a mythology and not necessarily facts
[02:35] - Python isn't just Java without the compiler
[03:34] - Design patterns are also a sign of weakness in a language
[04:06] - Lack of design patterns in Python (proof of lack)
[06:10] - Patterns are built into Python
[07:00] - Strategy pattern in Python the wrong and right way
[07:36] - The strategy pattern is invisible in languages with first-class functions
[08:07] - Some other language features in Python (first class functions, metaprogramming, iterators, closures)
[09:17] - The iterator pattern (iterators) is also built into Python
[09:36] - The observer pattern is also built into Python
[10:17] - Factory method pattern in Python (
[10:34] - Abstract Factory Pattern
[10:40] - Strategy pattern goes away becaise of first class functions
[11:08] - Drawing some useful conslusions
[12:20] - Drawing light on Python from the perspective of patterns
[12:31] - Thread Pool and Concurrency patterns (should we be talking about language features in Python for concurrency patterns???)
[13:49] - Channels (a model for concurrent processes)
[15:12] - PyCSP (implmentation of csp on top of Python)
[18:42] - Conclusions, summary, and questions
[19:17] - end (video goes on for a few more seconds waiting for questions)

There are very few references to design patterns in Python mailing lists and discussions around Python... design patterns could be a sign of weakness in a language... patterns are built into Python (hence very little discussion)...strategy pattern is invisible in languages with first class functions...

If you have to implement a design pattern, first look at language features and try to determine if any can be used instead of the pattern or at least assist in the implementation of the pattern while programming in that language.

Ask yourself if a language feature would make that pattern part of the language itself.

Concurrency patterns are a rich area to look at.


Monday, August 17, 2009



I took a sabbatical from my consulting work this summer to participate in the iAccelerator program. This is a program for early stage software startups, similar to YCombinator, but the first of it's kind (I believe) in India.

I enjoy teaching programming, I have taught programming classes at a college, done corporate workshops, and more recently have been working on a website for participatory learning. This time I was looking for a different kind of mentoring experience, something which would be free form and fun. After a discussion with Freeman, I decided to go spend the summer at iAccelerator.

In the first few days of the program we had introductions by all the teams where they spoke about their product and vision. This was followed by a session on team building and thinking out of the box, a few legal sessions, sessions on accounting and company law, and several other mentoring sessions. All this is very useful for early stage software stratups, especially when they are founded by techies who are excelent hackers but need some help with legal and business stuff.

iAccelerator had ten very motivated and talented teams. I split my time helping them with technical stuff and working on my participatory learning project.

We worked out of a large hall, converted very tastefully into a working area. Each team had their own space towards the walls, and the centre was furnished with couches, projector, a large whiteboard (where we had presentations and brainstorming sessions) and a wii for fun and entertainment.

Even though everyone had their own work areas, many people chose to work on the couches. Working from the couches was a lot more fun and it gave opportunities for discussions, knowledge exchange, and friendly bantering. The couch culture (as I call this) in retrospect has been a very important part of the cameredierie and the excellent environment we had among us.

A good, positive, and helpful environment is one of the best things in such a program. Whenever anyone was stuck with a problem, or needed help with something new, they would always find someone who had that knowledge and was willing to share it. Everyone had their unique strengths, and collectively we all had a lot of talent which flowed freely. Entrepreneurship is also often a journey of highs and lows. There are times when you feel like your product will take over the world, and then there are times which can be described as not so pleasant. Being in a good environment with friends, means that you are not in this alone. If someone is down he can be sure that his buddies will pull him up.

One team found their first few customers within the group itself, and another team made a kick-ass software for internal communication which had passionate users from day one.

Ever so often we would have mentors, and successful entrepreneurs come talk to us about various things related to business and startups. Many guests spoke about their journey as entrepreneurs, their successes, failures, and lessons learned. Besides these sessions, we were also invited for many talks given by various mentors to the IIM students. This was a benefit of being on the IIM-A campus. Several mentors also worked one on one with the teams to help them with their vision and business plans.

Since we all had technical skills, internally we did several TechTalks on topics such as security, Amazon EC2 & cloud computing, designing with GIMP, JQuery, version control with SVN and Git, and many more. Sometimes we played tech videos from the Internet and had discussions around them. These sessions were a lot of fun and we all learned from them.

Since Freeman has successfully created and exited a startup in the past, he had many insights about how to run a startup. He shared his knowledge and insights with everyone, helping with business plans, vision, fund raising pitches, and many other nitty gritties of creating startups.

However, it was not just work all the time. We had a great time watching MTV while eating breakfast in the college mess. I can now tell you that watching MTV is far better than reading the morning newspaper. We played ping pong (Freeman and me had some really fun and intense ping pong sessions) ... we played cricket... on weekends we watched movies on the office projector. We also had a lot of diverse talent in the group. Jeevan Ram rocked everyone with his dancing and Ghazal made us laugh with his mimicry.

It's amazing how well we all got to know each other in just a few months. Thinking back, this was the best time I have had after graduating. Most importantly, I cannot stress enough the fact that I made really good friends, and I am very grateful for the wonderful time we all spent together.

This year's iA program will end in a few weeks. I wish all the teams lot of success and good luck.

Friday, August 07, 2009

Custom JSON Encoder in Django

I have been messing around with how to display date formats (in questions and answers) for my web based learning site from the past couple days. After trying various things, I settled for what seems to be a web 2.0 standard for displaying dates - displaying dates as '2 days 4 hours ago' instead of the actual date and time '4th August, 2009 5:50 PM'. I like this because I do not have to deal with any browser localization issues. Everything is a delta between the UTC time a question was asked and the current UTC time (btw if you are working with datetime in Python, do read this blog post).

In my Django based application, I use JQuery to get questions and answers for forums on every topic page. The application sends back JSON to the Javascript functions which display the questions and answers from the JSON objects.

The DjangoJSONEncoder (scroll to the end of this module) provided by Django serializes dates in a specific format. I wanted to change this so that a date would be serialized as '2 days ago'.

A bit of Googling bought me to Jessy's very nice blog post, which explained very nicely how to create a custom Encoder to be used for serializing objects in Python. This seemed like a good idea, so I wrote my custom encoder to change the way dates are displayed;

class Web20DatesEncoder(DjangoJSONEncoder):
SECONDS_IN_HR = 60 * 60
def default(self, object):
if isinstance(object, datetime.datetime):
delta = - object
return self.get_delta_as_string(delta)
return super(Web20DatesEncoder, self).default(object)

def get_delta_as_string(self, delta):
days = delta.days
seconds = delta.seconds
hours = 0
minutes = 0
hours = seconds / self.SECONDS_IN_HR
seconds = seconds % self.SECONDS_IN_HR
minutes = seconds / self.SECONDS_IN_MIN
seconds = seconds % self.SECONDS_IN_MIN
delta_str = ''
if days > 0:
delta_str += str(days) + ' days, '
if hours > 0:
delta_str += str(hours) + ' hours, '
if days <= 0 and seconds > 0:
delta_str += str(seconds) + ' secs, '
return delta_str

However, I could not figure out how to hook this class into the call chain so that it is used when my QuerySet is serialized as such;

res = serializers.serialize(”json”, questions)

One solution which came up was to use the 'cls' parameter in the call to serialize.

res = serializers.serialize(”json”, questions, cls=Web20DatesEncoder)

However, when I ran my code, there was an error:

dump() got multiple values for keyword argument ‘cls’

Why do we get this error? Maybe (I still do understand Python well enough to be sure) because, the end_serialization method in django.core.Serializers.json.Serializer has the following line:

simplejson.dump(self.objects,, cls=DjangoJSONEncoder, **self.options)

As you can see it specifically passes cls=DjangoJSONEncoder to simplejson.dump(...), and this is the reason why we get the above error message.

Now I do not know what is the right way to resolve this issue, but here's how I did it. I created a subclass of JSONSerializer, such that my subclass used my custom encoder.

from django.core.serializers.json import Serializer as JSONSerializer

class DateModifyingJSONSerializer(JSONSerializer):
def end_serialization(self):
self.options.pop('stream', None)
self.options.pop('fields', None)
simplejson.dump(self.objects,, cls=Web20DatesEncoder, **self.options)

and the client code which earlier instantiated JSONSerializer now uses DateModifyingJSONSerializer.

#Here is the code snippet which uses the above class
json_serializer = DateModifyingJSONSerializer()
res = json_serializer.serialize(query_set)

#instead of old code shown below
#res = json_serializer.serialize(query_set)

I hope you find this solution useful. Please leave a comment if it helped you, or if you know a better way to perform this task.

Thursday, July 23, 2009

Git supports commiting specific parts of a file

While working on my web based learning project, I made a lot of changes to a Django view file without committing them. When I actually decided to commit, I realized that the changes in my view file should go into separate commits. I knew it was possible to commit parts of a file in Git, but I was not sure exactly how it could be done.

I asked the question on StackOverflow and got an answer within minutes :-)

Anyways here's how I did it. I had 3 files in my index:

The first 2 files were fine, but I did not want to commit the entire view file with the first two, I wanted only one change in that file to go in this commit.

Since I had already added the view file to the index, I first had to unstage it with:

git reset app/courses/

Then I did an interactive add to add only one hunk from the view file:

git add --patch apps/courses/

Running this command showed me the first hunk from the file and an option asking me if I wanted ti stage that hunk. I decided I did not want the first hunk, then it showed me the second hunk. This was the change I wanted, so I entered 'y' at the option and proceeded to enter 'n' for all the remaining hunks.

After this I wanted to ensure that the right hunk from the file was staged to be committed, so I typed:

git diff --color --cached apps/courses/

This showed me all the changed that were staged. After ensuring that the changes I wanted were the ones staged, I commited them with:

git commit

Saturday, July 11, 2009

JQuery selectors and future elements

I am using JQuery for the AJAX and dynamic aspects of my web based learning platform.

I like the fact that JQuery allows us to use CSS type selectors to select elements on the page to either manipulate them, add event handlers, or a host of other things.

So, the code below intercepts clicks on all links and dynamically adds a new link when a link is clicked.

<link rel="stylesheet" type="text/css" href="/site-media/al/style.css" />
<script type="text/javascript" src="./jquery-1.3.2.min.js"></script>

<script type="text/javascript">
$(document).ready(function() {
$("a").click(function() {
$("ul#questions").append("<li><a href='#'>click me</a></li>");
<ul id="questions">
<li><a href="#">Click Me: 0</a></li>
<textarea style="Width: 400px; Height: 200px" name="textarea" cols=20>

There is a problem with this code. Everytime you click on the link called "Click Me: 0" a new link is added, but if you click on the new link, then nothing happens. So, why does JQuery not work with dynamically added elements?

The reason is because the following call

$("a").click(function() {

only binds to the anchor elements present on the webpage when the $(document).ready() function is called. It will not bind to future elements. To bind to future elements as well, we have to use the live() function.

Try using this code to bind to the "a" elements and it will work.

$("a").live("click", function() {
$("div#questions").append("click me");

Thursday, July 02, 2009

Refreshing iptables

The last time I played around with IPTables was about 6 - 7 years back. I have been working with a Linux box again from the last few months and I am absolutely thrilled about it. Nevertheless, I have also been meaning to set up a simple IPTables firewall on my machine.

Since I had forgotten all my IPTables concepts, I decided to hunt the Internet for some good articles. I found some really nice resources to refresh my memory as well as learn new things.

Here is a nice 3 part video series from Linux Journal.
  1. iptables part 1
  2. iptables part 2
  3. iptables part 3
The entire series takes less than a half hour and is a good refresher or introduction to IPTables.

Here is a nice picture which explains how a packet is routed through various chains in the ip tables. If you are looking for a quick refresher, this might help you out.

If you want more details, here is a tutorial, and yet another tutorial, and Netfilter's excellent documentation on IPTables.

So now there is no excuse for not setting up a little firewall on your Linux box.

Monday, June 22, 2009

Inheritance vs. composition depending on how much is same and how much differs

I am reading the excellent Django book right now. In the 4th chapter on Django templates, there is an example of includes and inheritance in Django templates.

Without going into details about Django templates, the include is very similar to composition where we can include the text of another template for evaluation. Inheritance in Django templates works in a way similar to object inheritance. Django templates can specify certain blocks which can be redefined in subtemplates. The subtemplates use the rest of the parent template as is.

Now we have all learned that inheritance is used when we have a is-a relationship between classes, and composition is used when we have a contains-a relationship. This is absolutely right, but while reading about Django templates, I just realized another pattern in these relationships. This is really simple and perhaps many of you may have already have had this insight...

We use inheritance when we want to allow reuse of the bulk of one object in other objects such that it can change a few things as per it's requirements. So in a way we have a template which we want to allow reuse of in the greater part and modification of in the smaller part. On the other hand we use composition when we want to reuse another object in this object. So we have a template in which we want to plug some holes in functionality.

Does this make sense?

Monday, June 15, 2009

Slashy strings in Groovy

Groovy has done a great job of enhancing Java Strings. It offers a lot of features like String interpolation with GStrings, triple quoted Strings, multi-line Strings, and slashy Strings.

In this post I will talk about slashy Strings in Groovy. But before doing that let us see how we represent a regular expression in Java. Let's say I have a list of fully qualified file names and I want to match all files in my 'c:\tmp' directory.

I would create a String to represent my regex in Java like this:
String exp = "C:\\\\tmp\\\\.*"

The four '\' are needed because '\' is a meta character in regular expressions, so we need to represent a '\' as a '\\'. Because a '\' is used for escaping special characters in Java, we need to represent '\\' as '\\\\'. Wow doesn't this look cumbersome. Well, this is not just one case. Regular expressions make use of the '\' character for special classes. Everytime we want to use a '\w' or a '\d' or something like that we will have to use '\\w' and '\\d' instead.

Slashy Strings in Groovy give us a way around this by allowing us to represent regular expressions just like they would be represented without having to escape the '\' in Java.

Using them we can write

def file = /C:\\tmp\\.*/

Notice that slashy Strings have to be surrounded by forward slashes, and are mostly used when Strings need to represent regular expressions.

Saturday, May 30, 2009

Testing Groovy domain classes

If you are trying to test Grails domain class constraints by putting your unit test cases in the 'test/unit' directory, then your tests will fail because the domain objects will not have the 'valdate' method. This can be resolved in two ways:
  1. Place the test cases inside test/integration (which will slow things down)
  2. Use the method 'mockForConstraintsTests(Trail)' to create mock method in your domain class and continue writing your test cases in 'test/unit'

What follows is some example code around this finding.

I am working on a Groovy on Grails project for a website to help programmers keep up and refresh their skills.

I started with some domain classes and then moved on to write some unit tests. When we create a Grails project using
grails create-app
it creates several directories, one of which is a directory called 'test' for holding unit tests. This directory contains two directories, 'unit', and 'integration' for unit and integration tests respectively.

Now begins an interesting journey with writing unit tests in Groovy. I wanted to write tests for my domain classes. A simple domain class like the one I have below can contain only properties and constraints for those properties. This is what Trail.groovy looks like.

class Trail {
String shortName
String name
String description

def hasMany = [learningObjects:LearningObject]

static constraints = {
shortName(maxSize:6, blank:false)
name(maxSize:75, blank:false)
description(maxSize:2048, blank:false)

I wanted to start writing unit tests to test the constraints. When Grails creates a domain class, it injects several methods in it at runtime , one of which is a method called 'validate'. This method is called before the domain object is saved and it will return a false if the domain object has violated any constraints. So, I created a simple unit test to test the constraints of Trail.groovy

This is what my initial test case looked like.

import grails.test.*

class TrailTests extends GrailsUnitTestCase {

def trail

protected void setUp() {
trail = new Trail(shortName:'sname',
name:'Java 101',
description:'This is a basic Java course')

protected void tearDown() {

void testShortNameConstraints() {
trail.shortName = 'thisisaverylongname'
trail.shortName = ''

//... further methods not shown

When I ran my tests using
grails test-app

(See how well Grails integrated testing), I got a bunch of errors that told me that the 'validate' method was not found. If you enjoy looking at stack traces... feast your eyes:

Testcase: testShortNameConstraints took 0.159 sec
Caused an ERROR
No signature of method: Trail.validate() is applicable for argument types: () values: []
groovy.lang.MissingMethodException: No signature of method: Trail.validate() is applicable for argument types: () values: []
at TrailTests.testShortNameConstraints(TrailTests.groovy:20)
at _GrailsTest_groovy$_run_closure4.doCall(_GrailsTest_groovy:203)
at _GrailsTest_groovy$
at _GrailsTest_groovy$_run_closure2.doCall(_GrailsTest_groovy:147)
at _GrailsTest_groovy$_run_closure1_closure19.doCall(_GrailsTest_groovy:113)
at _GrailsTest_groovy$_run_closure1.doCall(_GrailsTest_groovy:96)
at TestApp$_run_closure1.doCall(TestApp.groovy:66)
at gant.Gant$_dispatch_closure4.doCall(Gant.groovy:324)
at gant.Gant$_dispatch_closure6.doCall(Gant.groovy:334)
at gant.Gant$_dispatch_closure6.doCall(Gant.groovy)
at gant.Gant.withBuildListeners(Gant.groovy:344)
at gant.Gant.this$2$withBuildListeners(Gant.groovy)
at gant.Gant$this$2$withBuildListeners.callCurrent(Unknown Source)
at gant.Gant.dispatch(Gant.groovy:334)
at gant.Gant.this$2$dispatch(Gant.groovy)
at gant.Gant.invokeMethod(Gant.groovy)
at gant.Gant.processTargets(Gant.groovy:495)
at gant.Gant.processTargets(Gant.groovy:480)

After some Googling I realized that the 'validate' method is injected into domain objects by the Grails framework, and therefore it does not exist when we instantiate the domain object from test code. I later found that if a test is placed inside 'test/integration' then the test cases are created as the objects would have been created by the Grails framework. So the domain objects would have the 'validate' method. But I also read in another place (I cannot remember where), that if possible it is k better to mock these special methods instead of putting tests in the integration directory, since these tests take longer to instantiate and can slow down the overral running time. And I found in yet another place that there exists a method called 'mockForConstraintsTests(Trail)' which will inject the 'validate' method along with some other methods. So, I added this mock method to my setup after which I was able to run my tests successfully.

protected void setUp() {
trail = new Trail(shortName:'sname',
name:'Java 101',
description:'This is a basic Java course')

The rest of the class is the same.

So to summarize, if you are trying to test constraints in Grails domain classes, you have two options:
  1. Place the test cases inside test/integration (which will slow things down)
  2. Use the method 'mockForConstraintsTests(Trail)' to create mock method in your domain class and continue writing your test cases in 'test/unit'
I will post more such episodes of how Google helped me resolve issues I faced while developing :-)

Tuesday, May 26, 2009

Groovy on Grails

Grails is a web application framework build on top of well known languages, frameworks and libraries. It uses the Groovy programming language which gives it power because of it's dynamic nature. Under the hoods, Grails uses Spring framework(for MVC), Hibernate (for OR mapping), Quartz (for scheduling), Log4J (for logging), JUnit for unit testing, and Canoo Webtest for functional testing.

We already have a Gazillion frameworks, do we need Grails ?

Typically web development in Java is a very long and arduous process. When a web project is started, many configuration files have to be set up after which the project development begins. Even then we have to create domain objects, basic controller functionality, the model, stylesheets, views, and other aspects of business logic. Grails makes things easier by allowing us to do all of these things rapidly and in fewer lines of code. So how does Grails allow us to do all this with fewer lines of code? Grails uses convention over configuration.

If you have developed a JEE app with Struts, you probably remember configuring struts-config.xml to map ActionBeans, Actions, Action classes, Global forwards, and so on. A typical development workflow with Struts is as follows:

  1. Create struts-config
  2. Create ActionBean classes and write their validators
  3. Create Action classes and delegate persistence and other things to model classes
  4. Create a Hibernate configuration file
  5. Create views with JSP's using Taglibs
  6. Write unit tests (these two are not really the last two... remember TDD)
  7. Write functional tests

Whew.... isn't that a lot of work to do? In Java I have always felt like there is just a lot of coding and configuration that needs to be done to make even the smallest web application.

Grails takes away a lot of this pain. This means we do not have to specify and code the smallest of things if we follow certain conventions. We however have the ability to break the conventions if we want to (in case of integrating with legacy databases, etc) by adding configuration.

So, for example lets assume we want to make a simple application to manage a music collection. We start with creating a Grails app with a grails command, after which we create a domain class. Thats it. Grails will scaffold the application for us, meaning it will create the entire infrastructure to give basic CRUD functionality with our domain object (this includes the controller, and views for adding, deleting, updating, and modifying data). So within minutes we have something we can use. Off course in most situations we will not want something as simple as this. We will have to extend the application in various ways. But I suspect, even after extending the application we will still have saved some time by using Grails, and there is another benefit. Because Grails gives us something really quickly, we can use the Agile approach where we quick feedback on what we are trying to accomplish, and every feedback loop actually gives us something that is usable and .

Because Grails is written in Groovy which is a JVM language, we can take advantage of all the Java libraries that we have become familiar with and also gain benefits from the JVM's optimizations.

A few other features of Grails which I have noticed and like are:

  1. Ability to upgrade to a newer version of Grails with a single command
  2. Automated creating of test classes (thus prodding you to write unit tests)
  3. Ability to use different databases for testing, development and deployment
  4. Command for creating domain objects, and controllers
  5. Inbuilt UI templating
  6. Host of third party plugins that extend the Grails framework
  7. Extremely simple mechanism to create Tag libraries
These are some of the features that I have read about, but I am sure there are many more.

If you build web applications in Java, Grails is definitely be something you should look into.

More about Grails coming up in future posts.

Monday, May 18, 2009

Internet and new media for teaching and learning

Today, Freeman and I did a video talk for a faculty development program at SCIT on how teachers can use the Internet and New Media for teaching and learning.

Some useful links for those who are new to this medium.

Google Groups is a good and simple way to start a mailing list.

Google Reader is a good blog reader.

For hosting a blog, I recommend either Blogger or Wordpress to start with. If you want more control, you can download the Wordpress software and install it in your own server.

ITunes is a good client software for listening to podcasts. But ITunes works only on Windows and Mac. For those running Linux, the Miro Player is a good option for video podcasts, and Amarok or JPodder for audio podcasts.

ITunes university has several podcasts hosted by universities worldwide.

If you want to create a podcast, you can use Audacity for recording them and Odeo for free hosting.

Those interested in Twitter can head straight to and here is an article describing how Twitter was used by a professor in the classroom.

To host your bookmarks on the net, you can use Delicious.

Free wiki hosting is available at PBWiki. If you want to install your own wiki, then download Mediawiki, or Twiki.

Moodle is the course management system that we spoke about.

Some educators who have inspired me by sharing their knowledge are: Stephen Downes, Konrad Glogowski, and Leigh Blackall

Friday, May 15, 2009

CompSci videos at Stanford

I came across some very interesting videos published by Stanford. These videos are recordings of some computer science courses offered at Stanford. I still have not had a chance to see all of them, but I am sure they will be very informative.

  • Programming methodology series at Stanford (lectures 23/24 may be useful for developers who already know programming) - video
  • Programming abstractions series at Stanford (looks like an interesting lectures. They focus on data structures, implications of using certain data structures, recursion, algorithm analysis like the Big O notation) - video
  • Programming paradigms series at Stanford (Talks about imperative, OO, & functional programming. Also discusses many good low level things like pointers, big/little Endian, representation of data as bytes, and also touches on algorithms, assembly code, etc) - video
  • Machine learning series at Stanford - video
I hope you find these useful. I will share more links as I discover them.

Sunday, April 26, 2009

Some more impressions of Python

List comprehensions: List comprehensions are pretty neat. Before studying Python I did not know what they were. This is a wonderful way to perform some operation on an entire list. Let me explain with an example. Let's say we have a list of numbers and want another list containing the double of all the numbers greater than 5 from the first list. In Java we would have to do something like this:

private static List getDoubleOfMembers(List numbers) {
List result = new ArrayList();
for(int num : numbers) {
if(num > 5) {
return result;

However, in Python we can perform this operation using list comprehension like this:

[n*2 for n in numbers n > 5]

Isn't this nice?

Lambda expressions: Python supports Lambda expressions but it does not support closures like Groovy. I wish Python supported closures.

Unchecked exceptions: The blogosphere has had more than it's share of discussions on checked vs. unchecked exceptions. I will not spam your feed reader with any more on this topic. Suffice to say, I like them. In case you want to read more about this debate, this Developer works article is a good starting point.

Pickle: Python has a module called pickle for object serialization and deserialization. Using pickle we can save an object to a file and then reconstruct it as well. We can also use pickle to transmit objects over Internet sockets. There is C implementation of pickle called cPickle which is many times faster. Python does not seem to use anything to signify that an object can be pickled (like the Serializable interface of Java), however, it does have a notion of picklable and non picklable objects. I am still not sure if Python has the equivalent of SerialVerUID for forward compatibility of serialized objects.

repr: Python has a function called repr() which converts an object into a canonical (stringified... if that is a word :-) ) form.
Call repr(obj) to get the string form of an obejct. The return value can be controlled by implementing the __repr__() function in the class. Typically the following should be true:

eval(repr(obj)) == obj

I like this statement. Joshua Bloch wrote in his book Effective Java, Item 9, that if a class's toString() method is documented, then it should return a string such that we should be able to recreate a similar object from that string using the class's String constructer. The repr() function fulfills that criteria.

Saturday, April 25, 2009

My first impressions of Python for the second time

I had worked a bit in Python many years back. Since then I have forgotten almost everything I learned back then. I think the phrase "Out of sight out of mind" applies perfectly to my mind.

Since the last few days, I have started relearning Python, and this time I am recording my impressions of Python after having come to it from a Java background.

Indentation: Python uses indentation to specify blocks of code, instead of curly braces. I like this, because we anyways indent code to increase readability, so why not achieve two tasks together. Code looks much cleaner without the curly braces. However there may be a little downside. Everyone in the team will have to set up their IDE's in the same way. Things might fall apart if some people use tabs and others use spaces for indentation.

Access modifiers: Python does not have public, private, and protected keywords. Everything is public. However, private members can be specified with a leading single underscore. If we use double leading underscores then Python's runtime will do name mangling and make it difficult to directly access those members. Note that I use the word difficult, and not impossible. Personally I like not having access modifiers. Using convention to specify a member as private is fine in my books, because I have never thought of access modifiers as a security feature. It is rather a feature that prevents well meaning programmers from making mistakes. If there is a simple convention like leading underscores signifying a private member, then well meaning programmers will not access it directly. We do not need to treat developers as children and put fences all around the place.

String interpolation: In a previous blog post I wrote about how Python supports string templating, but not interpolation. This is one feature I wish Python supported.

String and Unicode: Python strings are not Unicode by default. For Unicode strings there is a separate Unicode type. On the other hand in Java all strings are always Unicode. Update: Some friends just informed me that Python 3000 has Unicode strings by default.

Multiple inheritance: Python supports multiple inheritance. Again I am not sure I like this.

Numbers: I tried this code in Python

max64BitInt = 2**64 - 1
bigNum = max64BitInt + 126
print 'This is a big number ', bigNum

and it prints the answer correctly. So Python does not have the 64 bit limit for integers the way Java does. In Groovy, numbers can be BigInteger types and can be arbitrarily large. I do not know if this is true in Python. However what we have is definitely better than Java's support for working with large numbers.

Magic method: Python has magic methods. These methods begin and end with double underscores, like __str__(). Magic methods in Python have special meaning, in that they exist to be called back when a certain procedure is invoked. For instance the __len__() method of object o is invoked when we call the len(o) function. It returns the logical length of the object. This is a nice feature because it allows the system to have certain standard functions like len(), repr(), print(), etc and allow developers to have magic methods in their classes if they want their classes to respond to these standard functions. This is how Python also supports operator overloading, as well as list iteration and indexing.

Procedural and OO programming styles: Even though everything in Python is an object, Python supports both procedural and object oriented programming styles. I believe it is also possible to write functional code in Python. But I do not know enough about it. That is a topic for a future post. I like support for procedural programming, because it makes it easy to create small useful scripts. One of the reasons I could never write those in Java was because I would have to create a class and a main method and also put the script through the compile phase. But then again, Java was never meant to be a scripting language. Python also has globally available functions like len(), print(), repr() etc. For example len(o) takes an object o, (maybe a list or a string), and returns it's length. In Java we would have invoked o.length(). I guess there are pros and cons to both approaches. Java's style is more object oriented, but it does not guarantee uniformity. We use the length attribute to get an array's length and the size() method to get the length of a List. In Python if an object o has a logical length, we can get it by invoking len(o).

I have some more impressions on Python, which I will publish in the next post.

Wednesday, April 22, 2009

String interpolations in different languages

I am learning Groovy and refreshing my Python skills as well. While working on some examples, I realized that the operation of concatenating string to produce a larger string is a very frequently used operation. Very often parts of the string we are creating are already held in variables. A crude way of creating the larger string, is to simply concatenate everything as shown in this simple toString() method of the Person object.

public class Person {
//showing only relevant code
public String toString() {
return "Name: " + name + " salary: " + salary + " address: " + address;

However, this is not very readable. A better way is to use either string interpolation or templating. Java gives us a couple of ways to do templating.

We can either write the toString() method like this to become much more readable:

public String toString() {
return String.format("name: %s salary: %d address: %s", name, salary, address);

Or we can use the MessageFormat class:

return MessageFormat.format("name: {0} salary: {1} address: {2}", name, salary, address);

I personally think the last option is more readable. Moving to Python, it also gives us ways to do templating:

return "name: %s salary: %d address: %s" % (name, salary, address));

There is yet another way:

print "name: %(name)s salary: %(salary)d address: %(address)s" % locals()

Notice that both, Java and Python, support templating but not interpolation. Groovy on the other hand supports interpolation which is far more elegant and concise.

public class Person {
//showing only relevant parts of the code
public String toString() {
return """name: ${name} salary: ${salary} address: ${address}"""

Sunday, April 19, 2009

Create mock objects with metaprogramming

We usually create mock objects in our test suites for those classes which communicate with some external entity that we cannot (or do not care to) setup in our test cases. However, mocking is possible only when we have used dependency injection properly. If we have classes that instantiate their own dependencies then we cannot mock the dependencies. Sometimes our mock solutions may also be time intensive at runtime. I will explain this later in the post.

One solution (the correct one) is to refactor the code and use proper dependency injection to allow classes be be mocked. However, when we have a huge code base of non DI code, which cannot all be changed at a time, then we just have to make so with fewer unit tests. But all is not lost. If we use a dynamic language like Groovy to write the test cases then we can use metaprogramming to change a classes methods at runtime instead of creating mock classes.

I am building a command line Twitter client in Groovy. I have a class called FriendsTimelinePlugin.groovy which periodically fetches content from the friends timeline service in Twitter. There is a class called Net which encapsulates code to make GET and POST reqests. The FriendsTimelimePlugin.groovy class uses Net.GET(url) to invoke Twitter's API to get the XML feed. I obviously did not want to connect to Twitter for the unit test.

I had two options. The first one was to start an instance of Jetty in the in memory mode and get FriendsTimelinePlugin to use a local URL which would talk to my local instance of Jetty instead of Twitter. I could then create handlers in Jetty to mock the reply. This is a good solution, but it would involve additional classes to create the mock handlers for Jetty and it is also time expensive to setup Jetty at runtime. If we have several such unit tests, then it would take enormous time to start and stop Jetty for each unit test.

Below I show an example of how I mocked Net.groovy and FriendsTimelinePlugin.groovy using metaprogramming while creating unit tests.

public void setUp() {
//mock Net.GET(...)
Net.metaClass.static.GET = { url ->
def xml
if(url.contains('since')) {
xml = """<statuses>
//not showing the entire string containing status code
return xml
else {
xml = """<statuses>
//not showing the entire status string
return xml

//change the output PrintWriter in FriendsTimelinePlugin
def mockedOut = new PrintWriter(new StringBufferWriter(_buff))
FriendsTimelinePlugin.metaClass._out = mockedOut
FriendsTimelinePlugin.metaClass.filter = ['user_one', 'user_two']
FriendsTimelinePlugin.metaClass.interval = 500

In Groovy we can change the implementation of a method using the class' ExpandoMetaClass. This is done by associating a closure with the method to change. In my code, FriendsTimelinePlugin.groovy invokes the Net.GET(url) method to make a GET request to Twitter. I redefined the GET method at runtime so that it does not do any network communication. Instead it will simply return an xml which otherwise would have been returned after fetching it from Twitter. If you look at the code above you will see a line similar to this

Net.metaClass.static.GET = { url ->
//mock implementation
//directly return XML

This code replaces the actual implementation of the GET method in Net.groovy with the implementation provided in the closure. The metaClass' static property is used because Get is a static method. As you can see the mock implementation returns an XML string directly instead of fetching it from Twitter.

If you look towards the end of the setup method I am also changing _out in FriendsTimelimePlugin. This is the output stream to which it prints the latest twits. In my test case I have redirected it from System.out to an output stream which writes to a StringBuffer whose contents can be verified in the test case.

I realize that if this mechanism of using metaprogramming instead of creating mock objects is used injudiciously then it could lead to really bad code. Especially dangerous is mocking select methods of a class without understanding how and where it is used in the entire codebase. This can lead to undesired side effects and a totally unmanageable test suite. However, if used carefully this mechanism can not only make it possible to write test cases for classes, which were earlier not possible, it can also reduce the time for running a test suite because we will not have to incur the cost of starting/stopping test servers.

Friday, April 17, 2009

Eclipse's JUnit test runner forcibly terminates threads

Today I was writing a unit test which instantiated a class, which started a thread to do some stuff. I was hoping to test the side effect of that thread. However, to my surprise, I realized that Eclipse would forcibly terminate the test after the test method exited, even before the thread completed it's work.

I wrote a simple test to show this.

import junit.framework.TestCase;
import junit.textui.TestRunner;

* This class shows that Eclipse's JUnit test case runner will forcibly terminate
* the JVM and all running threads
public class ThreadTest extends TestCase {

static Runnable run = new Runnable() {
public void run() {
int cnt = 0;
while(true) System.out.println(cnt++);

public void testThread() {
Thread t = new Thread(run);

public static void main(String args[]) {
TestRunner runner = new TestRunner();;

This class basically starts a Thread which prints an increasing count in an infinite while loop. When I run this class as a unit test from Eclipse, the count is printed till about 1890 and then the thread terminates. However, when I run the main method (which runs the unit test using JUnit's TestRunner) then the test runs as expected.

I actually added the main method to eliminate the doubt that JUnit might be terminating the thread. I know this is unlikely, but I just wanted to be sure.

I just find it strange that this is the first time I have run into this issue. Especially after working for so many years with Eclipse. I find it hard to believe that I have never written a test case which started a thread and had to wait for it to do something. Or maybe this is a bug in the latest version of Eclipse (3.4.2).

For now I have worked around this issue by putting the main thread to sleep till a property indicating that that the thread has completed it's work is set.

Wednesday, April 15, 2009

Injecting methods in Groovy with ExpandoMetaClass

I am writing a Twitter client in Groovy. One of the things I want to do is create a thread which will poll Twitter for new updates at regular intervals, and go to sleep when it is not doing anything.

The way we write such code is to invoke the sleep method with the number of milliseconds to sleep. To sleep for 2 minutes, I would write:

Thread.sleep(1000 * 60 * 2)

This code is fine, but not very readable. I would have preferred to write something like this:


This is not possible in Java, but it is in Groovy. In Groovy we can inject methods into a class at runtime. This is done using the class' ExpandoMetaClass, as the example below shows.

Integer.metaClass.seconds = {return delegate * 1000}
Integer.metaClass.minutes = {return delegate * 1000 * 60}
Integer.metaClass.hours = {return delegate * 1000 * 60 * 60}
assert 3.seconds() == 1000 * 3
assert 5.minutes() == 1000 * 60 * 5
assert 4.hours() == 1000 * 60 * 60 * 4

The above code gets Integer's metaclass (the ExpandoMetaClass) and adds methods by setting certain properties to closures. Here the name of the property is the name of the method we want to inject, and the closure represents the code that will be invoked when the method is called. Finally the 'delegate' in the closure in this context refers to the object on which we call the method.

The ExpanoMetaClass also has a DSL to make it simpler to add methods. The example above can be rewritten as follows, thus saving a few keystrokes.

Integer.metaClass {
seconds {return delegate * 1000}
minutes {return delegate * 1000 * 60}
hours {return delegate * 1000 * 60 * 60}

Sunday, April 12, 2009

What is metaprogramming?

The word metaprogramming keeps coming up everywhere in stuff I read nowadays. Even though I sort of knew what it meant I wasn't really sure. So I decided to ask the question 'what exactly is metaprogramming' on SO.

I got some really nice answers and links to articles. I have written this blog post as an effort to summarize what I have learned and to be able to share it with others who might be interested.

According to Wikipedia

Metaprogramming is the writing of computer programs that write or manipulate other programs (or themselves) as their data, or that do part of the work at compile time that would otherwise be done at runtime. In many cases, this allows programmers to get more done in the same amount of time as they would take to write all the code manually, or it gives programs greater flexibility to efficiently handle new situations without recompilation."

The compiler is the simplest example of a metaprogram. It takes as input, code in some language, and creates code for another language. An example of metaprogramming in Java, is the use of generics. Here is a blog post I wrote (although for a different reason) some time back to show a simple example of how the compiler transforms Java code which has generics.

In languages like C and C++, metaprogramming can be done by the preprocessor.

In the Groovy programming language, there are two types of metaprogramming techniques: compile time metaprogramming, and runtime metaprogramming.

In compile time metaprogramming, Groovy allows us to hook into the compilation process at various stages, and modify the Abstract Syntax Tree which the compiler works on. (If you are interested, here is an Eclipse plugin to visualize the AST of a Java program)

An example of compile time metaprogramming is the use of the @Singleton annotation in Groovy. Whenever, we implement a Singleton, we always create a private constructor and a static method which gives us an instance of the Singleton. If we have several Singleton classes in our software, we have to implement this pattern for all of them. Using the @Singleton annotation gives us pattern reuse through AST transformations.

Let's say I want to create a Singleton class called MySingleton. In Groovy I can create it as follows without having to write a single line of plumbing code.

@Singleton class MySingleton {
//implementation of the class

During the compilation phase, some custom code will be invoked which will add the private constructor and a static instance() method to the class. Isn't this great? I think AST transformations can take reuse to a totally new level

Runtime metaprogramming can be done by using the Groovy MetaClass or more specifically the MetaObjectProtocol. Using the MetaClass, we can:
  • Add methods to objects at runtime
  • Determine if an object responds to a message or contains a property
  • Respond to calls made on non-existent methods
  • Respond to queries made on non-existent properties
Various builders in Groovy such as the MarkupBuilder and the SwingBuilder use the MetaObjectProtocol to achieve their magic. Here is some very simple code which uses SwingBuilder to create a simple GUI.

def swing = new SwingBuilder()
def gui = swing.frame(title:'GroovyTwitterClient',
defaultCloseOperation:WindowConstants.EXIT_ON_CLOSE) {
scrollPane {
tabbedPane = widget(new JTabbedPane())

What you see in the code above is a very simple DSL to create Swing GUI's. The code creates a JFrame, set's it's title, size, and defaultCloseOperation. Then it creates a JScrollPane and puts it in the JFrame. Finally it creates a JTabbedPane and puts it in the JScrollPane. If you look at the code carefully, you will realize that we call the method 'frame' on an instance of SwingBuilder and give it certain parameters and a closure. In the closure we call a method 'scrollPane' and give it another closure, and so on. Here each method call actually results in the creation of a Swing component. The parameters of the call are the properties which will be set on that component and the closure represents the component(s) to be added to that component. However, the most interesting thins is yet to come. The SwingBuilder class does not even have methods called 'frame', 'scollPane', and 'tabbedPane'. So then how does this work? The SwingBuilder class overrides invokeMethod(...) which is invoked for every single method call. So, when the 'frame' method is invoked on SwingBuilder, it goes through the MetaObjectProtocol to invokeMethod(...) . Here the call is intercepted and a swing component is created based on the name of the method.

This was just an overview. There is much more to metaprogramming and I will write more about it in future posts.

Thursday, April 09, 2009

Closures in lieu of inner classes in Groovy

Groovy does not support inner classes, at least not in version 1.6 . Having done Java coding for many years, I have become accustomed to using inner classes to encapsulate small related functionality, especially when it is needed for callbacks or some sort of configuration.

Now that I am learning Groovy, and writing a Twitter client, I needed to use inner classes, only to find out that Groovy does not support them. However, we can use closures and maps to simulate inner classes in Groovy. Before I explain this technique, I must say that it is simply a workaround, and at least in my opinion a not-so-good solution.

For this example I will show how I implemented a Jetty Handler using closures instead of an anonymous inner class. Jetty is a Java web server which supports Servlets and JSP's. I often use Jetty in my unit tests, because it can be started in-memory from an API call. Jetty can be configured in code with handlers, so that it will use these handlers to service a web request.

In the code below, I have created a simple unit test which makes a GET request and tests that the response is "Hello World". Offcourse this is a very trivial test, and would not be useful in production.

Here is the setUp() method from the test case, HelloWorldTest.groovy . I have not showed the entire test case for the sake of brevity.

Server _server //Jetty Server


public void setUp() {
//configure Jetty Server

_server = new Server(8080)


We set the handler to mock HttpRequests in the call _server.setHandler() . The code below shows how JettyHandlerFactory.groovy uses closures instead of inner classes to implement the handler.

import org.mortbay.jetty.handler.AbstractHandler;

import javax.servlet.http.HttpServletRequest;

import javax.servlet.http.HttpServletResponse;

public class JettyHandlerFactory{

public static AbstractHandler getHelloWorldHandler() {

//define the closure for a method of the inner class

def handle = { String target,

HttpServletRequest req,

HttpServletResponse res,

int dispatch ->



def builder = new groovy.xml.MarkupBuilder(res.getWriter())

builder.html {

head {}

body('Hello World') {





//define a Map for the method name and closure

def handlerMethods = ['handle' : handle]

return handlerMethods as AbstractHandler



Because Groovy does not support anonymous inner classes, what we had to do was to create a closure called handle. This closure contains the that will be run when the handle method of the handler is invoked. The name of the closure does not necessarily have to be handle. It is names this just to increase readability.

Then we create a Map called handlerMethods to hold closures for all the methods we want to override in AbstractHandler. In this case we want to override just one method; handle. The keys in the Map must be strings which are named to the method names we want to override. The values must be the closures which implement the method. These closures must accept input parameters that the method accepts. The closure above accepts 4 input parameters.

Finally the class is created by converting the Map into a class using the
return handleMethods as AbstractHandler

Tuesday, April 07, 2009

Sharpening the saw with online competitions

Some time back Jeff Atwood wrote about Sharpening the saw on his blog. He defines sharpening the saw as (which is rightfully attributed to Steven Covey):

Sharpening the saw is shorthand for anything you do that isn't programming, necessarily, but (theoretically) makes you a better programmer.

There are several things developers can do to sharpen their saw. One of them is participating in programming competitions. There are several websites which host online programming competitions, and I really like this concept. Participants can work on these competitions at a time of their convenience, and those who do not have "real world" competitions hosted in their cities and towns, can also participate.

Code Chef publishes online competitions which can be solved in any one of 35 programming languages. I like the fact that they give developers freedom to code in their favorite language. Many other online competitions do not give this freedom, and impose a particular programming languages on participants.

All the problems published thus far on Code Chef seem to be mathematical in nature. That's fine and also understandable. Such problems have a definite output and can be easily judged for correctness. It can be a bit difficult to objectively judge programs which are more design oriented.

My math skills have really depleted since high school, so I am looking forward not only to sharpening my saw but also my math skills.

Sunday, April 05, 2009

Groovy beans

A typical Java Bean is a Java class which is serializable, has a public no-arg constructor, private attributes, and getter/setter methods. I'd say this is a lot of stuff for a simple pojo.

Groovy beans did to pojo's what Java could not. They are actually simple. A typical Groovy bean needs to implement and add properties to it. We do not need to add any getters/setters, they are added automatically for us.

Here is a simple Groovy bean.

class Customer implements {
String firstName //will be a private attribute in the bytecode
String lastName //will be a private attribute
def middleInitial //beans can even contain dynamic properties

See.... Groovy beans do not have any getters or setters, and you will also have noticed that the fields are not private. Even though the source does not contain getter/setter methods and private attributes, the bytecode does. When this bean is converted into bytecode all attributes which do not have an access modifier will be treated as properties. What this means is, they will become private attributes and public getter/setter methods will be generated for them.

Now let's create an instance of this bean.

def customer = new Customer()
//Even though this looks like we are accessing public fields
//the bytecode actually invokes the setter methods of these properties
customer.firstName = "John"
customer.lastName = "Gardner"
customer.middleInitial = "U"
println 'Customer's Last Name: ' + customer.lastName

Creating an instance of a Groovy bean is very similar to Java, but assigning values to it's properties seems to be a bit different. When we assign the value "John" to firstName, it seems as if we are assigning a value to a public field. But this is not what happens. Internally Groovy converts this into a call to the setter method for that field. Thus we get the same end result as in Java, but with less verbosity.

We can also use the bean constructor to instantiate and initialize the bean together.

def customer1 = new Customer(firstName: "Bob",
lastName: "Steele",
middleInitial: "Z")

In the above code this is what really happens under the hoods. First the constructor is invoked, followed by calls to the corresponding setter methods.

This is wonderful. Groovy Beans are just like Java Beans, but much simpler and smaller.

Now, you must be thinking, but what if I want to add custom code to the getter or setter? That is simple, simply create your own getter or setter and Groovy will use your code instead of automatically generating it's getter or setter. Your getter/setter can either be public or even protected if you so wish.

But what if we want to create a simple field without getters and setters. How do we prevent Groovy from automatically inserting getter/setters in the bytecode. Again simply create a field and give it an access modifier (like 'private') and Groovy will not auto generate anything for you.

One question that comes to mind, is how do we create package private attributes? Since Groovy 1.6, a package private field can be created by annotating it with @PackageScope

Thursday, April 02, 2009

Control structures in Groovy

To control program flow in Groovy we can use all control structures available in Java, but with additional power.

In Java, branching statements like 'if', and 'while' accept an expression which evaluates to a boolean value. We cannot use an object where a boolean is required. Groovy, on the other hand has the concept of Groovy truth, which coerces non boolean values into boolean values. Thus statement like these are perfectly valid in Groovy.
List list = ['apples']
if(list) {
println 'This list is not empty'
Here the list object evaluates to a boolean value. Any non empty list will evaluate to true. This article explains coercion rules for the Groovy truth.

Groovy truth allows us to use non booleans inside conditionals. It evaluates objects to their (truth) boolean values. With Groovy truth we can use booleans, Strings, GStrings, List, Map, Numbers, and any arbitrary object where a boolean is expected.

Groovy also gives greater power to switch statements. In Java we can only use integers, characters, and Enum constants in switch statements. However, Groovy allows the use of expressions that evaluate to arbitrary objects in switch as well as case statements.


def printGameType(name) {
def boardGames = ['Chess', 'Carrom', 'Monopoly']
def ballGames = ['Soccer', 'Football', 'Basketball']
def ballAndBatGame = ['Baseball', 'Cricket']
switch(name) {
case boardGames:
println """${name} is a board game"""
case ballGames:
println """${name} is a ball game"""
case ballAndBatGame:
println """${name} is a ball and bat game"""
println """Could not determine game type of ${name}"""

The output from running the above program is:

Soccer is a ball game
Cricket is a ball and bat game
Could not determine game type of Cricket
Could not determine game type of Tag

In Groovy, whenever a case expression is evaluated, the isCase() method of the object (which the case expression evaluates to) is invoked with the object provided in the switch statement as a parameter. In the above example, the isCase() method of each of the List objects is invoked with the String provided in the switch statement. If an object does not have an isCase() method then it's equals() method is invoked.

These new features makes Groovy a very expressive language to represent control logic.

Wednesday, April 01, 2009

Groovy Maps

Using Maps in Groovy is much simpler than in Java. Some key differences are:

  • Map declaration and instantiation are simpler

  • Elements of a Map can be accessed either using the bean style or array style

  • Iterating across Maps is far less verbose in Groovy than in Java

The code sample below is self explanatory

//Declaring and instantiating a Map
def emptyMap = [:]

//Populating a Map
//Notice that the keys which are strings are not
//enclosed within quotes
def population = [Estonia:1299371,

//Non string keys must be enclosed in ()
def numbers = [(Integer.valueOf("1")):"One",

//Accessing Map elements
//Maps can be accessed using bean style and
//array style notations respectively
println "Estonia population: " + population.Estonia
println "Finland population: " + population['Finland']
//Use default values in case key does not exist (if key does not exist, then
//it will be added with the default value)
println "Spain population: " + population.get("Spain", 40525002)

//Iterating over all the entries in a Map
println "Population stats"
population.each { key, value ->
println key + " : " + value

println "Numbers as words"
numbers.each() {
println it.key + ":" + it.value

//Search within a Map
println 'Countries with a population greater than 1 million'
result = population.findAll {key, value ->
value > 5000000
println 'population > 5000000 ' + result.keySet()

Output from running the above program:

Estonia population: 1299371
Finland population: 5250275
Spain population: 40525002
Population stats
Estonia : 1299371
Finland : 5250275
Honduras : 7792854
HongKong : 7055071
Spain : 40525002
Numbers as words
Countries with a population greater than 1 million
population > 5000000 ["HongKong", "Finland", "Spain", "Honduras"]

Friday, March 27, 2009

Groovy types

Java is a statically typed language, which means we have to declare the type of a variable at compile time. Python is dynamically typed. which means that references can never to bound to types at compile time. Groovy on the other hand has optional typing. What this means is Groovy references can be statically typed like in Java, or dynamically typed like in Python.

String staticStr = new String("this is a statically typed String");
staticStr = 8;
println """The type of staticStr is: ${staticStr.class.getName()} """
println """It's value is: $staticStr"""

The output form running the above program is:

String staticStr = new String("this is a statically typed String");
staticStr = 8;
println """The type of staticStr is: ${staticStr.class.getName()} """
println """It's value is: $staticStr"""

As you can see, staticStr remains a string even after it is assigned 8. I will leave it as an exercise to figure out how 8 got assigned to a string type.

//s is not declared to be of any type but holds a String object
def dynamicStr = "this ia a dynamic string"
dynamicStr = 90
println """Type of dynamicStr is ${dynamicStr.class.getName()}"""

The output from running the above code is:

Type of dynamicStr is java.lang.Integer

So, why does Groovy support optional typing? Groovy tries to be feature rich language with support for closures and meta-programming. Both these need Groovy to be dynamically typed. However Groovy also promises compatibility with Java. Hence there are times when we may want it to be statically typed. Groovy takes the unique approach of being an optionally typed language, which allows it to be feature rich and compatible with Java.

I would like to take a little detour here and quote something I read:

"The recent trend is towards dynamically typed languages, which use inference to reduce the amount of code to be written. Java, C# and C++ are "static typed" languages, whereas JavaScript, Ruby, Python and PHP (used by Facebook) use "dynamic typing". In static languages, the programmer must declare each variable's class (such as string, integer, or list) before the program runs; in dynamic typing, it is assessed as the program runs. Fowler says Ruby is between two and five times more productive than Java. "In static languages, there are all these speed bumps. You can just express yourself more rapidly in languages like Ruby and Python, and that gives you an edge. That edge does make a significant productivity difference." - Source: The Guardian

I will not discuss details of static vs. dynamic typing in this blog post, but suffice to say that dynamic typing makes it possible for Groovy to have flexible closures, and metaprogramming. Groovy supports static types to allow inter operation with Java. This is very nice, because not only can a Groovy program take advantage of existing Java libraries, but Java programmers who are not yet fully familiar with Groovy can start Groovy programming with a known paradigm and proceed with closures and metaprogramming as their knowledge of Groovy increases.

Wednesday, March 25, 2009

Groovy Ranges

Groovy Collections offer us a class in addition to Java Collections, called Range. Range is a subclass of java.util.List and is used for creating lists of sequential values. Some ranges can be:
Notice that we use the .. operator to create a range.

However ranges are not limited to numbers and characters. Any object which implements the methods compareTo(), next(), and previous() can be used in a Range.

Ranges typically have a left and right border, and they can be either inclusive (containing border elements) or exclusive (sans the right border element).

Ranges are usually used to extract a subset of elements from a list or used in switch statements to match a range of cases together.

The code below is self explanatory:

//A simple Groovy Range
def inclusiveRange = 1..10
println 'The size of inclusiveRange is: ' + inclusiveRange.size()

//This range does not include the right border element
def exclusiveRange = 1..<10 exclusiverange1 =" 1<..10" negativepositiverange =" -5..5" negativedescrange =" -1..-10" negativeascrange =" -10..-1" out =" new" out =" new" out =" new" out =" new" out =" new" list =" [0,1,2,3,4,5,6,7,8,9]"> 0}

Output from running the above program:

The size of inclusiveRange is: 10
inclusiveRange: 1,2,3,4,5,6,7,8,9,10,
exclusiveRange: 1,2,3,4,5,6,7,8,9,
negativePositiveRange: -5,-4,-3,-2,-1,0,1,2,3,4,5,
negativeDescRange: -1,-2,-3,-4,-5,-6,-7,-8,-9,-10,
negativeAscRange: -10,-9,-8,-7,-6,-5,-4,-3,-2,-1,
sublist of first 4 element from list: [0, 1, 2, 3]
sublist of last 2 element from list: [9, 8]
negativeAscRange,grep(-5..-3): [-5, -4, -3]
matching all positive values in negativePositiveRange: [1, 2, 3, 4, 5]

Thursday, March 19, 2009

Groovy closures

Java does not have closures, but till sometime back it was a highly debated topic on various Java forums. Groovy is a language which compiles to Java bytecode and does have closures.

Closures are very prominent in many programming languages, such as Groovy, Python, Ruby, Javascript, and probably many more.

So what are closures? A closure is a block of code which can be passed to a function. Closures in Groovy are normal objects, which do not have any visible state.

Closures are most often used in iteration logic and resource handling.

Whenever we iterate across a collection, we normally do so to perform some logic on all or some members of the collection. Closures make it easy to represent that logic in a very concise and readable way.

Closures can also be used for resource handling. Very often we open a file resource do something with the file (read from it, write to it, or both), and then close the file. The first and last parts, that is opening and closing the file are really plumbing code. The interesting part is what we do with it. Using closures it is possible to create objects which will take care of the plumbing and will perform some operations on the file, such that the code which represents the operations are given to it in a closure.

Here is an example. Let's assume we have a list of Toy objects, where each toy object has a color which can be accessed with the getColor() method. I want all the blue toys and my sister wants all the pink toys.

Here is code to get toys for me using regular Java syntax:

Color myFavColor = Color.BLUE;
List toysIWant = new ArrayList()
Iterator iter = toys.iterator();
while(iter.hasNext()) {
Toy toy = (Toy);
if(toy.getColor().equals(myFavColor)) {

Here is code to get toys for my sister using closures:

def mySistersFavColor = Color.PINK
def toysMySisterWants = toys.grep() {
it.getColor() == mySistersFavColor

What really happened in the second code snippet was, the list object has a grep method which takes a closure. It calls that closure for every item in the list and passes that item to the closure as the variable 'it'. If the expression in the closure evaluates to true, then the item is added to an internal list. This list is returned to the caller when grep returns.

You might be thinking that this can be done in Java using anonymous inner classes. That is partially right. But there are two very important differences. Anonymous inner classes cannot refer to non static local variables in the scope they are declared in, which means they could not have referred to the variables 'myFavCOlor', and 'mySistersFavColor'. Another difference is that anonymous inner classes are very verbose. This may not seem important, but many people have said that it is something that makes anonymous inner classes unusable.