Tuesday 15 May 2012

Maybe an Answer to Null in Java?

Null References: The Billion Dollar Mistake

Tony hoare famously said of his invention of the null reference in 1965;

"This has led to innumerable errors, vulnerabilities, and system crashes, which have probably caused a billion dollars of pain and damage in the last forty years"

Today we still suffer much pain from this, with Java's infamous NullPointerException being one of the most frustrating avoidable errors that can occur, almost always as a result of programmer error. So we have to ask ourselves, why are we still suffering so much?

Neither Type Safe nor Object Orientated

The concept that you can return a null pointer from a method call seems to go against the concept of both the Object Orientation Paradigm and a secure Type System. For example if a method declares a return type of Map the type system will allow you to return a HashMap, but if you try to return a List it would cause a compiler error. On an object that is returned you expect to be able to call a method declared on Map, such as Map#put, this is valid with HashMap as it extends Map but not on List. The question then arises why it is acceptable to return null because when you try to call null#put you get the dreaded NullPointerException.

Contagion

Once you allow null pointers into your code, very much like a virus, they quickly infect your whole system. The symptoms that are exhibited are excessive null pointer checks and error stack traces containing NullPointerExceptions. You find your code starts to get very ugly as you have to insert null checks not only at the place that the null occurs but everywhere, quiet often just out of the fear of what might be that can grip a developer.

The Null Object Pattern

The first solution to consider is the Null Pointer Pattern, where an object with neutral behaviour can be created and returned instead of null. While, if possible, this is probably the best way to deal with the problem it is not sufficient to deal with all the possible use cases we come across, for example finding a value in a list, so we need something else to add to our tool box.

Annotating the problem

There are annotations around that can be used to assist in reducing the chances of null pointer exceptions, most notably Intellij's @NotNull and @Nullable, which allow static analysis of the code to help sugest places where problems could arise.

Maybe an Answer?

If the type system could enforce the handling of nulls at source you could happily code elsewhere without any chance of a null creeping in. Other programming languages have solutions to this problem, with one of these being the Maybe type in Haskell. From the Haskell documentation on Maybe;

The Maybe type encapsulates an optional value. A value of type Maybe a either contains a value of type a (represented as Just a), or it is empty (represented as Nothing). Using Maybe is a good way to deal with errors or exceptional cases without resorting to drastic measures such as error.
The Java implementation, for which you can find GitHub details below, consists of the Maybe interface and two concrete classes Just, which wraps the object being returned, and Nothing, which represents the absence of an object.

Using Maybe

So what would its usage look like? A couple of such examples are here.

In this example the method declaration clearly tells the client code that the method may not return an object of type Bar, but it will return an Object of type Maybe. The Maybe construct also allows the type system to enforce that Foo#doSomething does not then pass through the possible null it may otherwise have done. This also displays a convenience method which allows you to get the contained object or return a default value instead, in this case an implementation of the Null Object pattern,

The following shows the same example but this time allowing the method to return null. This shows how the null checks can make the code less concise or how a null reference can be mistakenly either returned to the calling client or passed onto another method.

Where Are Null Checks Still Acceptable?

At the boundaries between your code and third party libraries, such as the Java collections, as well as for external inputs to your system. I would not even use it for lazy initialization rather again make it explicit that this value may not be set, there are many times I have seen a field set in the getter method only for it to be accessed at a later time from another method directly assuming it was set.

Full Java Implementation of Maybe

The full Java implementation of Maybe can be found on my GitHub page.

https://github.com/leneghan/maybe

You can also find an implementation in Googles guava library called Optional.

http://docs.guava-libraries ... /Optional.html

Thursday 26 April 2012

Scala Dojo 04 - implicit java


Summary

Today we look at how easily you can write Scala that works with Java code as well as looking at a powerful language feature, implicit conversions and parameters.

Implicit Java

This week you will find some Java classes as well as the usual Scala ones. You will be implementing functions on the UserLookup Scala class, which itself queries a Java implemented DataSource class through its findUsersmethod, from which will be returned a Java collection of User's. As usual work though by getting the tests to pass one at a time, and keep an eye on the couple of comments which hint at the direction you may want to take.

Building the project

You can either build and run the tests with either Maven or the Simple Build Tool. SBT is incredabilly easy to to install and it is highly recommended you give it a try as it is the standard Scala build tool.

Getting the code

To get started you can go to the repo and fork the project into your own github repository by clicking the fork button on the top right. This is the preferred method as it means we can easily share our solutions with each other but if you don't wish to do this you can create a local copy from my repository by simply executing the git clone command.
  1. go to https://github.com/leneghan/scala_dojo_04_implicitjava
  2. click fork button on the top right of the screen
  3. git clone https://github.com/<GIT_HUB_NAME>/scala_dojo_04_implicitjava.git
or
git clone https://github.com/leneghan/scala_dojo_04_implicitjava.git

Getting started with git

If you dont have git currently installed you can download it here from the git website. When using Git I found this one page cheat sheet useful, it also comes with some pointers about good VCS practice, Git_Cheat_Sheet_grey.pdf

Resources

Labels:

Monday 16 April 2012

A Lightning Introduction To Erlang - Slides

These are the slides of a lightning talk I recently gave on erlang, covering its history, general characteristics and finishing with the obligatory hello world code snippet.


Monday 2 April 2012

Scala Dojo 03 - funs with OO

Summary

This weeks dojo will delve into the other halve of Scala's dual personality, Object Orientation. We will be looking at how objects are created and operated on as well as discovering how Traits can help modularise your code and remove more duplication than Interfaces alone.

Funs with Object Orientation

This week you will find 6 tests that you will have to complete as well as then make them pass, though you will find a couple of helper methods that assert some of their functionality. Along with tests there is an Item object hierarchy, a User and a couple of Traits with only the method signatures implemented. The skeleton maven project I have created for the dojo can be found here on github.

SBT

You will find that this project now builds with both Maven and SimpleBuildTool. SBT is incredabilly easy to install and it is highly recommended you give it a try as it is the standard Scala build tool.

Getting the code

To get started you can go to the repo and fork the project into your own github repository by clicking the fork button on the top right. This is the preferred method as it means we can easily share our solutions with each other but if you don't wish to do this you can create a local copy from my repository by simply executing the git clone command.
  1. go to https://github.com/leneghan/scala_dojo_03_funswithoo
  2. click fork button on the top right of the screen
  3. git clone https://github.com/<GIT_HUB_NAME>/scala_dojo_03_funswithoo.git
or
git clone https://github.com/leneghan/scala_dojo_03_funswithoo.git

Getting started with git

If you dont have git currently installed you can download it here from the git website. When using Git I found this one page cheat sheet useful, it also comes with some pointers about good VCS practice, Git_Cheat_Sheet_grey.pdf

Resources

Labels:

Thursday 22 March 2012

Experiences of Using Git-Svn on a Large Project

I have been dabbling with Git for a little while now on side projects and I have felt that it was a very powerful tool that offered a big improvement over the last generation of version control systems, but it was only this week that I started using it in anger at work on a large project and so far I have not been disappointed.

Git and Svn

As we use Svn as our VCS I am using git-svn to connect between my local Git repo and the remote Svn one. The first thing you notice when moving to Git is the increased speed of many of the actions that previously took what felt like several seconds, for example fetching the commit log is now pretty much instant as is looking at a files history, in fact I have found myself using IDE features such as annotate much more now. I can annotate a file, click to examine all the other files touched in the same commit and then look at a couple of diffs all without any visible lag.

Cloning

The first thing you have to do is get a git clone of your SVN repo on your machine. Beware, this will take a while, to clone our SVN took approximately 2 hours and resulted in a git repo of 300Mb. This would have taken longer without the --log-window-size option and can also be used with the git-svn fetch command.

git svn clone -s --log-window=10000 https://host.com/svnr gitr

You can also limit the amount of work git-svn has to do by specifying the revision that you wish to clone from.

git svn clone -r 2908 https://host.com/svnr gitr

Pushing and Pulling

The most obvious difference between using git-svn rather than git by itself is that whereas you would push and pull to the remote git repo, instead you dcommit and either fetch or rebase, the later of which reorders the commits that come in to put yours on top.

git svn dcommit
sit svn [fetch|rebase]

Technicolor

A nice feature that comes built in, but switched of by default, is the color coding on the command line which makes the key information stand out, especially when running a quick diff before a commit. To switch this feature on you just need to enter the following.

git config --global color.ui "auto"

To enhance the command line further you may find the script that a friend sent to me useful. It modifies your command prompt not only to tell you the name of the branch you have checked out but also, by the colour of the branch name, whether you have any uncommitted changes.

user@host:~/git/git_svn_bash_prompt (master) $ _

The script itself can be found here on github;

https://gist.github.com/726967

To get it to work just download it and add the following line to your .bash_profile or equivalent.

source git/git_svn_bash_prompt/git_svn_bash_prompt.sh

The Log

Looking at the commit log in git is not only faster but also more user friendly because of features such as the automatic pagination and the different types of layout available. The oneline display option is so useful that I set it up as an alias.

alias gitlog="git log --pretty=oneline"

Merging

Another instant win for me was how easy it was to move my commits from one branch to another. Instead typing this;

svn merge -c 188973 --ignore-ancestry https://someurl.com/svn/MYPROJ/components/server/branches/theBranch .

It was now a simple matter of cherry picking;

git cherry-pick e4df5

Branching

I also found the simplicity of creating a branch made it easier to work on more than on thing at a time. Quite often I will find myself working on a feature only to be asked to look at an urgent issue or to make a quick change somewhere. It is so much simpler now that I can either stash the work if it is incomplete or commit it locally, branch from trunk then make and commit the change, before going back to carry on with the feature.

Encouraging Uptake

While you can use git-svn in isolation with everyone else working away on SVN it can be much more useful if you can encourage others to work on git too. This is obviously more difficult on a large project with many different developers of varying experiences. To this end we have as part of our monthly tech book club ordered 20 copies of the Pro Git book, which we will be following up with some practical sessions to introduce people to the basic workflow.

To be continued ...

I will try to follow this up in the next few weeks with more observations of how using git-svn helps or hinders as opposed to vanilla svn.

Saturday 10 March 2012

Reducing Java Memory Usage and Garbage Collections with the UseCompressedOops VM Option


Overview

A live trial of the UseCompressedOops JVM argument (-XX:+UseCompressedOops) has shown to reduce the memory usage of the application by 14% and the number of garbage collections by 24%. This means reduced latency due to fewer pauses when performing a full Garbage collection, reduced load on the CPU due to fewer minor Garbage Collections, and a reduced memory requirement on the machine.
The results of this trial clearly show that this VM option should be used, though since the release of 6u23 this option is actually set on by default.

Background

The application currently runs on the 64 bit version of the 6u21 JVM and due to the larger 64 bit memory adresses the java heap is significantly larger than when running on a 32bit JVM. Fortunately  since version 6u14 there has been a JVM option, UseCompressedOops, intended to win back much of the memory lost due to the larger address space. How this is achieved is by representing the addresses on the heap in 32-bits then, when they are loaded, scaled by a factor of 8 and added to a base address to decode them to the full 64-bits. You can read a more detailed write up here;

https://wikis.oracle.com/display/HotSpotInternals/CompressedOops

One thing to beware of is that the maximum heap size of the JVM will be limited to 32Gb.

Trial

We set a single application server to use use compressed oops and compared its memory and GC characteristics over the following hours with a placebo without compressed oops.

Results

Garbage Collection
Over a 1 hour period the following number of garbage collections occurred.
ReductionMinor GCFull GCAvg. Sec. Between GC
Without UseCompressedOops0%37949.3
With UseCompressedOops24%292312.2
Memory Usage
This table shows the average memory over the hour retained in the old generation after a full GC, this is a good guide to the overall reduction in memory used by the application.
ReducedAverage
Without UseCompressedOops0%685M
With UseCompressedOops14%593M

Friday 9 March 2012

Scala Dojo 02 - funs with lists


Summary

This weeks dojo should start to push you a little harder and you may need more than the hour to complete all the challenges. Along the way you will be introduced to Scala's Lists and functions, these two pages are good starts on the subject;

Funs with Lists

There are 8 tests that you will need to go green, plus a bonus one at the end for 2 additional points. This week the tests are not exhaustive and so a certain amount of good sport is expected and as such points will be deducted for implementations that are not in the spirit of the game, for example hard coding return values. You'll also find some tests offer bonus points for implementing them in a particular way. The skeleton maven project I have created for the dojo can be found here on github, as last time you should not change the existing tests, but feel free to add new ones along the way.

Getting the code

To get started you can go to the repo and fork the project into your own github repository by clicking the fork button on the top right. This is the preferred method as it means we can easily share our solutions with each other but if you don't wish to do this you can create a local copy from my repository by simply executing the git clone command.
  1. go to https://github.com/leneghan/scala_dojo_02_funswithlists
  2. click fork button on the top right of the screen
  3. git clone https://github.com/<GIT_HUB_NAME>/scala_dojo_02_funswithlists.git
or
git clone https://github.com/leneghan/scala_dojo_02_funswithlists.git

Getting started with git

If you dont have git currently installed you can download it here from the git website. When using Git I found this one page cheat sheet useful, it also comes with some pointers about good VCS practice, Git_Cheat_Sheet_grey.pdf

Resources

Scala Dojo 01 - The MailBox


Summary

This inaugural dojo challenge should ease you into Scala gently, with the solution only requiring you to write about 6 lines of code. On the way you'll get Scala and Git running on your machine, hopefully set up a GitHub account, started to become familiar with the Scala syntax, patten matching and the ScalaTest framework.

Go here to see an overiew of this series of Scala dojos

The Mailbox problem

This problem involves completing the implementation of the MailBox.scala class, with the rules of how it should respond to different messages expressed in the tests. There is one rule in Dojo Club, get the tests to go greenThe skeleton maven project I have created for the dojo can be found here on github. Though you should not change the existing tests feel free to add new tests along the way.

Getting the code

To get started you can fork the project into your own github repository, this is the preferred method as it means we can easily share our solutions with each other. If you don't wish to do this you can create a local copy from my repository by simply executing the following command.
git clone https://github.com/leneghan/scala_dojo.git

Getting started with git

If you dont have git currently installed you can download it here from the git website. When using Git I found this one page cheat sheet useful, it also comes with some pointers about good VCS practice, Git_Cheat_Sheet_grey.pdf

Resources

Labels:

A Coding Dojo Series to Teach Scala


I created this series of Scala dojos to introduce developers at work to functional programming concepts and social coding with Git. As I create them I will post them here over the coming weeks, feel free to join in yourselves and do please provide any feedback that may be useful.

Summary

This series of dojos is intended to take you from a Scala novice to wizard in 8 weeks. To help encourage people to share their solutions I have set up a Scala Dojo Leaderboard to track those who complete the challenges, with additional bonus points available along the way. 

Format

The dojos are held once a week in a meeting room, preferably one with a round table. People bring along laptops if they have one and then either pair up or work solo on the problems.

Rules

Make the tests go green. Simple.

The Dojos

Collaborating with git and GitHub

The dojos will be using git and GitHub to collaborate on the problems, it is a great way to see how others have solved the coding challenges. Once you have an account with github you can fork any of the projects into your own github repository with the click of a single button.
To get a local copy of the code from the remote repository, either the master or your own, run this command.
git clone [GIT_REPOSITORY_LOCATION]
If you do not have git currently installed you can download it here from the git website. When using Git I found this one page cheat sheet useful, it also comes with some pointers about good VCS practice, Git_Cheat_Sheet_grey.pdf

Resources

How Scala Can Improve Your Life

Or How I Learned to Stop Worrying and Love less Code


These are the slides of a presentation I recently gave at work extolling the benefits of Scala and, maybe more importantly, why Java developers need to learn about functional programming.  

I should also warn you that it is unlikely that a programming can improve your life, but I guess you'll never know until you try, the next one might be a keeper.