Sunday, 18 December 2011

Scala, Clojure, Java Hello World

I finally took the plunge and decided to spare some time for functional programming. As being a java engineer, I wanted to have a look at jvm compliant functional languages which are Scala and Clojure.
Scala seems to be available since 2003, although public interest raised in recent years, probably after java 1.5 release. Scala is regarded as an object oriented, functional and statically typed language. It seems Scala was designed to attract java developers from the beginning.
On the other hand, Clojure is a dynamic Lisp dialect,running on jvm. Clojure is not an object oriented language like Scala, though it is possible to achieve runtime polymorphism and interface declaration.
Scala reminds me complexity of C++, by trying to be functional and object oriented at the same time, it also introduces a new type type system which is a bit resource draining for me.
In contrast to Scala, Clojure is dynamic and does not require a static type system, most operations are done with well known vector, list, map and set data structures. Clojure gives feeling of a natural functional programming impression.
At the weekend, I wondered how a simple “Hello World” program would perform in both languages and decided to run two versions of “Hello World”.

Clojure version of Hello World
Clojure : 1.3.0
Java : Jdk 1.6.0_22
user=> (time (dotimes [_  5e4] (println "Hello World")))
"Elapsed time: 4249.292309 msecs"
nil
user=>
user=> (time (dotimes [_  5e5] (println "Hello World")))
"Elapsed time: 42021.368085 msecs"
nil
user=>

Scala version of Hello World
Scala : 2.9.0.1
Java : Jdk 1.6.0_22

val start = System.currentTimeMillis (); 
for (i <- 1 to 50000 ) {
    println ("Hello World");
}
println ("Elapsed time: " + (System.currentTimeMillis ()- start));
Elapsed time: 5135
val start = System.currentTimeMillis (); 
for (i <- 1 to 500000 ) {
    println ("Hello World");
}
println ("Elapsed time: " + (System.currentTimeMillis ()- start)); 
Elapsed time: 52589

Java version of Hello World
Java : Jdk 1.6.0_22


long start= System.currentTimeMillis();
for (int i=0; i<5e4; i++)
   System.out.println("Hello World");
System.out.println("Took " + (System.currentTimeMillis()- start)); 
Took 2809


long start= System.currentTimeMillis();
for (int i=0; i<5e5; i++)
   System.out.println("Hello World");
System.out.println("Took " + (System.currentTimeMillis()- start)); 
Took 27092


I was expecting Scala would perform better than Clojure, but results were a bit surprising for me. I knew there were some performance improvements in Clojure 1.3 regarding to numeric operations by using primitive hints, but there wasn’t any numeric operation in the hello world demo. Maybe Scala code wasn't equivalent of Clojure one.

Friday, 25 November 2011

War of the Languages

It wasn’t until recently that new JVM based languages spread all around. Clojure, Scala, Ceylon and Kotlin are  the newest ones joining the rush and elderly ones are Groovy, JRuby and Jython.
Clojure and Scala are new frontiers of functional language domain, stating immutability and no state sharing like their predecessors Erlang, Haskell and Lisp.
On the other side Ceylon and Kotlin are commercially initiated languages, as a response to to Oracle’s Sun acquisition. As an IDE company Jett Brains launched its own Kotlin language, while JBoss drove Ceylon under the management of Gaving King.
Personally I can understand Ceylon and Kotlin language initiatives as they are supported by JBoss and JetBrains respectively. It is obvious that JBoss as a J2EE player does not want to depend on Oracle’s Java plans and lose market share in the long term. Same logic is valid for JetBrains as well. I suspect both companies does not want to wait for Oracle’s Java plan or to sued due to any kind of Java patent infringements.
When it comes to functional languages, I don’t think none of them will be a main stream language for another decade unless todays micro processor design does not change. It is evident everything is stateful in microprocessors, from binary calculations to signed/unsigned numbers and assembler commands. In all these cases, a state check exists. I can say that functional-program contradicts with today’s basic principles of computing.
When reviewing those new languages, I think there is a need to highlight what’s wrong with current Java, what kind of obstacles are exist? In my opinion there are some certain issues needing improvement, like nasty null checks, repetitive type casting, broken garbage collection and memory leaks in class loaders, lack of true isolated class loaders with self memory management parameters, no built-in immutable object support, no modular system to deploy applications with a small dependency footprint (JRE 1.7 size is around 19mb).
To be honest, I don’t think Java will die soon due to those aforementioned issues, it is also clear that new emerging languages are not completely addressing today’s software development problems. I think we are in a corner stone that “one size fits all” doesn’t work any more. It’s high likely there are going to be software systems including components developed by different languages. For example a front end web application might be java based while a backend pricing/exchange application can be built onto Erlang/Scala to benefit from native immutability/messaging support.
I would like to see picking right tool for the job, instead of pushing one tool for everything approach. I imagine neither Java nor any of those aforementioned languages will address today’s software development issues alone, instead they will live in the same ecosystem altogether.

Monday, 21 November 2011

My initial rabbitmq experiment

Last weekend I wanted to spend some time on messaging software available as open source, did some investigations and found a couple of messaging products. I noticed that most of products generally implemented JMS, including Apache ActiveMQ, Jboss Hornet, but RabbitMQ was an exception. It was completely different than those JMS implementations from ground.
First of all, rabbitmq development is focused on a wiring protocol called Advanced Messaging Queing Protocol (AMQP) instead of JMS. To achieve high performance requirements rabbitmq is built on Erlang language. Erlang itself is a functional, interpreted language concentrated on reliability, clustering and immutability of messages. It is not surprising to see guys at rabbitmq made good decision by choosing Erlang as development language of rabbitmq.
As messaging platforms require very low latency figures and clustering, it would not be reasonable to see Java for such a messaging platform. By design Java itself is not a functional language and immutability of objects requires a lot effort, not to mention that Erlang comes with Open Telecom Platform (OTP) tools, which provides a bunch of tools for clustering, distributed communication and database management .
After spending some time on comparison of messaging products and reading reviews. I decided to try rabbitmq on my Windows laptop. I followed rabbitmq windows installation guide, which directed me to Erlang binaries download page, followed Erlang instructions to setup an Erlang environment.
Having Erlang and rabbitmq installed, I tried to use rabbitmq web based management plug-in after invoking rabbitmq-plugins enable rabbitmq_management command at command prompt. Unfortunately for some reason web admin console did not run. Later I decided to use Windows zip distribution instead of Windows service binaries and after downloading and extraction them to C:\BF\rabbitmq_server-2.7.0 folder, I followed well documented rabbitmq configuration guideline and created RABBITMQ_BASE environment variable pointing to C:\BF\rabbitmq_server-2.7.0\RABBITMQ_BASE.
Finally I was able to run rabbitmq via sbin/rabbitmq-server command and see web console screen at http://localhost:55672/#/ with default authentication pair guest/guest provided.
To summarize, my initial rabbitmq impression was good, RabbitMQ is well documented, there are plenty of screencasts, videos and tutorials to accelerate rabbitmq learning curve. It is also worth mentioning that it achieves a high performance throughput and high availability  thanks to underlying Erlang runtime.

RabbitMQ Environment Variable
RabbitMQ Environment Variable
RabbitMQ startup console
RabbitMQ Startup Console
RabbitMQ Web Console
RabbitMQ Web Console

Monday, 14 November 2011

Java class loader and Maven plugin


I recently spent some time to develop a maven plugin which was supposed to run keyword-data driven spread sheet tests via an internally developed testing framework. The framework was running tests as a stand-alone application and tests were executed by the framework’s class loader which was a bad practice as testing harness and test projects were mixed in the same context and there wasn’t any abstraction between them.

Ideally testing framework and test projects should be de-coupled, since test projects and the testing framework have their own individual dependency trees. When it comes to using Maven as a project build tool, it builds up a dependency tree according to its dependency mechanism, which means it could mix dependency of test projects and testing framework and create a new breed of dependency tree.



maven-dependency-project
A sample view of a test project’s dependency tree.



maven-dependency-frw
A sample view of the testing framework’s dependency tree

When classloaders and dependency trees are considered, I personally think Servlet containers are perfect examples of decoupling and class loaders, they have been around for decades, are proven to be best samples of class loading over the years.

For the testing framework maven-plugin development, I started to do refactoring to extract out an interface/api definition of the testing framework so that those refactored interfaces could be served as a contract between the testing framework and test projects as servlets-api serves. In fact when servlets-api is looked at in detail, it can be observed that it’s a contract between a servlet’s container and web application.

Once having a testing framework contract api, rest of the maven plugin development and class loading involved steps creating another test projects class loader, scanning test spread sheets and facades and running those facades through test projects class loader.

Test project class loader creation

private ClassLoader createTestProjectClassLoader() throws MojoExecutionException {
    List<String> testClasspathElements = null;
    try {
        testClasspathElements = this.mavenProject.getTestClasspathElements();
    } catch (DependencyResolutionRequiredException e) {
        new MojoExecutionException("Dependency resolution failed", e);
    }


    List<URL> projectClasspathList = new ArrayList<URL>();
    for (String element : testClasspathElements) {
        File elementFile = new File(element);
        URL url;
        try {
            url = elementFile.toURI().toURL();
            projectClasspathList.add(url);
            }
        } catch (MalformedURLException e) {
            throw new MojoExecutionException(element + " is an invalid classpath element", e);
        }
    }
    ClassLoader pluginClassLoader = getClass().getClassLoader();
    ClassLoader mavenClassLoader = pluginClassLaoder.getParent();
    ClassLoader testProjectClassLoader =
        new URLClassLoader(projectClasspathList.toArray(new URL[projectClasspathList.size()]), mavenClassLoader);
    return testProjectClassLoader;
}


By using different class loaders in the maven plugin for the framework initialization and spread sheet test execution, any possible jar hell issue was avoided and testing framework context and test project contexts were separated.

As a final word, I would like to emphasize that the key point of achieving different class loaders is having a concrete api/interface definition which can be served as contract between two parties.  Once a contracting interface/api is established, it can be loaded by a parent class loader which is used by both class loaders as shown in the example.

Monday, 15 August 2011

Hazelcast, Distributed Data Grid


Being on the threshold of high computing decade, there is a growing demand for ultra fast computing and data access. Until recently it has been quite a tough task to accomplish distributed messaging and data processing solutions, particularly requiring C/C++ interactions over JNI for java applications.

Luckily java platform has been fortified for parallel computing after jdk 1.5 platform release which includes a new concurrency package and improved util.collections. Probably these new features and technology advancements, multi core processors, high speed networks, like InfiniBand, made possible developing java-native parallel computing and data grid solutions.

Hazelcast is one of the data grid innovations promoting itself as “open source clustering and highly scalable data distribution platform for Java” amongst these latest java platform enhancements.

Using Hazelcast is quite easy as it’s a single deployable jar and does not include any external dependency. It scales linearly as the number of nodes increases on the grid. Hazelcast can fit the following scenarios perfectly.
  • Share data/state among many servers (e.g. web session sharing)
  • Cache your data (distributed cache)
  • Cluster your application
  • Provide secure communication among servers
  • Partition your in-memory data
  • Distribute workload onto many servers
  • Take advantage of parallel processing
  • Provide fail-safe data management

There is a demo application presenting Hazelcast’s scalability on a 50-node grid processing Mozilla crash reports.

Wednesday, 10 August 2011

jaxws-maven-plugin

I like maven as a build management tool. It provides a predefined and flexible folder structure, a dependency management mechanism enabling using versioned dependencies, plugins, project organization as modules and profiles.
All these things nowadays are considered quite common for a java project, probably nobody wants to revert back to a manual build system.
Maven is so popular that there are hundreds of plugins of it, from unit test execution to stub/proxy generation. Having a huge availability of plugins may cause taking risk of unfound bugs in these plugins and having delayed delivery times. Generally plugins are contributed by open source committers so there isn’t any dedicated resource for most plugins to fix bugs found in a plugin.
One of these popular plugins is jaxws-maven-plugin generally used for java proxy creation from a wsdl file or vice-versa. It’s developed by java.net and including some bugs like all other plugins.
I was thinking of jaxws-maven-plugin as a high quality tool, at least expecting some dedicated support from java.net and Sun/Oracle guys. Apparently I was wrong as this tool includes a bug not fixed more than two years!.
The bug can be observed while trying to create proxy classes from two or more wsdl files by creating two execution sections, see below, inside the plugin configuration.
Second.wsdl file is not processed by the below plugin configuration, might cause us to spend time wondering why the second one is not processed.
Thanks to another guy for pointing out this issue and finding the actual underlying bug, I didn’t spend much time for a workaround.
But the main issue with that particular plugin is that this bug was reported in Jan’ 09 and it hasn't being fixed so far!.


Default jaxws configuration


<plugin>
 <groupId>org.codehaus.mojo</groupId>
 <artifactId>jaxws-maven-plugin</artifactId>
 <version>1.10</version>
 <executions>
  <execution>
   <id>execution-one</id>
   <configuration>
    <wsdlDirectory>src/main/resources</wsdlDirectory>
    <wsdlFiles>
     <wsdllFile>First.wsdl</wsdllFile>
    </wsdlFiles>
    <packageName>tr.asahin.first.webservice</packageName>
    <keep>true</keep>
   </configuration>
   <goals>
    <goal>wsimport</goal>
   </goals>
  </execution>
  <execution>
   <id>execution-twot</id>
   <phase>generate-sources</phase>
   <configuration>
    <wsdlDirectory>src/main/resources</wsdlDirectory>
    <wsdlFiles>
     <wsdllFile>Second.wsdl</wsdllFile>
    </wsdlFiles>
    <packageName>tr.asahin.second.webservice</packageName>
    <keep>true</keep>
    <extension>true</extension>
   </configuration>
   <goals>
    <goal>wsimport</goal>
   </goals>
  </execution>
 </executions>
</plugin>



Applying a workaround

<plugin>
 <groupId>org.codehaus.mojo</groupId>
 <artifactId>jaxws-maven-plugin</artifactId>
 <version>1.10</version>
 <executions>
  <execution>
   <id>execution-one</id>
   <configuration>
    <wsdlDirectory>src/main/resources</wsdlDirectory>
    <wsdlFiles>
     <wsdllFile>First.wsdl</wsdllFile>
    </wsdlFiles>
    <packageName>tr.asahin.first.webservice</packageName>
    <keep>true</keep>
    <staleFile>${project.build.directory}/jaxws/stale/first.txt</staleFile>
   </configuration>
   <goals>
    <goal>wsimport</goal>
   </goals>
  </execution>
  <execution>
   <id>execution-twot</id>
   <phase>generate-sources</phase>
   <configuration>
    <wsdlDirectory>src/main/resources</wsdlDirectory>
    <wsdlFiles>
     <wsdllFile>Second.wsdl</wsdllFile>
    </wsdlFiles>
    <packageName>tr.asahin.second.webservice</packageName>
    <keep>true</keep>
    <staleFile>${project.build.directory}/jaxws/stale/second.txt</staleFile>
    <extension>true</extension>
   </configuration>
   <goals>
    <goal>wsimport</goal>
   </goals>
  </execution>
 </executions>
</plugin>

Wednesday, 3 August 2011

Java Thread Deadlocks

Generally deadlocks occur when two threads try to lock an object held by the counterpart thread.
Java has some internal locking mechanism, one of them is using synchronized keyword. When a thread enters into synchronized method, thread acquires object's lock, and releases the lock after leaving the synchronized method.
I think the deadlock issue in java can be visualized as below to understand better the issue.

Thread Deadlock















Here is a junit code for generating  a java deadlock.
import junit.framework.Assert;
import org.junit.Test;

/**
 * @author asahin
 */
public class ThreadTests {

    @Test
    public void testDeadLock() throws InterruptedException {

        final Writer writer = new Writer();
        final Reader reader = new Reader();
        writer.setReader(reader);
        reader.setWriter(writer);

        Thread readerThread = new Thread(new Runnable() {
            public void run() {
                try {
                    reader.readerOperation();
                } catch (InterruptedException e) {
                    e.printStackTrace();
                }
            }
        });

        Thread writerThread = new Thread(new Runnable() {
            public void run() {
                try {
                    writer.writerOperation();
                } catch (InterruptedException e) {
                    e.printStackTrace();
                }
            }
        });

        readerThread.start();
        writerThread.start();

        readerThread.join(5000);
        writerThread.join(5000);
        Assert.assertTrue("Expecting readerThread alive", readerThread.isAlive());
        Assert.assertTrue("Expecting writerThread alive", writerThread.isAlive());
    }

    public static class Reader {
        public void setWriter(Writer writer) {
            this.writer = writer;
        }

        private Writer writer;

        public synchronized void readerOperation() throws InterruptedException {
            //rest little bit
            Thread.sleep(1000);
            System.out.println("Thread using Reader is:" + Thread.currentThread());
            writer.writerOperation();
        }

    }

    public static class Writer {
        private Reader reader;

        public void setReader(Reader reader) {
            this.reader = reader;
        }

        public synchronized void writerOperation() throws InterruptedException {
            //rest little bit
            Thread.sleep(1000);
            System.out.println("Thread using Writer is:" + Thread.currentThread());
            reader.readerOperation();
        }
    }

}

Verification of deadlock

Thread[readerThread,5,main] inside Reader
Thread[writerThread,5,main] inside Writer
2011-08-03 13:28:26
Full thread dump Java HotSpot(TM) Client VM (19.1-b02 mixed mode, sharing):
....
....
....
Found one Java-level deadlock:
=============================
"writerThread":
  waiting to lock monitor 0x01b1d99c (object 0x23d51640, a tr.asahin.ThreadTests$Reader),
  which is held by "readerThread"
"readerThread":
  waiting to lock monitor 0x01b18cc4 (object 0x23d4f960, a tr.asahin.ThreadTests$Writer),
  which is held by "writerThread"

Java stack information for the threads listed above:
===================================================
"writerThread":
 at tr.asahin.ThreadTests$Reader.readerOperation(ThreadTests.java:57)
 - waiting to lock <0x23d51640> (a tr.asahin.ThreadTests$Reader)
 at tr.asahin.ThreadTests$Writer.writerOperation(ThreadTests.java:75)
 - locked <0x23d4f960> (a tr.asahin.ThreadTests$Writer)
 at tr.asahin.ThreadTests$2.run(ThreadTests.java:32)
 at java.lang.Thread.run(Thread.java:662)
"readerThread":
 at tr.asahin.ThreadTests$Writer.writerOperation(ThreadTests.java:73)
 - waiting to lock <0x23d4f960> (a tr.asahin.ThreadTests$Writer)
 at tr.asahin.ThreadTests$Reader.readerOperation(ThreadTests.java:59)
 - locked <0x23d51640> (a tr.asahin.ThreadTests$Reader)
 at tr.asahin.ThreadTests$1.run(ThreadTests.java:22)
 at java.lang.Thread.run(Thread.java:662)

Found 1 deadlock.