Java/Maven

CopyOnWriteArrayList in Java

I am reading Java Concurrency in Practise. Priceless if you know a thing or two about concurrency already, but would like to brush up on the theory and also learn one or two new developments. The book is extremely well-written.

And here is a little something I would like to share. In the book Brian Goetz glides over this, but this is so amazing, that it deserves a deeper look.

// CopyOnWriteTest.java

import java.util.ArrayList;
import java.util.Iterator;
import java.util.List;
import java.util.concurrent.CopyOnWriteArrayList;

public class CopyOnWriteTest {

	private static void f( List l ) {
		l.add( 2 );
	}

	/**
	 * @param args
	 */
	public static void main(String[] args) {
		List copyOnWriteList = new CopyOnWriteArrayList();
		List simpleList = new ArrayList();
		copyOnWriteList.add( 1 );
		simpleList.add( 1 );
		Iterator copyOnWriteIterator1 = copyOnWriteList.iterator();
		Iterator simpleListIterator1 = simpleList.iterator();
		f( copyOnWriteList );
		Iterator copyOnWriteIterator2 = copyOnWriteList.iterator();
		Iterator simpleListIterator2 = simpleList.iterator();
		// debug point on the output line just below
		System.out.println(copyOnWriteList);
	}

}

And when we debug into it we see this amazing picture (comments in red on the right):

CopyOnWriteArrayList debug screen

CopyOnWriteArrayList debug screen

As you see the normal non-thread safe ArrayList iterator holds reference to the collection itself. Whereas the clever little CopyOnWrite iterator creates a clone of the underlying array every time a modification is made, so whenever an iterator is called it just gets the latest clone – which doesn’t change throughout the life of the iterator. Rather new clones are created.

Now there is another way of doing this – where we don’t abstract away the reference and therefore when the collection is modified in a function we don’t see that in the caller. But this is not true here and if you let the little example run, you will see it print [1, 2], which is brilliant! 2 is added in the function f.

I should point out that Vector, which is thread safe, looks exactly like the ArrayList here.

For those searching for uniqueness guarantees there is CopyOnWriteArraySet, which uses CopyOnWriteArrayList underneath, so all of the same features hold.

Links

Standard
Java/Maven, Programming

Testing threads with JUnit

Problem
Sometimes there is a need to test a simple piece of multi-threaded code. For example, how will your code behave, when two threads read and then modify one of the variables? Well, if you know your threads you can devise and code the scenario, so that thread access is exactly right. For example, by using a mutex or semaphore. But how do you assert that your test succeeds? I mean besides sitting next to it with a debug session and checking things on every step.

JUnit is a very nice framework, but it does not handle assertions within other threads as well as one would hope it could.

Solution
I will use JUnit 4.0, but this applies to earlier versions as well.

In the sample below we have two test cases, testCaseNaive() will try to do the thing that should work out of the box. However, as far as JUnit is concerned the test succeeds with flashing lights and shooting ribbons. The second test case, testCase2Threads(), uses a somewhat hackish trick. And this trick does involve you creating the two runnables right here in the test.

The idea is quite simple, assertions in JUnit propagate up through the stack by throwing an exception, which is then caught by the test runner. If we get to execute the last statement in the thread as planned – all is well, no assertions were violated, if not – something has gone wrong. And in the case of a fail we rely on the test runner to print out the exception.

// TestMyClass.java

import java.util.ArrayList;
import java.util.Vector;
import junit.framework.Assert;
import org.junit.Test;

public class TestMyClass {

	@Test public void testCaseNaive() throws InterruptedException{
		Runnable runnable1 = new Runnable() {
			@Override public void run() {
				Assert.fail();
			}			
		};
		Thread t1 = new Thread( runnable1 );
		t1.start();
		t1.join();
	}
	
	@Test public void testCase2Threads() throws InterruptedException {
		final ArrayList< Integer > threadsCompleted = new ArrayList< Integer >();
		Runnable runnable1 = new Runnable() {
			@Override public void run() {
				Assert.fail();
				threadsCompleted.add(1);
			}
		};
		
		Runnable runnable2 = new Runnable() {
			@Override public void run() {
				Assert.assertTrue( true );
				threadsCompleted.add(2);
			}
		};
		
		Thread t1 = new Thread( runnable1 );
		Thread t2 = new Thread( runnable2 );
		
		t1.start();
		t2.start();
		t1.join();
		t2.join();
		
		System.out.println( "Threads completed: " + threadsCompleted );
		Assert.assertEquals(2, threadsCompleted.size());
	}
}
// testCaseNaive() passes, but the output still contains this
Exception in thread "Thread-0" junit.framework.AssertionFailedError: null
	at junit.framework.Assert.fail(Assert.java:47)
	at junit.framework.Assert.fail(Assert.java:53)
	at TestMyClass$1.run(TestMyClass.java:13)
	at java.lang.Thread.run(Unknown Source)
// testCase2Threads() fails and the output is:
Exception in thread "Thread-1" junit.framework.AssertionFailedError: null
	at junit.framework.Assert.fail(Assert.java:47)
	at junit.framework.Assert.fail(Assert.java:53)
	at TestMyClass$2.run(TestMyClass.java:25)
	at java.lang.Thread.run(Unknown Source)
Threads completed: [2]

Generally speaking, I have not found a nice way to assert fine detail inside a thread. The exceptions are cut off from the main thread and therefore never reach the runner. So the only way to look at these tests is to test the side effects after the test is complete or in the middle of the execution, if you can stop your threads reliably.

Standard
Java/Maven

Accessing Java Private Attributes and Methods

Java reflection is a very powerful framework.

Sometimes its power seems outside the usual Java methodology.

For example, I needed to manually change a private variable in one of the classes I was testing.

Java reflection has an API that allows this. Of course, the API does not know if one is in a test or production source code. It works anywhere!

Below is an example of three classes.

  • IntWrapper is the class under question.
  • IntWrapperTest is the JUnit class
  • RegularMainClass is a class with a main method, almost a copy of IntWrapperTest.

It is worth noting that the testers and the testee are in separate packages. So one can change the private variable or call a private method of any class in any package!

package com.otherexample;

public class IntWrapper {

	private int i;
	
	public IntWrapper(int i) {
		this.i = i;
	}

	private void incrementI() {
		i++;
	}
	
	public int getI() {
		return i;
	}
	
}
package com.example;

import java.lang.reflect.Field;
import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Method;

import org.junit.Test;

import com.otherexample.IntWrapper;

import static org.junit.Assert.*;

public class IntWrapperTest {

	@Test
	public void testPrivateMembers() {
		try {
			IntWrapper iw = new IntWrapper( 15 );
			assertEquals(15, iw.getI() );
			Field f = iw.getClass().getDeclaredField("i");
		
			f.setAccessible(true);
			f.setInt(iw, 18);
			assertEquals(18, iw.getI() );
			
			Method m = iw.getClass().getDeclaredMethod("incrementI");
			m.setAccessible(true);
			m.invoke(iw);
			assertEquals(19, iw.getI() );
		} catch (SecurityException e) {
			e.printStackTrace();
		} catch (NoSuchFieldException e) {
			e.printStackTrace();
		} catch (IllegalArgumentException e) {
			e.printStackTrace();
		} catch (IllegalAccessException e) {
			e.printStackTrace();
		} catch (NoSuchMethodException e) {
			e.printStackTrace();
		} catch (InvocationTargetException e) {
			e.printStackTrace();
		}
	}
	
}
package com.example;

import java.lang.reflect.Field;
import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Method;

import com.otherexample.IntWrapper;


public class RegularMainClass {

	public static void main(String[] args) {
		try {
			IntWrapper iw = new IntWrapper( 15 );
			if( 15 != iw.getI() ) System.err.println("Assertion failed");
			else System.out.println("all is well 1");
			Field f = iw.getClass().getDeclaredField("i");
		
			f.setAccessible(true);
			f.setInt(iw, 18);
			if( 18 != iw.getI() ) System.err.println("Assertion failed");
			else System.out.println("all is well 2");
			
			Method m = iw.getClass().getDeclaredMethod("incrementI");
			m.setAccessible(true);
			m.invoke(iw);
			if( 19 != iw.getI() ) System.err.println("Assertion failed");
			else System.out.println("all is well 3");

		} catch (SecurityException e) {
			e.printStackTrace();
		} catch (NoSuchFieldException e) {
			e.printStackTrace();
		} catch (IllegalArgumentException e) {
			e.printStackTrace();
		} catch (IllegalAccessException e) {
			e.printStackTrace();
		} catch (NoSuchMethodException e) {
			e.printStackTrace();
		} catch (InvocationTargetException e) {
			e.printStackTrace();
		}
	}

}
Standard
Java/Maven

Ignoring Java Generics Safety

Java Generics (something like List <String>) provides a very nice way to group similar operation on different types of objects.

There is a way around it though.

In C++ the template invocation actually causes a new instance of the underlying type to be created and compiled. Java only creates the one type instance and then uses the generics as a variable when performing the type checks. Generics allow the compiler to enforce more type safety checks.

More type safety is a brilliant idea and I really like generics, so I have no practical use for this work around to fool the compiler safety checks. Let me know if you find one.


import java.util.ArrayList;
import java.util.Collection;

public class Q1{

	public static void main( String[] args ) {
		
		ArrayList<Integer> is = new ArrayList<Integer>();
		// list is [  ]
		
		is.add(2);
		// list is [ 2 ]
		
		Collection<?> ac = is;
		ac.add( null );
		
		// null is the only value that can be added to the ac list.
		// list is [ 2, null ]		
		
		(( ArrayList<String> ) ac).add("andrejserafim");
		
		// the above will give you an unchecked cast warning, 
		// but will compile and work
		
		// list is [ 2, null, "andrejserafim" ]
		
	}

}

Initially the list is tagged with ArrayList<Integer>.

The cast to Collection<?> wipes off that tag.

Then ArrayList<String> cast writes a new tag onto the list now stating it’s a list of strings.

Standard
Latex

IEEETran page margins

This post has nothing to do with getting papers ready for IEEE, it is just using the template they provide for something else, like a University coursework.

IEEEtran_HOWTO says:

Authors should not attempt to manually alter the margins, paper size …

But sometimes one just has to. And then using packages like ‘geometry’ does not help – they confilict with IEEE settings and mess up the entire layout.

I use a4paper option. IEEETran was designed for Letter paper. To make things compatible the desginers made it so the layout will look exactly the same if one used A4 or Letter.

This is great, unless you want only A4 – then the paper has a huge bottom page margin. Which is not birlliant if you have a page limit and is already over it.

Fixing this appeared to be quite easy. Just add:

\newcommand{\CLASSINPUTtoptextmargin}{2cm}
\newcommand{\CLASSINPUTbottomtextmargin}{2cm}
\documentclass[journal, a4paper]{IEEEtran}
.....
\begin{document}
.....
\end{document}

Inspired by this FAQ

Standard
Web (PHP, Javascript)

Javascript arguments by value

Let us consider the following situation: one has several widgets, which have to be loaded onto the page dynamically, but each can take a while to load. A solution is to do the loading asynchronously for each component with a nice progress bar displayed to the user.

For example, SIMILE Timeline uses the asynchronous approach to load the data for historical timelines.

After the timelines have been configured, one has to load the data into them using loadXML function.

var eventSource = new Timeline.DefaultEventSource( 0 );
var timeline = Timeline.create(td, bands, Timeline.HORIZONTAL);
timeline.loadXML( eventSourceURL, function( xml, url ) { 
  eventSource.loadXML( xml, url );
} );

The setup seen above works perfectly for a single timeline. But what if we have a loop to create a few timeline widgets?

var dataSources = [ 
  "https://andrejserafim.wordpress.com/?data=1", 
  "https://andrejserafim.wordpress.com/?data=2", 
  "https://andrejserafim.wordpress.com/?data=3" ];

for( var ds in dataSources ) {
  var eventSourceURL = dataSources[ ds ];
  var eventSource = new Timeline.DefaultEventSource( 0 );
  // associate eventSource with timeline through _bands_ configuration
  var timeline = Timeline.create(td, bands, Timeline.HORIZONTAL);
  timeline.loadXML( eventSourceURL, function( xml, url ) { 
    eventSource.loadXML( xml, url );
  } );
}

The result is not anything one could predict, all 3 timelines will have the data from the 3rd data source or no data at all. This happens because the data is loaded at some point after the execution of the code presented has finished. When the anonymous function is created it remembers the reference to the eventSource variable in this loop.

For some reason the context of execution in terms of that one variable is the same for all three timelines.

Fortunately, one can supply a default value to function arguments in Javascript. So we could freeze the reference to eventSource in an array and pass a key as a parameter to the function. Let me show you what I mean:


var dataSources = [ 
  "https://andrejserafim.wordpress.com/?data=1", 
  "https://andrejserafim.wordpress.com/?data=2", 
  "https://andrejserafim.wordpress.com/?data=3" ];

// we are going to distinguish each eventSource
var eventSources = [];

for( var ds in dataSources ) {
  var eventSourceURL = dataSources[ ds ];
  // storing all eventSource values for future asynchronous use.
  eventSources[ ds ] = new Timeline.DefaultEventSource( 0 );
  // associate eventSources[ ds ] with timeline through _bands_ configuration
  var timeline = Timeline.create(td, bands, Timeline.HORIZONTAL);
  // notice the three arguments and the defaults.
  timeline.loadXML( eventSourceURL, function( xml, url, ds ) { 
    eventSources[ ds ].loadXML( xml, url );
  }.defaults( ds ) );
}

Now each anonymous function is associated with its own eventSource. And we get three distinct timelines.

One might notice that the defaults construct is not standard Javascript. It is described in more detail here. And can be added by inserting this code somewhere in the loading section of your page:


Function.prototype.defaults = function()
{
  var _f = this;
  var _a = Array(_f.length-arguments.length).concat(
    Array.prototype.slice.apply(arguments));
  return function()
  {
    return _f.apply(_f, Array.prototype.slice.apply(arguments).concat(
      _a.slice(arguments.length, _a.length)));
  }
}

Standard