Saturday, September 19, 2009

Challenges in software development

As a developer I find writing code as the easiest part of my job as at most times I have total control over it. But working on complex software poses a number of challenges and writing code can sometimes be the last thing I am worried about.

The first challenge and probably the most irritating one is 'Marking the territory'. I hate to hear 'its my component', 'I know the best way to deal with it' , ' you worry about your component' and other variants of it. I guess its human behavior to be a bit protective of your work, but at times it goes out of hand. More often than not this curbs the flow of fresh ideas in the team.

The second challenge and an equally irritating one is 'Passing on the buck'. Now if you are working on a complex software, you will definitely be able to solve a particular problem in many ways (meaning in many components). However there is generally a best solution. But people have a tendency to pass on the buck to some other component by saying 'let them deal with this problem'. I hate it when this happens without reason. A problem passed on to other component must always be accompanied by a detailed explanation of why the problem has to be solved by the other component.

'Absolutely ridiculous schedules' is something that has me stumped most times and has also helped me in improving my negotiation skills :). Software delivery schedules seem to designed by people who have no idea of what it takes to develop software. Ok the last statement is bit of an exaggeration but there is always a tussle between management and developers. Management wants things to be ready day before yesterday, and developers need another year to write a Hello World program (ok exaggerating again :P ). Different technologies require different amount of effort, for instance writing Java code in Eclipse is a much easier job than writing an equal amount of JavaScript. People coming up with schedules need to understand these differences. In my opinion the management needs to sit with developers every once in a while and understand the various challenges they are facing with the technology involved. Or a better option is to make it mandatory for managers to write or test code (not a whole lot but at least some contribution). But probably the best option would be to teach developers to manage things in addition to developing software. I have seen a few people who are successful with the last option, but mostly its a tussle!

'Shouting from rooftops near a deadline' - Some folks have a tendency to create a lot of noise near a deadline to make it appear that without them everything will fall apart. Whenever this happens I always think 'you could have at least whispered a bit earlier and maybe we would have lesser problems to deal with now'.

Most of these challenges will vanish if only people are open to new ideas, do the right thing and have trust in others to do the right thing. If only if....

Saturday, September 12, 2009

Method for dealing with temporary files in a java server program

Method for dealing with temporary files in a server program

Problem

To handle temporary files Java provides two mechanisms

- File class provides a set of createTempFile methods which ensure that each file gets a unique name

- File class has a method deleteOnExit method which ensures that the file gets deleted when the JVM terminates.

But these methods prove insufficient while dealing with temporary files in a (java) server process. The server process (jvm) runs for a long time before crashing/restart, hence files cannot be cleaned only on process (jvm) start or process (jvm) exit, it has to be done regularly so that the files do not accumulate on the hard-disk.

Scenarios when manual deletion of files is not possible

  • An image file is referenced by html sent back to the web-browser. There is no way of knowing when the image file has been requested and downloaded by the web-browser.
  • A file cannot be deleted manually because an input-stream is opened on the file and passed to an asynchronous API.

Possible Solutions

  • File system itself to provide a file expiry attribute
  • Java to provide a file expiry attribute

The above 2 are not available

Proposed Solution

Provide a file expiry time while creating a temp file. All such files are marked to be deleted when the expiry time is over and can be deleted in an independent thread running on the server.


Sample Implementation

public class TempFile extends File {

private Date d = null;

/*

* The default expiry time is 1 day

*/

private Date getDefaultExpiry() {

Calendar c = Calendar.getInstance();

c.setTime(new Date());

c.add(Calendar.DATE, 1);

return c.getTime();

}

/*

* Constructor from super class

*/

public TempFile(String pathname) {

super(pathname);

setExpiry(getDefaultExpiry());

}

/*

* Constructors with expiry parameter

*/

public TempFile(String pathname, Date expiry) {

super(pathname);

setExpiry(expiry);

}

/*

* Let the user set the expiry attribute after construction of object

*/

public void setExpiry(Date d) {

this.d = d;

FileGarbageCollector collector = FileGarbageCollector.getInstance();

collector.addFile(this);

}

}

public class FileGarbageCollector implements Runnable {

private static FileGarbageCollector ref = null;

Thread t = null;

public static FileGarbageCollector getInstance() {

if (ref == null) {

ref = new FileGarbageCollector();

}

return ref;

}

/*

* The files for which an expiry time is set.

*/

private static List files = null;

private FileGarbageCollector() {

files = new ArrayList();

}

public void addFile(TempFile f) {

files.add(f);

if (t == null || !t.isAlive()) {

t = new Thread(ref);

t.start();

}

}

public void run() {

while (files.size() > 0) {

Date d = new Date();

for (int i = 0; i < files.size(); i++)

if ( d.getTime() > files.get(i).getExpiry().getTime() ) {

TempFile t = files.remove(i);

i--;

t.delete();

System.out.println(t.getPath() + " deleted at "

+ new Date());

}

}

try {

Thread.sleep(8000);

} catch (InterruptedException e) {

e.printStackTrace();

}

}

}

}