Forum Controls
Spotlight Features
Replies: 12 - Pages: 1  
  Click to reply to this thread Reply

A Look At The Java Class Loader

At 8:03 AM on Oct 15, 2005, Kumar Matcha wrote:

Class loading is one of the most powerful mechanisms provided by the Java Language Specification. All Java programmers should know how the class loading mechanism works and what can be done to suit their needs. By understanding the class loading mechanism you can save time that would otherwise be spent on debugging ClassNotFoundException , ClassCastException , etc.

Class Loaders

In a Java Virtual Machine (JVM), each and every class is loaded by some instance of a java.lang.ClassLoader. The ClassLoader class is located in the java.lang package and you can extend it to add your own functionality to class loading.

When a new JVM is started by java HelloWorld, the bootstrap class loader is responsible for loading key java classes like java.lang.Object and other runtime code into memory. The runtime classes are packaged inside jre/lib/rt.jar file. We cannot find the details of the bootstrap class loader in the java language specification, since this is a native implementation. For this reason the behavior of the bootstrap class loader will differ across JVMs.

Maze Behind Class Loaders

All class loaders are of the type java.lang.ClassLoader. Other than the bootstrap class loader all class loaders have a parent class loader. These two statements are different and are important for the correct working of any class loaders written by a developer. The most important aspect is to correctly set the parent class loader. The parent class loader for any class loader is the class loader instance that loaded that class loader.

We have two ways to set the parent class loader:

public class CustomClassLoader extends ClassLoader{

public CustomClassLoader(){
super(CustomClassLoader.class.getClassLoader());
}
}

Or

public class CustomClassLoader extends ClassLoader{

public CustomClassLoader(){
super(getClass().getClassLoader());
}
}

The first constructor is the preferred one, because calling the method getClass() from within a constructor should be discouraged, since the object initialization will be complete only at the exit of the constructor code. Thus if the parent class loader is correctly set, whenever a class is requested out of a ClassLoader instance using loadClass(String name) method, if it cannot find the class, it should ask the parent first. If the parent cannot find the class, the findClass(String name) method is invoked. The default implementation of findClass(String name) will throw ClassNotFoundException and developers are expected to implement this method when they subclass java.lang.ClassLoader to make custom class loaders.

Inside the findClass(String name) method, the class loader needs to fetch the byte codes from some arbitrary source. The source may be a file system, a network URL, another application that can spit out byte codes on the fly, or any similar source that is capable of generating byte code compliant with the Java byte code specification. Once the byte code is retrieved, the method should call the defineClass() method, and the runtime is very particular about which instance of the ClassLoader is calling the method. Thus if two ClassLoader instances define byte codes from the same or different sources, the defined classes are different.

For example, lets say Ive a main class called MyProgram. MyProgram is loaded by the application class loader, and it created instances of two class loaders CustomClassLoader1 and CustomClassLoader2 which are capable of finding the byte codes of another class Student from some source. This means the class definition of the Student class is not in the application class path or extension class path. In such a scenario the MyProgram class asks the custom class loaders to load the Student class, Student will be loaded and Student.class will be defined independently by both CustomClassLoader1 and CustomClassLoader2. This has some serious implications in java. In case some static initialization code is put in the Student class, and if we want this code to be executed one and only once in a JVM, the code will be executed twice in the JVM with our setup, once each when the class is separately loaded by both CustomClassLoaders. If we have two instances of Student class loaded by these CustomClassLoaders say student1 and student2, then student1 and student2 are not type-compatible. In other words,

Student student3 = (Student) student2;

will throw ClassCastException, because the JVM sees these two as separate, distinct class types, since they are defined by different ClassLoader instances.


The Need For Your Own Class loader

For those who wish to control the JVMs class loading behavior, the developers need to write their own class loader. Let us say that we are running an application and we are making use of a class called Student. Assuming that the Student class is updated with a better version on the fly, i.e. when the application is running, and we need to make a call to the updated class. If you are wondering that the bootstrap class loader that has loaded the application will do this for you, then you are wrong. Javas class loading behavior is such that, once it has loaded the classes, it will not reload the new class. How to overcome this issue has been the question on every developers mind. The answer is simple. Write your own class loader and then use your class loader to load the classes. When a class has been modified on the run time, then you need to create a new instance of your class loader to load the class. Remember, that once you have created a new instance of your class loader, then you should make sure that you no longer make reference to the old class loader, because when two instances of the same object is loaded by different class loaders then they are treated as incompatible types.

Writing Your Own Class loader

The solution to control class loading is to implement custom class loaders. Any custom class loader should have java.lang.ClassLoader as its direct or distant super class. Moreover you need to set the parent class loader in the constructor. Then you have to override the findClass() method. Here is an implementation of a custom class loader.



import java.io.DataInputStream;
import java.io.File;
import java.io.FileInputStream;
import java.io.IOException;
import java.util.Enumeration;
import java.util.Hashtable;
public class CustomClassLoader extends ClassLoader {
    public CustomClassLoader(){
        super(CustomClassLoader.class.getClassLoader());
    }
  
    public Class loadClass(String className) throws ClassNotFoundException {
         return findClass(className);
    }
 
    public Class findClass(String className){
        byte classByte[];
        Class result=null;
        result = (Class)classes.get(className);
        if(result != null){
            return result;
        }
        
        try{
            return findSystemClass(className);
        }catch(Exception e){
        }
        try{
           String classPath =    ((String)ClassLoader.getSystemResource(className.replace('.',File.separatorChar)+".class").getFile()).substring(1);
           classByte = loadClassData(classPath);
            result = defineClass(className,classByte,0,classByte.length,null);
            classes.put(className,result);
            return result;
        }catch(Exception e){
            return null;
        } 
    }
 
    private byte[] loadClassData(String className) throws IOException{
 
        File f ;
        f = new File(className);
        int size = (int)f.length();
        byte buff[] = new byte[size];
        FileInputStream fis = new FileInputStream(f);
        DataInputStream dis = new DataInputStream(fis);
        dis.readFully(buff);
        dis.close();
        return buff;
    }
 
    private Hashtable classes = new Hashtable();
}</i>
 
 
Here is how to use the CustomClassLoader. 
 
<i>public class CustomClassLoaderTest {
 
     public static void main(String [] args) throws Exception{
        CustomClassLoader test = new CustomClassLoader();
        test.loadClass(com.test.HelloWorld);
     }
}


Summary

Many J2EE application servers have a  hot deployment  capability, where we can reload an application with a new version of class definition, without bringing the server VM down. Such application servers make use of custom class loaders. Even if we dont use an application server, we can create and use custom class loaders to fine-control class loading mechanisms in our Java applications. So have fun with custom loaders.

1 . At 3:41 PM on Oct 15, 2005, Matt Casters DeveloperZone Top 100 wrote:
  Click to reply to this thread Reply

Whatch out with Singletons

Just a word of caution if you're building your own classloaders and use singleton design in your programs.

It cost me *some* debugging time at one point.

Here's what happed: In my tool Kettle, I'm using plugins to load custom made or 3rd party transformation steps. Each plugin has its own (custom) classloader.
Now that's alright, but remember that if you use sigleton designs yourself (or in a jar loaded by the classloader), then those singletons are only singleton for every unique instance of the classloader!!!
I didn't know this and just created new classloader instances every time I needed one. A certain singleton was not singleton and kept getting created every time again. (once you know this, you can google for it and find it ;-))

To solve it I ended up writing a hashmap to keep track of the created classloaders (with associated singltons)... you gessed it in a singleton.

Oh well, Spring next time hu?

Just my ¬.02!

Cheers,

Matt
Matt Casters on Data Integration and Kettle
2 . At 8:42 PM on Oct 15, 2005, Paul Speed wrote:
  Click to reply to this thread Reply

Re: Whatch out with Singletons

Parent classloaders can contain classes used by the child classloaders. If you need to share a singleton among multiple child class loaders then the easiest way is to make sure the class gets loaded in the parent loader.

Not sure if that would have worked in your situation but it seems like it might have.
-Paul
3 . At 4:31 AM on Oct 16, 2005, Matt Casters DeveloperZone Top 100 wrote:
  Click to reply to this thread Reply

Re: Whatch out with Singletons

Well, my view on it is that if you're loading plugins and that you want to keep the loaded classes seperated.
I only want to have a single plugin "see" the base-classes, NOT the other plugins.
I was afraid that if I did it like you suggested, I would end up in dependecy hell in the long run.

Cheers,
Matt
Matt Casters on Data Integration and Kettle
4 . At 5:39 PM on Oct 16, 2005, Anthony Goubard wrote:
  Click to reply to this thread Reply

Re: A Look At The Java Class Loader

Indeed, the ClassLoader class is propably the most powerful class of the JDK.

Here are a few interesting one I've been playing with.

- ChildFirstClassLoader
I used this class loader for the Servlet container I've written.
The ClassLoaders included in the JDK will try to load the classes using the parent ClassLoader before your ClassLoader. I had with it a conflict with Log4j classes which were loaded by another ClassLoader (the parent) than the Servlet which was using Log4j.

- JDistro ClassLoader
This one is quite funny, it replaces some of class it should normally load by other classes.
e.g. java.awt.Window -> xxx/awt/XWindow where XWindow is an javax.swing.InternalFrame.

- Ant ClassLoader
This one will load the given JAR files. Very useful if you have a plug-in structure for your application like I have for Ant Commander. You have also a way to specify if you want it as a parent first or child first ClassLoader.

That's it!
Never forget to use the ClassLoader as early as possible in your program and load the first class with it using the reflection API to avoid conflicts.
Anthony Goubard
JLearnIt, Ant Commander, XINS, japplis.com.
5 . At 10:42 PM on Oct 16, 2005, Kevin wrote:
  Click to reply to this thread Reply

Re: A Look At The Java Class Loader

A very simple yet powerful custom classloader is to extend URLClassLoader but make public its URL[] classpath via addUrl(). This allows you to add at runtime jar/zip files and directories to search for classes within dynamically.

Another thing to worry about, which another poster talked about is the current classloader design always looks for classes up the parent chain. If you compile a given set of code loaded by a custom classloader against a certain version of a library, and that library is loaded up the chain but a different version of it is, you can get yourself into runtime nightmares when deploying a "plugin" (custom loaded classes) at runtime or loading it for the first time. To solve this, simply control the loadClass() method by first checking the classpath by calling findClass() first, then resorting to the parent loader.

Something very kewl and powerful with your custom classloaders is the use of delegation yourself. Most developers will call the super.loadClass() or super.findClass() call and if, like my previous paragraph states, they get clever and look in the classpath of the classloader first, then go to the parent, you've effectively changed the way the classloader looks/finds and loads classes. IF you take that one step further, you can delegate the finding of classes to any number of classloaders. For example, a custom "shared" classloader in the case of a web application is the Application Context loader. By having a shared classloader by making it the parent to all custom classloaders, you effectively have a location that your custom classloaders can search for shared libraries and classes. Suppose you have two (or more) plugins loading and you have some logic that determines they have the same library (perhaps a compatible library version, or the exact same version). By adding the URL of that library to the shared classloader classpath, both (or more) plugins can now share the library without creating individual instances of classes within it. The shared loader has the application classloader as its parent, and all custom plugin classloaders use the shared loader as its parent. Each plugin classloader looks in its classpath first, then delegates to the parent loader (the shared one in this case). OR, you can even make the shared loader a singleton instance (loaded by the application classpath) and call to it directly from your custom classloader, not making it a parent, but instead allowing each custom classloader to check a flag if it should use the shared loader or not. If a custom classloader doesn't need to, why make an extra call to the parent? In other words, you can add logic easily that can determine things like "Did I have a library that is shared, so should I make a call to check for a class in the shared loader, or not waste the extra method call and go right to my parent". By making it a singleton and not a parent to a custom classloader, you can now avoid extra method calls to the shared library if a certain class doesn't need to use it.

You can take a look at our project, www.platonos.org. The plugin engine sub project has a pretty tricked PluginClassLoader that does a number of things. We deal with dependencies, threaded plugin activation, lazy activation of plugins, as well as support a bundled plugin format we call PAR (Plugin ARchive) that allows you to bundle a plugin includined embeded jar/zip libraries the plugin depends on in a single file. We do on the fly extraction of classes embeded in jar/zip files that are bundled with the plugin, as well as support local expanded disc based plugins. We have the starting of unload/reload capabilities, however because of the use of dependencies, there are some snafu's that make unloading of plugins (by unload we mean true GC of all plugin classes so the bytecode is actually removed from the JVM) very difficult in the more complex plugin scenarios. However, it does work in the more simpler plugins, those that simply implement an interface of another plugin. Even so, the process of unload/reload at runtime in a GUI app is daunting. It's possible, but if one GUI plugin unloads and a chain reaction starts happening, you could see the GUI sort of disappear then reappear, not something anyone wants to see in their professional Java GUI app.
6 . At 3:32 AM on Oct 17, 2005, Gregor Kovac wrote:
  Click to reply to this thread Reply

Re: A Look At The Java Class Loader

Hi!

I have a server that should also support hot deployment.
I have a main class calles MainServer that is loaded by my ServerClassLoader. So every class thats is created in Server class via new operator (MyClass a = new MyClass()) is also loaded by the same ServerClassLoader.
When the version of a class (MyClass for example) changes I'd like to call method so that every loaded class is discarded and new ones start to load. I'm not sure how to do that.
Any help?
7 . At 4:21 AM on Oct 17, 2005, Kamil Shamgunov wrote:
  Click to reply to this thread Reply

Re: A Look At The Java Class Loader

If someone interested in tricks with classloading, you can see JPackit project sources.
JPackit can pack mostly any application with it's dependencies into single jar file and then use
some classloading tricks to load and launch it.
http://www.sourceforge.net/projects/jpackit/
8 . At 8:08 AM on Oct 17, 2005, Serge Bureau DeveloperZone Top 100 wrote:
  Click to reply to this thread Reply

Re: A Look At The Java Class Loader

Thank you for the examples.

One that I used. the NetworkClassLoader is similar to the one fron Sun but with the capability to remove classes.

http://wotonomy.sourceforge.net/docs/net/wotonomy/util/NetworkClassLoader.html
9 . At 8:15 AM on Oct 17, 2005, Serge Bureau DeveloperZone Top 100 wrote:
  Click to reply to this thread Reply

Re: A Look At The Java Class Loader

> Hi!
>
> I have a server that should also support hot
> deployment.
> I have a main class calles MainServer that is loaded
> by my ServerClassLoader. So every class thats is
> created in Server class via new operator (MyClass a =
> new MyClass()) is also loaded by the same
> ServerClassLoader.
> When the version of a class (MyClass for example)
> changes I'd like to call method so that every loaded
> class is discarded and new ones start to load. I'm
> not sure how to do that.
> Any help?

Usually a ClassLoader keeps a collection of the class he loaded, you have to make sure this list contains no more reference to the class. You of course also have to make sure nobody else has a reference to the class.

If your classloader has private access to the collection, it must be changed or provide a protected or public accessor to it.
10 . At 1:24 AM on Oct 18, 2005, Steve Jakob wrote:
  Click to reply to this thread Reply

Re: A Look At The Java Class Loader

My first reaction when I read your message was, why worry about it? If only MyClass has changed, why shouldn't it make use of the other classes already cached in your classloader?

If you anticipate having several interdependant classes changing though, I'd say it depends on how the classloader figures out when it needs to reload classes:

a) if you're checking against some sort of timestamp or version code each time a class is accessed, other classes will get reloaded anyway. I suspect this sort of architecture could impose a significant overhead, though.

b) if you're signalling your classloader of the need for a reload (perhaps as a result of some sort of periodic "freshness check" on the JAR(s) from which the class was loaded) you could just dump all cached classes and reload as they're needed. As Serge noted, classloaders generally track the classes they've loaded.

Objects which have already been created will obviously refer to the original class, keeping a reference to it. If you want to ensure that ONLY the new class is used you'll need to have a way to track these instantiated objects and dump/recreate them when required.

Steve.
11 . At 3:34 PM on Oct 20, 2005, Tom Dyer wrote:
  Click to reply to this thread Reply

Re: A Look At The Java Class Loader

The best, and most overlooked, book on classloaders and other topics is "Component Development for the Java Platform" by Stuart Halloway. Everything you every wanted to know about classloaders, and more.

http://www.relevancellc.com/main/resources#7
12 . At 10:42 AM on Mar 14, 2006, lukasjob wrote:
  Click to reply to this thread Reply

Re: A Look At The Java Class Loader

hi!

question: is the contents reliable? .. since that book ('Component Development for the Java Platform') was published in 2001 ..

thanks

im really trying to find a good text about class
loaders if anyone knows one please suggest
thanks

thread.rss_message