Table of Contents

---

Comparison Viewer Overview

The Comparison Viewer workspace in MBAT3.0 is built on the concept of layers which can be composited together. Each layer handles the loading and displaying of data types, such as 2D images, 3D image volumes, 3D atlases, and 3D surfaces. The user will be allowed to create an arbitrary number of layers to composite together. The user will be able to interactively hide or show and change the opacity of each layer. Depending on the data type being displayed, the user will be able to interactively adjust different properties for that layer, such as brightness and contrast, image plane, orientation, and rotation.

mbViewer Architecture

The underlying mbViewer architecture of the Comparison Viewer workspace is an extensible design built on a scalable plugin architecture. It consists of a core framework (dlGraphics, dlJGraphics) and tool plugins (mbViewerPlugins), as shown in the figure below.

Framework

The mbViewer API framework consists of 2 main modules - dlGraphics that handles the low-level graphics rendering for each layer and dlJGraphics that defines the common class interface for loading data and handling the higher-level Java/Swing interaction. Based on this framework, the mbViewerPlugins implement the actual loading of specific data types into the layer, customize the handling of user events, and implement the GUI controls for the layer.

• dlGraphics - Handles all graphics objects (ie: textures) and rendering to the canvas using the OpenGL API.
• dlJGraphics - Defines the class interface for loading of data into OpenGL objects. Also defines the abstract classes to handle all user events (mouse clicks, key strokes) and GUI objects for the dlLayers using Java/Swing API.
• mbViewerPlugins - implements and specializes the dlJGraphics interfaces for particular data types.

---

mbViewer API

---

dlGraphics

---

• dlLayer: interface for all layers
  • memory management
    • dispose( GL gl )
  • layer properties
    • setName( String sName )
    • setIsVisible()
    • setOpacity( float fOpacity)
  • rendering
    • display( GL gl )
    • drawObjectID( GL gl )
    • drawOverlay( GL gl )
    • drawStatusBar( GL gl )
  • saving/restoring state
    • serialize()
    • deserialize()
  • link layers
    • getLinkedObjectList()
    • clearAndUpdateLinkedObjects()
• dlGLTexture2D: class for representing image of arbitrary size
  • create texture
    • dlGLTexture2D( TextureData )
  • get dimensions
    • getHeight()
    • getWidth()
  • dynamically update image data
    • updateImage( TextureData )
  • access subimage data
    • getNumSubImages()
    • getSubImage( int )
• dlLayerImage2D: class for handling 2D images; contains multiple axes where each axis has a list of textures
  • add texture to axis (appends to end of list of textures)
    • addTexture( int nAxis, dlGLTexture2D )
    • addTexture( int nAxis, TextureData? )
  • set current axis/texture to display
    • setActiveAxis( int nAxis )
    • setActiveTexture( int nIndex )
    • nextTexture()
    • prevTexture()
  • get axis/texture
    • getActiveAxis()
    • getActiveTexture()
    • getActiveTextureIndex()
    • getTexture( int nAxis, int nIndex )
  • set image transformation properties (absolute and relative)
    • setTextureRotation( float fAngle )
    • setTextureScale( float x, float y )
    • setTextureTranslation( float dx, float dy )
    • setTextureRotationRel( float fAngle )
    • setTextureScaleRel( float x, float y )
    • setTextureTranslationRel( float dx, float dy )
  • set LUT (look up table)
    • setTransferFunction( GL gl, ColorModel cm)

---

dlJGraphics

---

• JLayer: interface to load objects into GL objects and handle user events
  • interface to load objects:
    • load( GL gl, mbObject obj )
  • interface to create GUI layer
    • JLayerGUI createLayerGUIFactory()
  • interface to get last transformation (for applying same transformation to all viewports)
    • getLastTransformation()
  • interface to handle user events
    • keyPressed( KeyEvent e )
    • mouseDragged( MouseEvent e, MOUSEMODE nMouseMode, int dx, int dy, int winx, int winy, int viewportx, int viewporty )
    • mousePressed( MouseEvent e, MOUSEMODE nMouseMode, float fRelX, float fRelY, int winx, int winy )
    • mouseWheelMoved( MouseWheelEvent e )
• JLayerGUI: base class that contains GUI controls for layer visibility and expanding properties control
  • update GUI/layer
    • updateValues( boolean bUpdateFromDataModel)
  • JPanel for customized Swing controls
    • getPanelProperties()

---

mbViewerPlugins

---

• net.nbirn.mbat.plugins.viewer.JLayerImage2D
  • class JLayerImage2D extends dlLayerImage2D implements JLayer
    • loads 2D images from file, Image2D search result, Image2DSeries search result, and BufferedImage
    • creates JLayerGUI2D for layer
  • class JLayerGUI2D extends JLayerGUI
    • creates GUI to control layer opacity, image slice, axis, brightness, contrast, flip

• net.nbirn.mbat.plugins.viewer.JLayerImage3D
  • class JLayerImage3D extends JLayerImage2D
    • load image volume data from analyze image file into memory
    • dynamically streams image data for active axis/texture from memory
    • renders axes thumbnails as overlay
    • creates JLayerGUI2D for layer

• net.nbirn.mbat.plugins.viewer.JLayerImage3DAtlas
  • class JLayerImage3DAtlas extends JLayerImage3D
    • load image volume and label data from analyze image file into memory
    • loads label file (.bgx,.ilf,.obo) into graph structure
    • dynamically streams image data for active axis/texture from memory
    • renders axes thumbnails as overlay
    • creates JLayerGUI3DAtlas for layer
    • on mouse over, displays the name of the label
    • on mouse left double-click, selects the label

---

Linking of layers

Layers can be linked together so an action performed on one layer is mirrored in another. For example, moving linked layers together or co-navigating multiple layers. When linking layers, the master layer is the currently active layer or the layer to which the action is being directly applied. The slave or linked layers are the layers that will mirror the master action.

The general design is to recursively call the corresponding event listener (ie: mouse or key listener events) for each layer. Each dlLayer maintains a list of its linked layers that can be obtained by calling getLinkedObjectList(). Implementing linked layers differs slightly based on the behavior you want to accomplish and the class from which your plugin is derived.

JLayerImage2D subclasses

For events where the layers respond identically to the input (ie: key press), you simply recursively call the listener (Example 1). For events where the layers respond differently (ie: based on mouse position) but you want to force the layers to respond identically, you can explicitly iterate over the linked layers and set their state (Example 2).

    // Example 1: simply pass events to linked layers
    // This is useful when all layers respond identically to the event
    public void mousePressed ( ... )   
    {
        // respond to event...

        // recursively call listener
        super.mousePressed(...);
    }

    // Example 2: trigger some behavior based on the master layer
    // This is useful when the slave layers wouldn't normally respond identically to the event
    public void mousePressed ( ... )   
    {
        // if this layer is the active/master layer
        if( getIsActive() )
        {
            // compute state...
            state.setState1(...);
            state.setState2(...);
            ...

            // foreach linked layer
            for( dlCLinkedObject linkedObject : getLinkedObjectList() )
            {
                if( linkedObject != this && linkedObject instanceof JLayerImage2DSubClass )
                {
                    // explicitly apply master state to slave/linked layer
                    ((JLayerImage2DSubClass )linkedObject).setState( state );
                }
            }
        }
        // recursively call listener so derived classes' mousePressed() will be called
        super.mousePressed(...);
   }

dlLayer subclasses

If your plugin derives from dlLayer and it can be extended by other classes, you need to define the recursive event listener calls in your class. In order to avoid infinite recursion, you need to wrap the recursive calls in a getIsActive() block, as shown below (see the JLayerImage2D mouse and key listener methods for more examples):

    // Example 3: for classes derived from dlLayer, implement the recursive call
    public void mousePressed ( ... )   
    {
        // if this layer is the active/master layer
        if( getIsActive() )
        {
            // respond to event
            ...

            // foreach linked layer
            for( dlCLinkedObject linkedObject : getLinkedObjectList() )
            {
                if( linkedObject != this && linkedObject instanceof MyClass )
                {
                    // recursively call event listener on linked layers
                    ((MyClass)linkedObject).mousePressed(...);
                }
            }
        }
    }

JLayerGUI subclasses

If you want the linked layers to respond to the GUI controls, you need to retrieve the JLayerGUI object for all the linked layers. Since linked layers can exist across viewports, you must search all viewports for the JLayerGUI. Once you have the slave JLayerGUI objects, you can then set the state of the slave GUI controls to match those of the master. You must also find the table row of the slave JLayerGUI object so you can issue a JTable update, as shown below:

    public void itemChanged ( ItemEvent e )
    {
       // process item event...
       int nValue = ...;
       m_SliderSpinnerValue.setValue( nValue );

       //----------------------------------------------------
       // link layers
       //----------------------------------------------------
       // if active (master) layer
       if( getLayer().getIsActive() && getTableLayers() != null )
       {
           // foreach linked object
           for( dlCLinkedObject linkObject : getLayer().getLinkedObjectList() )
           {
               // if not itself
               if( getLayer() != linkObject )
               {
                   // get the GUI object for this layer (search all viewports)
                   JLayerGUI layerGUI = getViewportManager().getJLayerGUI( (dlLayer)linkObject );
                  
                   // get table row index for layer
                   int nRow = getTableLayers().getLayerRowIndex( (dlLayer)linkObject );
                   // set state
                   if( layerGUI instanceof JLayerGUI2D )
                   {
                       // update only if enabled
                       if( ((JLayerGUI2D)layerGUI).getSliderSpinnerValue().isEnabled() )
                       {                     
                           ((JLayerGUI2D)layerGUI).getSliderSpinnerValue().setValue( nValue );
                           // update table rows (if the linked layer is in another viewport, don't update the table)
                           if( nRow != -1 )
                               getTableLayers().updateRows( nRow, nRow );
                        }
                    }
                }
            } 
        }
    }

Other notes

The mbViewer core will update the layer's linked object list whenever a new layer is linked, unlinked, or deleted. To force a layer update, you can call clearAndUpdateLinkedObjects(). However, since this method recursively searches through the linked layers, it is advisable to avoid calling this method in any performance/rendering dependent code.

---

DarenLee^? - 22 Mar 2009