LONI: Laboratory of Neuro Imaging | Resources | Computing
LONI > About LONI > Resources

Computing Resources

Rapid advancements in imaging technology have provided researchers with the ability to produce very high-resolution, time-varying, multidimensional datasets of the human brain. Population-based longitudinal studies using this data drive a continually-increasing demand for compute power.

COMPUTATIONAL SERVERS

SUN MICROSYSTEMS V20z CLUSTER & X2200 M2 CLUSTER
Today, LONI utilizes a 306-node, dual-processor SUN Microsystems V20z cluster, one of the largest V20z installations in the world.  Each V20z compute node has dual 64-bit 2.4 gigahertz AMD Opteron 250 CPUs with 8 gigabytes of memory. LONI has additionally integrated an 80-node, eight-processor SUN Microsystems X2200 M2 installation. Each X2200 M2 has eight 64-bit 1.1 gigahertz AMD Opteron 2354 cores with 16 gigabytes of memory.

DELL DEVELOPMENT CLUSTER
In addition to the SUN clusters, LONI has a 64-node Dell development cluster, with each node using dual 64-bit 3.6 gigahertz Intel EM64T processors and 4 gigabytes of memory.

SGI SUPERCOMPUTERS
To augment the facility's cluster resources, LONI has a 64-processor SGI Origin 3800 SMP supercomputer with 32 gigabytes of memory. A comparable 32-processor SGI Onyx2 Reality Monster with 16 gigabytes of memory and a 6-processor SGI Onyx2 with 8 gigabytes of memory SMP supercomputers are utilized to drive graphics-intensive applications and interactive real-time multidimensional visualization of structural brain models and volumetric datasets.

GRID MANAGEMENT

LONI GRID MONITOR VISUALIZATION
The LONI Grid Monitor is a web applet that allows for real-time inspection of the status of the background computational Grid. The three-dimensional visualization gives users a quick, intuitive feel for the current usage of the cluster; each bar on the ring represents a single execution node, and the height of each bar represents that node's usage.

SUN GRID ENGINE
To facilitate the submission and execution of compute jobs in this heterogeneous compute environment, SUN's Grid Engine (SGE) is used to virtualize the resources above into a compute service. A grid layer sits atop the compute resources and submits jobs to available resources according to user-defined criteria such as CPU type, processor count, etc. The laboratory has successfully integrated the latest version of the LONI Pipeline (http://pipeline.loni.ucla.edu) with SGE using SUN's JGDI interface. This allows jobs to be natively submitted from the LONI Pipeline to the grid without the need for external scripts. Furthermore, the LONI Pipeline can directly control the grid, significantly increasing the operating environment's versatility and efficacy, and improving overall end-user experience.

 
STORAGE

Institutions and scientists worldwide rely on the facility's resources to conduct research. LONI has made a decisive move towards fault-tolerant, high-availability systems design to ensure near 24/7 functionality.

Concurrent with its graphics and computation systems, the laboratory uses a fault-tolerant storage area network (SAN) to accommodate current and projected storage requirements. The SAN hardware infrastructure is composed of :

  • the cluster and supercomputers previously mentioned
  • RAID storage
  • dual robotic tape silos
  • a full complement of Brocade fibre channel switches, delivering up to 800 megabytes per second data throughput.

Alternate paths exist throughout the fabric so that no single point of failure exists, guaranteeing access to critical data and processing power.  Two quad-processor 500MHz MIPS-R14000 processor SGI Origin300 servers mediate all data transactions and provide networking services.  A high availability application ties both servers together and ensures failover in the case of hardware failure. 

StorageTek SL 8500 and PowderHorn 9310
LONI relies on two silos, a Storagetek SL8500 and a Powderhorn 9310, to store mirrored copies of the facility's offline tape data.  These tape robots are housed in two different locations, ensuring that catastrophic events in any one data center will leave a copy of all tape data intact in another data center.  Seven high-speed 40-gigabyte and two high-capacity 400-gigabyte tape drives provide LONI's tape services.  To leverage the available SAN throughput, an SGI TP9500, TP9400 and a TP9100 RAID5 arrays as well as SUN 3510s provide nearly 50 terabytes of fault-tolerant disk storage.

NETWORKING

The LONI intranet consists of a switched 10baseT, 100baseT and 1000baseGigE ethernet as well as an IEEE 802.11a/b/g-compliant wireless network. The laboratory is connected to the vBNS backbone of Internet2 via redundant fiber optic 1000BaseGigE lines terminated on two independent routers.  While UCLA Campus Telecommunications Services (CTS) is responsible for the network and provides fault tolerance, network monitoring and load balancing, the redundant uplinks ensure that the facility's network connectivity will be maintained in the case of a single router failure.  The facility has a firewall providing network security.  Furthermore, LONI has implemented a virtual private network to facilitate access to internal resources by authorized users.  A virtual private network connection establishes an encrypted tunnel between client and server, ensuring that communications over the Internet is secure.

 
 
IN THIS SECTION:
 
 
LONI Copyright Reserved