Table of Content

Uniform Web Services API

Background

Web services is a standard way to access remote functionality cross-platform, and assemble applications. We have several data types accessed by atlases: microarray data, 2D images, 3D volumes, surfaces, segmentations, annotations, phenotype/behavioral data, FMRI, time series, etc. Some of these data types have common representation models (e.g. MAGE). These models are typically large and exist in multiple incarnations. The level of detail they provide often is not needed for data discovery and common data access and integration tasks. So it would be useful to envelope such data in a common set of services that would expose the most essential data characteristics and represent the common denominator queries against each particular data type (e.g. getGenes, getProbes, getStructures...) that any dataset of this type shall respond to. Such services would support multiple clients, including atlases, BDR interface, mediator, etc.

Goals

  1. Develop a set of web service signatures for each data type.
  2. Return data/result sets in a common schema.

Web Service Signatures

Feature Requests
  1. Ability to mix "AND" and "OR" in queries
  2. Ability to request "pages" of results

DataType API
  1. 2D, 3D data
    XCEDE API (suggested by XCEDE GROUP) or visit Slice3:Remote Data Handling Page for more info
    1. Tree Functions
      (Get/Put) Service Application Description Input Parameters Output Implementations
      Get Projects   Returns list of projects      
      Get Project   Returns high level project information Project core elements    
      Get Project Detail   Returns full XML document for a project Project identifier    
      Get Subjects   Returns list of subjects Project identifier (O); Experimental Group    
      Get Subject   Returns high level subject information Subject Identifier    
        Subject Detail   Returns full XML document for a subject Subject Identifier    
      Get Visits          
      Get Visit          
      Get Visit Detail          
      Get Studies          
      Get Get Study          
      Get Get Study Detail          
      Get Series          
      Get Get Series          
      Get Get Series Detail          
      Get Data Acquisitions          
      Get Get Data Acquisition          
      Get Get Data Acquisition Detail          
      Get Assessments   Get list of assessments none; Project ID; Subject ID    
      Get Assessment Detail   Get information on an assessment Assessment ID    
      Get Assessment Data   Get data on assessments pairing of subject ID(s) and assessment ID(s)    
    2. Digest Functions
      (Get/Put) Service Application Description Input Parameters Output Implementations
      Get Subject   Get a simplified return of information on a subject Subject ID column for subject id, concept, value, observer, pointer, where information came from (e.g. RDF triple)  
      Get Group of Images   Get a simplified return of information on a collection (e.g. study) of images      
    3. Query Functions
      (Get/Put) Service Application Description Input Parameters Output Implementations
      Get Visits by Assessment and Data Available   Returns a list of visits based on assessments (collected that visit and/or overall) and by data available for that visit.      
    4. Data Functions
      (Get/Put) Service Application Description Input Parameters Output Implementations
      get data size estimate          
      get Data          
    5. Utility Functions
      (Get/Put) Service Application Description Input Parameters Output Implementations
      get Webservice Functions Supported   Returns a list of available functions from webservice      
      Get methods     return lists of identifiers, you can iterate through them to retrieve additional content      
      General Format   (arbitrary for single DB queries, undefined return for multi-site/DB queries) + + (required for multi-site/DB queries)        
  2. Microarray data
    Methods Summary
       getGenes(String geneCode, String geneName, String geneFunction, String keyWord)
          Get the gene information by either gene code, gene name, gene function or keyword.
       getProbe(String ProbesID String geneName, String geneCode)
          Get the probe information by either probe id, gene code, or gene name.
       getStructures(String structureName, String geneCode, String geneName)
          Get the gene information by either gene code, gene name, gene function or keyword.

Available Common Schema

XCEDE
XCEDE is the common schema providing access to BIRN databases. HID and the emerging AIDB are being wrapped in XCEDE (see http://www.na-mic.org/Wiki/index.php/Slicer3:Remote_Data_Handling), and - as deployed at BIRN-CC: http://bcc-dev-mediator.nbirn.net:8080/axis/services/HidQuerierWS?wsdl. Web services are being written against XCEDE, so both HID and AIDB will be accessible through XCEDE web services. The goal, therefore, is to route common metadata requests against gene expression, 2d images, 3d volumes data, in XCEDE, and extend XCEDE to support additional requests.

MAGE-ML/FUGE/MAGEv2
MAGE-ML is derived from Microarray Gene Expression Object Model (MAGE-OM), which is developed and described using the Unified Modelling Language (UML. MAGE-ML is by purpose used to describe microarray designs, microarray manufacturing information, microarray experiment setup and execution information, gene expression data and data analysis results.

MAGEv2 is being built on top of FuGE as an extension to add in microarray specific classes (extending Data as ArrayDesign, DesignElementData, etc, Material as Array, QPCRPlate, etc, and DimensionElement as DesignElement extended by Feature, Reporter, and CompositeElement).

Issues

The issues include (for gene expression data – but extrapolate to 2d and 3d data):
  1. figure out how search requests and outputs as implemented in MBAT (http://www.loni.ucla.edu/twiki/bin/view/MouseBIRN/WebServices), MA module in BIRN (http://microarray.nbirn.net/), and GN (http://www.genenetwork.org/CGIDoc.html), map to XCEDE
  2. Examine MAGE and see how the same requests and output can be expressed in MAGE. Then, depending on the results of (1), either abandon MAGE in favor of XCEDE, or include MAGE constructs in XCEDE wrappers for gene expression sources (as a foreign namespace?). This shall be done vis-à-vis common information requirements of client applications (e.g. GetProbes?, GetGenes?, GetStructures?, etc.)
  3. In parallel, review the schema used in the MA module, for whether it sufficiently reflects information model for GE data, and update as necessary
  4. Make sure the mediator can connect with XCEDE sources, be it a database source or a web source in XCEDE wrapper (see http://mediator.nbirn.net:8080/axis/services/MedTestBService?wsdl – this would involve passing web service calls via the ExecuteQuery? method, and conversion between XCEDE and mediator’s recordset)
  5. Results of mapping the common requests to XCEDE (in (1) and (2)) shall be discussed at testbed AHMs this spring. At the same time, identify additional sources or databases to be wrapped in the same API. Then finalize the signatures (mid-Summer?)
  6. Make sure terms used in queries and in the output, are tagged with BIRNLex terms (e.g. develop controlled vocabularies for each term)
  7. Implement web services for the GN and MA module (incl testing/deployment) (by the fall AHM)
  8. Based on results of (3), update data publication tools (i.e. software for loading data from common CSV and text files into the MA module), make sure controlled vocabularies are enforced (by the fall AHM);
  9. By the fall AHM – make sure AIDB’s XCEDE wrapper supports the services as well.
  10. Publish and document web services; develop a series of examples of how they can be called from various programming environments and applications