CCTS 1.1 Architecture

From CTMS_WIKI

Introduction

Background

The caBIG Clinical Trials Suite (CCTS) is an enterprise clinical trials system being designed primarily for use in trial sites. The suite is comprised of a collection of interoperable modules covering a broad range of key areas in cancer clinical trials management. These include patient registration via C3PR, patient scheduling via PSC, adverse events reporting via caAERS, lab analysis via LabViewer, and clinical data management via C3D. Integration between these applications is centered around five key scenarios: Study Creation, Register Subject, Load Labs in CDMS, Lab-driven AE Creation, and AE-Triggered Schedule Change. The implementation is based upon the caGrid infrastructure with caXchange as the Enterprise Service Bus for reliable message routing and GAARDS providing robust security.

Scope

This architecture and design document is primarily concerned with suite-level functionality that is focused on the shared responsibilities and information between multiple components in CCTS. Secondarily, it is concerned with the technology bindings between the expressed functionality and the caGrid components. Finally, it is concerned with the reusable architecture components that are available to all suite components that offer scalability and extensibility.

Related Documentation

Architectural Goals and Constraints

Primarily, CCTS architecture aspires to promote interoperability to a broad range of Clinical Trials software by providing a robust implementation platform to share semantically annotated data set. CCTS leverages caGrid infrastructure capabilities (e.g., caGrid Security, caGrid Analytical Services), NCICB technologies (e.g., Common Security Module) and other CCTS infrastructure (built within caXchange).

One key goal of CCTS is laying the foundation for semantic interoperability between various applications supporting the clinical trial space. As a key component, the Biomedical Research Integrated Domain Group (BRIDG) Domain Analysis Model (DAM) provides common semantics for all of the CCTS applications. In practice, this means that the use cases selected for analysis as part of CCTS 1.5 were ultimately described in terms of BRIDG concepts.

Methodology

The CCTS architecture and development teams leveraged the Unified Process Framework (UPF) to define workflows, process, progress, and milestones. Through the UPF, a version of the Unified Process (UP), SME’s are consulted with regard to both the priority of business cases as well as in understanding the breadth and scope of these business cases. The business cases are used to derive use cases for specification and ultimately for realization. That is, not only were the core scenarios derived from the SME’s, but these business scenarios form the basis of the analysis artifacts used to functionally decompose the distributed system and to seek for areas of efficiency and effectiveness.

Application View

The applications that fall within CCTS include:

Cancer Center Participant Registry (C3PR)

The Cancer Center Participant Registry (C3PR) is a web-based application used for end-to-end registration of patients to clinical trials including capturing the consent signed date, eligibility criteria, stratification, randomization, and screening, as well as coordinating multi-site clinical trial registration.

Cancer Adverse Events Reporting System (caAERS)

The Cancer Adverse Event Reporting System (caAERS) is an open source, standards-based, web application for documenting, managing, reporting, and analyzing adverse events (AEs). The system operates as both a repository for capturing and tracking routine and serious AEs and as a tool for preparing and submitting expedited AE reports to regulatory agencies.

Patient Study Calendar (PSC)

A software product for creating and editing study calendar templates, tracking patient activities as they occur, and managing patient calendars during the conduct of a study.

Labviewer/CTODS

A web interface for viewing lab data fed into the CTODS database from a LIMS system.

C3D Connector

A service wrapper for C3D, an NCI-supported Clinical Data Management System.

caXchange

Central hub for exchanging clinical trials information and integrating caGrid-based services.

Use Case View

The use cases selected for implementation in CCTS include:

Create Study

The Protocol Coordinator (different apps have different name for this responsibility) is ready to define a new study to the system (criteria is defined locally at each site). He enters the protocol information into C3PR. The data is recorded in the database. Once the study is ready, the Protocol Coordinator initiates a process and the Create Study message is routed to the other CCTS component applications which record the study in their systems.

Register Subject

The CRA verifies that the subject meets the eligibility criteria and if so, registers the patient to the study in C3PR. Once the patient is registered to a study and has been assigned to an Epoch, notification needs to go out to multiple systems (PSC, CDMS, caAERS, LabViewer). The message includes the Epoch Start Date and Epoch name. The message to the PSC will trigger the generation of the study calendar for that subject. The CRA will view the schedule of upcoming visits and the associated activities for that subject in the study calendar.

CT Load Labs

An HL7 message is generated from an external system, e.g. a LIMS. It is sent to caXchange, which routes the message to the LabViewer where the data is stored after being cross-referenced with a study and subject.

Load Labs to CDMS

The CRA looks at the scheduled study-specific visit for a given patient in the PSC. He identifies all the lab tests that should have been scheduled/completed for that visit date. The CRA needs all the lab values that would fall within the timeframe of the identified and previous visit dates. The CRA then reviews the lab tests conducted during this timeframe and identifies study-relevant test results to be loaded into the CDMS (for example, a diabetic patient may have also had a A1C done as part of standard care – in this case, this value would not be stored in the CDMS).

Lab-based Adverse Event

The CRA inspects the lab tests conducted in a particular timeframe and identifies a lab that may indicate an adverse event. The lab test and value is selected by the CRA and routed to caAERS. caAERS cross-references the lab with the appropriate subject and study and stores the data. When the subject is viewed in caAERS, the lab value is displayed for inclusion in a lab-based adverse event.

Lab-based AE-triggered Schedule Change

A subject has a lab-based AE which has been entered/pre-populated into caAERS. If the CRA determines that the AE meets the study criteria for dose or schedule change, they can send a notification via caAERS to the study calendar that there may be a change in the subject’s schedule. A CRA who uses the PSC application will see an AE alert when (s)he tries to schedule this subject for the next treatment. (S)he will then have to take an action to ensure that the AE alert has been addressed before continuing treatment. Then, if required, the subject’s schedule will be modified by the CRA.

Engineering View

CCTS Release 1 is a messaging-centric collaboration of systems that interoperate to achieve core business functionality particular to the Cancer Clinical Trial problem space. These messages derive from a common concept structure, the BRIDG model, and are functionally decomposed based on the message sequences noted in Section 2.3.

The functionality supporting the interoperability scenarios is exposed via messaging interfaces that are realized through web services. These web services are portable and scalable with the intent that they may be re-used and leveraged to extend core CCTS functionality. The following diagram provides a high-level architectural view of the Phase 1 implementation.

Security

CCTS leverages the caGrid security infrastructure to provide communication level security, as well as authentication and an integration with authorization. The following depicts a high-level overview of caGrid security infrastructure:

CCTS includes the following components:

• GTS for managing trust between services that are exchanging messages
• Dorian for provisioning users, authentication, and obtaining a user certificate from user credentials
• CDS for managing delegated credentials and policies
• WebSSO for managing single sign-on sessions (not depicted)
• Common Security Module for connecting local security policies with the grid components (not depicted)

Single Sign-on

Because of the multiple applications involved in CCTS, a single sign-on (SSO) security component that integrates common security framework and the applications is required. This leverages the existing user's public certificate by passing tickets between the applications that required user sign-in and then acting as a principle proxy. The following diagram depicts the implementation:

In the case where the user does not already have a Single Sign-on (SSO) ticket, the user’s log-in is intercepted by the CAS Agent and passed to the Single sign-on (SSO) server. The credentials are authenticated, and a SAML assertion is generated. This is passed to Dorian, which returns a Grid Proxy certificate, binding the individual’s credentials to the Grid for a particular session. This certificate is validated against the trust fabric (realized via the certificate revocation list) held within SyncGTS. The Grid Proxy Certificate is sent to the Delegation Service, which houses policies determining which applications a user should have access to. These user-specific attributes, in addition to the original Grid Proxy, make up the response to the CAS Agent, which has verified that the user has an active SSO ticket. At this point, the user attributes and Proxy Certificate can be passed to Grid Services and Applications for user access.

Security Extensibility

CCTS leverages the grid infrastructure for user management (authentication and grid identity management in Dorian) and secure communication (via SSL). Authentication of users is performed through a grid-based, pluggable AuthenticationService that returns a SAML document. By leveraging the open standard of SAML, any authentication mechanism (including hardware authentication) can be leveraged (see http://www.oasis-open.org/committees/security/). The SAML document is then passed to Dorian to get a signed certificate that represents the user’s grid identity. Although this leverages standard PKI technologies, grid identity management relies on a centralized Dorian service for management. In addition, a centralized delegation component (CDS) is leveraged to manage delegation policies and delegated credentials. This allows for applications to delegate user credentials so that other web applications and caXchange can act on the user’s behalf. Furthermore, to smooth the integration with the delegation service, a WebSSO component is leveraged. This allows web application developers to define delegation/SSO policies in configuration files and not have to write code to interact directly with the delegation service. The delegation service itself is a wrapper to CAS (see http://www.ja-sig.org/products/cas/), which is JA-SIG project for enabling single sign-on. This central component could be leveraged directly to enable delegation and SSO without having to interact with the grid delegation component directly. However, grid identities are still maintained in Dorian, and thus there is always an implicit reliance on the grid infrastructure to enable security.

Enterprise Service Bus

The ESB selected for the Release Phase of CCTS is ServiceMix (http://servicemix.apache.org/home.html). ServiceMix is compliant to the Java Business Integration (JBI) specification (JSR208). For the purposes of CCTS, ServiceMix offers the following benefits:

• Adaptability and Traceability to business scenarios
• Flexibility with respect to implementation technologies
• Avoidance of vendor (solution) lock in
• Extensible transformation engine for messages
• Routing
• Transaction control for long-running transactions that trace a quality of service requirement
• Multi-cast Routing
• Integrating heterogeneous application interface technologies

caXchange wraps the ESB, abstracting and providing a number of services in addition to those provided by a standard ESB (such as ServiceMix). Throughout this document, the ESB and caXchange are referenced interchangeably. For all intents and purposes, caXchange is the Enterprise Service Bus for CCTS - it abstracts and provides all operations needed on an ESB.

The following diagram depicts the logical deployment of caXchange:

Reliable Messaging

Messages that are passed in the CCTS environment must be reliably delivered (henceforth known as reliability) because of the sensitive nature of the Clinical Trials Suite business processes. For example, if an adverse event generated from caAERS does not reach PSC for a potential schedule change, it may put the patient at risk because of cognitive dissonance on the part of the healthcare team. Some possible reasons for messaging failure may be that the ESB goes down, the target application goes down, or messaging is interrupted.

Grid service calls by their synchronous nature are reliable. If the message fails, then an error is returned to the caller and the larger transaction can be aborted appropriately. However, as demonstrated in the figure below, caXchange uses a combination of both synchronous and asynchronous messaging to pass information in the distributed environment. Java Message Service (JMS) and ActiveMQ work together to provide fault tolerance and messaging reliability, and are both natively supported by ServiceMix.

Likewise, return messages from the various applications through the ServiceMix ESB are reliable using the same infrastructure. The status of multi-cast asynchronous messages is posted to a queue for retrieval by the initiating application.

Alignment of ServiceMix with caGrid

ServiceMix requires some adaptation to align with the caGrid backplane. Specifically, it does not use the same messaging protocols as those used by caGrid. This makes some aspects of messaging, such as security, very difficult. Therefore, the inbound binding component (a JMS queue) is wrapped with a caGrid inbound binding component to preserve the integrity of the sensitive data (e.g., patient data). This new caXchange component receives the message, make any necessary calls for features not supported by the JMS queue natively (e.g., delegation), and then post the message to the queue. caXchange also provides a number of other services, such as message validation, transaction control, etc.

Contract-based Integration

The ESB’s relationship to the applications is handled by a single interface control document (ICD). This document describes the interface specifications in detail that support the messaging requirements that arise from the interoperability scenarios. Specifically, it details:

• Interface Specifications
• Message Specifications
• Transaction Control
• Logging

It offers a common interface to handle connections to all applications and all business scenarios. The ICD forms the basis of contract-based integration, and details the explicit and implicit functionality that the ESB offers in CCTS.

See the CCTS 1.1 Interface Specification for more specific details.

Transaction Control

Transaction control is an essential feature of caXchange in healthcare applications. It allows applications initiating a messaging sequence to establish caXchange as a business proxy for message handling. This means specifically that whatever business function is intended for execution by the initiating application can be handled by the ESB, allowing the initiating application to either defer or proxy its responsibilities upon the completion of that business function. There is a correlation between interoperability messages and this business semantic.

For example, the CCTS suite requires the registration of patients for studies. The starting application that handles patient registration is C3PR, but it must in turn notify all other suite applications that the subject has been registered to a study, allowing those applications to perform their consequential activity. The PSC application is required to create a patient study calendar once it has been notified that a patient is registered for a given study.

Patient Registration (C3PR) -> Patient Study Calendar Creation (PSC)

Should PSC for some reason fail to take this action, then the business semantic is invalid, and some remedial action must take place:

• The suite must be notified that the registration of that patient is void
• The ESB must continue to send a Registration message to the PSC application until the study calendar is created. This could occur to perpetuity.
• Some combination of the above must happen

In fact, the human expectation is that when a patient is registered in C3PR that a patient study calendar will be created. Thus it is inappropriate to allow the ESB to endlessly attempt to re-send the message to PSC.

To arrive at an outer limit for this business-level timeout, the following assumptions exist:

• 4 seconds for each call to a service provider
• Total 3 tries per service provider
• 3 second network delay
• 3 second transaction processing by service provider

The interface control denotes the following rules for transaction control:

• The message recipient service implements a method to rollback. This rollback operation accepts the same message as the original operation.
• The recepient that receives the rollback() from the ESB is able to manage data integrity (i.e. if original commit() failed by that service, it will know that it needs to ignore the rollback(), or manage situations when rollback() is received after data are used by the user, etc.)
• All service providers interacting with the ESB agree to a timeout period of 25 seconds (see above). This is implemented as a worst possible case threshold for determining business semantic failure (e.g., the patient cannot reliably be held to be related to a study among all systems).
• If a particular receiver times out after 25 seconds, caXchange will submit a rollback to all other service providers within that transaction. If no rollback is received, service providers can assume a commit.

Additionally, each application flags data that is persisted as manually entered (through the application) or received through a message (received via controlled messaging transaction). This allows the individual applications to differentiate between information that should be rolled back and data that should be omitted from a rollback procedure.

Messaging

Semantics

In order to implement the use cases and workflows within CCTS, a messaging infrastructure is leveraged to exchange data between different applications. This infrastructure consisted of both message structures based on BRIDG model semantics as well as a physical infrastructure (for example, grid services and ESB). These messages contain information that supports both discrete business functionality ("register subject", for example) and the information component ("patient id", for example). These messages are unwrapped and examined to find the explicit rules covering their routing and processing based on predefined business rules.

For example, in order to facilitate a patient registration to a study, a "patient registration message" is communicated from C3PR to PSC, C3D, caAERS, and LabViewer. These messages contain the necessary data from C3PR that the other applications need to place a "subject" on a "study". In order for the other applications to be able to handle the message, it must be semantically harmonized (pre-coordinated) across the applications. Furthermore, it must either be syntactically harmonized or a translation must be performed.

Semantic harmonization is facilitated by the BRIDG model, which provides a basis for the informational semantics of the business process. BRIDG is a domain model for the analysis of clinical trials that provides a core set of concepts that are shared across each application participating in a business case. The subset of BRIDG necessary to meet the messaging use cases has been adopted and extended for use within CCTS. The CCTS messaging semantics are supported by BRIDG to the fullest extent possible. The BRIDG model can be located at www.bridgmodel.org.

The model leveraged in CCTS has classes related to participants, organizations, labs, and adverse events. Each message in the workflow does not need to capture all of the semantics of the entire model, so for each message, the BRIDG-derived model is subsetted into a smaller model that captures a single message.

Syntactics

It is necessary but not sufficient to harmonize the informational semantics of the messages across both applications and scenarios. If the meaning of the inbound and outbound messages are the same, then the application should be able to consume of them or some formatted translation of them. However, it is convenient that the syntax, or structure, of the message is also harmonized, and it is necessary that the structure is known to each application. To this end, an XML Schema is produced from the messaging models that realizes the semantics of the BRIDG harmonized model. The following diagram shows the registration message XML schema.

Messaging Identifiers

In order for messages to be locally persisted and hooked into existing data, a consistent set of identifiers should be used to cross-link data. This document describes those identifiers on a message-by-message basis.

CCTS currently supports the following messages:

• Create Study: study message sent from C3PR to PSC, caAERS, and LabViewer
• Register Subject: registration message sent from C3PR to PSC, caAERS, LabViewer, and C3D; response required from C3D
• CT Load Labs: lab messages sent into LabViewer through caXchange
• Load Labs: lab messages sent from LabViewer to C3D
• Lab-based AE: lab message sent from LabViewer to caAERS
• Schedule Modification: schedule modification message sent from caAERS to PSC

The following identifiers are sent for cross-linking:

Semantics and Syntactics Infrastructure

As described in the previous sections, messages are based upon BRIDG-derived models that are realized with corresponding XML Schemas that define the syntactic structure. The semantics of the messages, while BRIDG derived, is registered in the caDSR (Cancer Data Standards Repository). This provides a central repository for classes and attributes to be registered as data elements with semantic concepts registered in the EVS (Enterprise Vocabulary Service). Centralizing the registration of these models allows for common data elements to be shared across models, providing for a basis for semantic interoperability, as well as grid and non-grid based mechanisms to discover and consume of the semantic metadata. In a similar fashion, XML Schemas are registered and shared in the GME (Global Model Exchange), which is a centralized grid service for advertising and consuming of XML Schemas. This allows for the syntactics of the messages to be available when developing and consuming of services outside of the context of the service itself.

Discovery

While the caDSR and GME provide for central registration and discovery of the semantics and syntactics of messaging models, the IndexService provides for the dynamic advertisement and discovery of running grid services. While this is not leveraged in any of the CCTS workflows, it is an important use case that the grid infrastructure satisfies and is thus supported by CCTS. While running, the grid services periodically update a centralized grid service (IndexService) that stores the metadata describing the service. This includes the service signature (method signatures), institutional metadata about service ownership, and the semantic metadata that is registered in the caDSR. This allows for clients to dynamically discover the service based upon any of these metadata attributes.

Inter-application Linking

One of the primary interoperability requirements of the caBIG Clinical Trials Suite (CCTS) is to be able to link from one application to another. The link must take the user to a particular page in the target application based upon the context of where the user was in the source application. For example, if the user is looking at a subject on a study in the source application, the inter-application link should take the user to the same subject on the same study in the target application. This is known in the larger technical community as deep linking. In CCTS, it is termed hot-linking because the link may contain other contextual information, such as security information.

In addition, users have different requirements for how windows should behave when linking between applications. There are a handful of common approaches:

1. Links act as regular hyperlinks and replace the current application with the linked application
2. Links always open into new windows
3. Links to specific applications always open in the same window, e.g. a link to app C from app A and app B both would appear in the same window outside of the app A and app B window

Inter-application links accept a well-defined set of parameters for the HTTP request such that particular parts of the application can be linked into. The format is:

BASE_URL/RESOURCE?PARAMS

The parts of this link are:

• BASE_URL: the URL that the web application can be found on (e.g. https://someserver:8080/caaerswebapp
• RESOURCE: each application will support any number of published resources as extra parts appended to the BASE_URL (e.g. /study/subject, /study/template, etc.). These will be guaranteed to be maintained from version to version of the application. Changes will have to be approved by CCB.
• PARAMS: the set of parameters used by the application to render the resource appropriately. These have standard names for shared concepts, including:
  • studySubjectGridId: the grid identifier for the study subject

Published Links

Extensions

In the future, an architecture where there is a centralized hot-link router will be investigated.

Windowing

CCTS provides for a customizable hot-link windowing approach. Administrators can not only configure the URL of hotlinks, but also the window name. The following options are available for each type of hotlink:

• _blank: always opens into a new window
• _self: always replaces the current window
• NAME: opens into a specific window, replacing the contents

The following window NAMES corresponding to the applications are used as defaults:

• caaers
• c3pr
• psc
• labviewer
• c3d

By leveraging JavaScript, the window of NAME not only opens in the same window (which should be handled by the browser), but also gains focus and comes to the front of the other windows.

In addition, detailed window titles are provided for users to be immediately oriented to what is contained in each tab or window. Each application implements the following window title convention:

• APP - DETAILS

where APP is the acronym of your application (e.g. caAERS, C3PR, Labviewer, PSC) and DETAILS describes the current page a user has open. The following DETAILS are provided:

• FIRSTNAME LASTNAME (MRN) - SHORTTITLE (COORDINATING STUDY ID)
• SHORTTITLE (COORDINATING STUDY ID)

Deployment View

CCTS has a number of deployment recommendations that are discussed in the Installation Guide. The following production deployment view provides for a separatation of security, application, and other grid functionality.

Application Grid Service Deployment

Grid components can be fairly complex to deploy, involving deploying multiple wsrf webapps, deploying skeletons across project wsrfs, and deploying implementations into specific wsrfs. This is greatly simplified with the CCTS application grid service deployment architecture.

Each application provides a wsrf war file for deployment. This requires that configuration of the services be externalized (e.g. in an external properties file). The following conventions are used:

• Each application team provides a wsrf-NNNN.war file in addition to their web application war/installation. This provides the skeleton and implementation for all exposed services.
• Each application team leverages an externalized configuration file. Where possible and appropriate, these files are colocated in the same directory and follow a consistent file naming convention. Furthermore, wherever possible, property names are consistent.

Packaging

One of the primary requirements of CCTS is that it be a packaged bundle. Documentation and each of the CCTS components must be packaged onto a single piece of external media (e.g. a DVD) or into a compressed file (e.g. a ZIP file). Most external components will be referenced from the installation guide and will not be bundled.

CCTS currently supports the following messages:

• Create Study: study message sent from C3PR to PSC, caAERS, and LabViewer
• Register Subject: registration message sent from C3PR to PSC, caAERS, LabViewer, and C3D; response required from C3D
• CT Load Labs: lab messages sent into LabViewer through caXchange
• Load Labs: lab messages sent from LabViewer to C3D
• Lab-based AE: lab message sent from LabViewer to caAERS
• Schedule Modification: schedule modification message sent from caAERS to PSC

The following identifiers are sent for cross-linking:

Testing View

There is a clear need for a set of milestone tests ("smoke tests") to validate portions of the CCTS deployment. These should take the form of manual or automated tests that determine whether particular components are deployed correctly and provide sufficient details in error messages for debugging. In addition, smoke tests should exist for the integration of the components.

Web Applications

Each provides a "happy.jsp" that validates the installation of the web application. The level of validation is left to the individual software products. However, the following are primary considerations:

• Validation of web functionality, database connectivity, and configurations for the application and CCTS
• Providing appropriate and descriptive error messages

A common skeleton happy.jps is provided by the caAERS team and reused by the other teams.

Grid Services

Each software product provides a "happy" grid service that validates the installation of the grid layer of the application. The level of validation is left to the individual software products. However, the following are primary considerations:

• Validation of grid functionality, database connectivity, and configurations for the application and CCTS
• Providing appropriate and descriptive error messages to log files and in the response

A common skeleton SmokeTestService is provided by the C3PR team and reused by the other teams.

Integration

Integration of all the applications is tested by coordinated messaging from caXchange. caXchange forwards pre-defined messages for each of the integration scenarios. The applications process them normally, and also provide a mechanism for undoing the changes (e.g. by dropping the appropriate data from the database).

Note: data will require to be preloaded, and some intermediate steps are needed (e.g. PSC template)

caGrid Core

caXchange provides a mechanism to test the caGrid core security infrastructure by obtaining credentials from Dorian, invoking a caXchange service, and obtaining delegated credentials from CDS.