Proje Geliştirme ve
Koordinasyon Birimi

2015-06-12 Portugal: SMDV: Space Mission Data Visualization


 
2015-06-12 Portugal: SMDV: Space Mission Data Visualization

Description of the project offered: 
ESA products are typically distributed over DVD or FTP/HTTP links (such as the ESA EO website: https://goo.gl/M76Mpn). The data packages are distributed in an unnatural way that does not facilitate data acquisition and integration with existing systems (processing, visualization, etc.).

Researchers and general ESA product users build software products based on these data packages, but the data source packages for a particular mission can easily be several gigabytes of information.

We propose a highly available, powerful and resilient web based service that provide to the users a clean and well defined API to perform:
- Search - The user is able to perform searches and retrieve data based on searches;
- Computations - The user is able to perform computations/transformations over the data;
- Storage - The user is able to store data resulted from a search or a computation and use it in a following search or computation.

The second part of the project will be the creation of a website where the users can visualize the ESA missions data regardless of their processing level.

Since all the space mission data variables are time series or images, the website shall provide to the user the ability to:
- Create Data Sets based on variables
- Generate charts based on the the Data Sets:
-- Parallel Coordinates;
-- Scatter Plot Matrix;
-- Correlate Data set variables;
-- Satellite Positioning over a Google maps layer;
-- Images over a Google maps layer;
- Collaborative workplace;

We want to provide an added value to the user, so the website must also have the ability to download data (resulted from a search or directly from the catalogue) to a remote location. The user can furthermore choose the variables and fields he wants in the final product.

As been said before, SMDV is divided in two main components: SMDV Web Service and SMDV Web Page.
The Web Page will be implemented accessing only the WebService for data search and retrieval. It shall also provide to the user the ability to store local defined data, such as results from previous computations, display data in several ways (tabular, chart, georeferenced) and manage user sessions in general.

Since the researcher users probably perform transformations and additional computations over the data sets, we shall provide an easy to migrate system where the scripts or existing software programs may be integrated with the SMDV Web Page.

A model based graphical language shall be created in order to facilitate the representation and reuse of the existing pieces of transformation software the researchers developed.
This language will be composed by the elements TestScripts and TransformationJobs.
A testScript can be parameterized and represents a actual script or software program that the researcher developed. It also shall be possible to program on the fly the actual computation/transformation that a TestScript may perform. For this the TestScript shall be able to access the WebService and provide to the user a wide range of languages so he can implement the computation. Nowadays several languages are implemented to compile into the Java Virtual Machine, so we believe that implementing this mechanism is an achievable task. Other languages, such as proprietary ones (MATLAB for instance) may be considered for integration as well.

Like so, a TestScript is an element of the language that implements the logic of a particular step in the transformation process and maps all the script/software inputs to himself.
A TransformationJob is a collection of TestScripts organized in a graph. The graph represent the actual execution. So, when two TestSripts have the same source TestScript means that they can be executed in parallel. The system shall be able to understand these and divide as much as possible the workload over the a cluster grid.
Just the TransformationJob can be placed in the schedule to be executed.
Project proposer:   Mario Ulisses Costa (Portugal)
Partner role:  Project participant 
Partner organisation:  Small or medium-sized enterprise (SME)
Call for proposal title:  N/A