FARSCAPE WS1 Data Systems

Thursday 13 Jun 2024, 11:00 → 22:15 Europe/London

Stanislas Pamela (UKAEA), Tom Farmer (Research Data Management)

FARSCAPE Workstream 1 fortnightly meetings

---

This meeting is intended to be comprehensive strategy session focusing on the critical aspects of data management. This meeting aims to bring together key stakeholders and experts to discuss, evaluate, and enhance our data management practices.

The sessions plan to cover

• Metadata Management
  Understanding the role of metadata in improving data discoverability, quality, and governance. We will discuss best practices for creating and maintaining metadata standards.

• Data Catalogues
  Exploring the implementation and benefits of data catalogues in organizing and centralizing data assets. We'll review tools and strategies for building an effective data catalogue.

• Data Archiving
  Evaluating our current data archiving practices and identifying opportunities for improvement. We'll discuss policies for long-term data retention, compliance, and cost-effective storage solutions.

• Data Pipelines
  Reviewing the design, development, and maintenance of data pipelines to ensure efficient data flow from ingestion to analysis. We'll examine tools and technologies that can optimize our data processing workflows.

Representatives from the groups and experiments at UKAEA will have an opportunity to discuss their data management process and how it can be improved. Initially, facilities/groups that will be represented are:

• HIVE,
• MRF,
• ELSA,
• CHIMERA/PEGASUS

Agenda + Minutes

---

Present: AP, TF, ... and friends

11:00 → 11:15 Data Infrastructure Roadmap

Presentation and discussion of the data infrastructure roadmap

Speaker: Tom Farmer (Research Data Management)

11:15 → 11:25 Metadata Schemas

An update on progress developing metadata schemas. This includes high level schemas, as well as those relevant to specific facilities.

To discuss: schemas required for HIVE MVP

Speaker: Adam Parker (High Performance Data Analytics)

11:25 → 11:35 Metadata Ingestion Implementations

An update on progress developing a specific implementations of metadata ingestion. This includes development of:
- A UI for manual metadata ingestion for HIVE
- Plugins for automated metadata extraction and ingestion from files generated in data capture, processing or analysis.

Speakers: Bhargav Garikipati, Ajay Rawat

11:35 → 11:45 Metadata Ingestion System

An update on development of the metadata ingestion system. This includes:
- The client implementation which allows specific ingestion plugins to be developed.
- The metadata schema server (metacat).

Speaker: Derek Sandiford (Research Data Management)

11:45 → 11:55 Data Pipelines

Update on the progress of data pipeline work.

It will include a discussion of the remaining steps to be completed for HIVE before September.

Speaker: Kirill Palamartchouk (Research Data Management)

11:55 → 12:00 AOB and Review of New Actions

Review of additional actions which have arisen during the meeting

Speaker: Tom Farmer (Research Data Management)