CVMFS Discussion with Jump Trading

Name: CVMFS Discussion with Jump Trading
Start: 2025-02-03T11:00:00+00:00
End: 2025-02-03T14:30:00+00:00
Location: C7

Monday 3 Feb 2025, 11:00 → 14:30 Europe/London

C7.2.01 (C7)

C7.2.01

C7

Culham Campus Abingdon, Oxfordshire, OX14 3DB

Adam Parker (Data Solutions Unit, UKAEA), Matt Harvey (Jump Trading)

Description

Jump Trading is an international trading firm committed to applying cutting-edge research to global financial markets, and operates substantial HPC and data infrastructure to support that mission.

This session will share some of Jump’s experiences managing a business-critical data archive subject to exponential growth (many 100s PB, doubling time ~ 2 years). In particular, describing the process of re-architecting this live production system to transition from traditional on-premises block (GPFS) storage to cloud object storage, using CERN’s CVMFS filesystem.

Matt Harvey (Jump Trading)

Hide

Present: AP, ...

Agenda + Discussion Points

Introductions
Background on CVMFS (unaware what it is)
Current UKAEA Status + Plans
- Including summary of our experiments / systems
- CSD3 + Cumulus
Jump Trading approach
- How is CVMFS being used for data
- Current infrastructure
  - Amount of data
  - Number of systems
  - Caches/Squids/Varnish
- Special considerations for data
- How much support are you getting from CERN
- Suitability for AI/ML workflows
General Data Handling
- Previously discussed Globus
- Data transfer/distribution
- OSDF/Pelican
- Adios?
Technical Talk at 3pm (C7 Harwell)

There are minutes attached to this event. Show them.

- 11:00 → 11:15
  
  Introductions 15m
  
  Speaker: Adam Parker (Data Solutions Unit, UKAEA)
- 11:15 → 12:00
  
  Roundtable Discussion 45m
  
  Speakers: Adam Parker (Data Solutions Unit, UKAEA), Alejandra Gonzalez-Beltran, Andrew Lahiff (Advanced Computing), Jonathan Hollocombe (Data Solutions Unit, UKAEA), Matt Harvey (Jump Trading), Nathan Cummings (High Performance Data Analytics), Samuel (Computing) Jackson (UKAEA), Shaun de Witt, Stephen Dixon (The UDA man)
- 12:00 → 13:00
  
  Lunch 1h
- 13:00 → 13:45
  
  Roundtable Discussion 45m
  
  Speakers: Adam Parker (Data Solutions Unit, UKAEA), Alejandra Gonzalez-Beltran, Andrew Lahiff (Advanced Computing), Jonathan Hollocombe (Data Solutions Unit, UKAEA), Matt Harvey (Jump Trading), Nathan Cummings (High Performance Data Analytics), Samuel (Computing) Jackson (UKAEA), Shaun de Witt, Stephen Dixon (The UDA man)
- 13:45 → 14:00
  
  Coffee 15m
- 14:00 → 14:30
  
  Closing Remarks and AOB 30m
  
  Speakers: Adam Parker (Data Solutions Unit, UKAEA), Alejandra Gonzalez-Beltran, Andrew Lahiff (Advanced Computing), Jonathan Hollocombe (Data Solutions Unit, UKAEA), Matt Harvey (Jump Trading), Nathan Cummings (High Performance Data Analytics), Samuel (Computing) Jackson (UKAEA), Shaun de Witt, Stephen Dixon (The UDA man)