CVMFS Discussion with Jump Trading

→ Europe/London
C7.2.01 (C7)

C7.2.01

C7

Culham Campus Abingdon, Oxfordshire, OX14 3DB
Adam Parker (Data Solutions Unit, UKAEA), Matt Harvey (Jump Trading)
Description

Jump Trading is an international trading firm committed to applying cutting-edge research to global financial markets, and operates substantial HPC and data infrastructure to support that mission. 

This session will share some of Jump’s experiences managing a business-critical data archive subject to exponential growth (many 100s PB, doubling time ~ 2 years). In particular, describing the process of re-architecting this live production system to transition from traditional on-premises block (GPFS) storage to cloud object storage, using CERN’s CVMFS filesystem.

Matt Harvey (Jump Trading)

CVMFS 2024
CVMFS - UKAEA Feb 2025.pdf
Matt's Talk
Hide

Present: AP, ...

Agenda + Discussion Points

  • Introductions
  • Background on CVMFS (unaware what it is)

  • Current UKAEA Status + Plans

    • Including summary of our experiments / systems
    • CSD3 + Cumulus

  • Jump Trading approach

    • How is CVMFS being used for data
    • Current infrastructure
      • Amount of data
      • Number of systems
      • Caches/Squids/Varnish
    • Special considerations for data
    • How much support are you getting from CERN
    • Suitability for AI/ML workflows

  • General Data Handling

    • Previously discussed Globus
    • Data transfer/distribution
    • OSDF/Pelican
    • Adios?

  • Technical Talk at 3pm (C7 Harwell)

There are minutes attached to this event. Show them.