this post was submitted on 02 Sep 2024
893 points (99.1% liked)
Programmer Humor
32380 readers
1208 users here now
Post funny things about programming here! (Or just rant about your favourite programming language.)
Rules:
- Posts must be relevant to programming, programmers, or computer science.
- No NSFW content.
- Jokes must be in good taste. No hate speech, bigotry, etc.
founded 5 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
The data are stored, so it's not a live-feed problem. It is an inordinate amount of data that's stored though. I don't actually understand this well enough to explain it well, so I'm going to quote from a book [1]. Apologies for wall of text.
It's very cool, I wish I knew more about this. A figure I found for approximate data rate is 5GB/s per instrument. I think that's for the European XFELS.
Citation: [1]: Yoon, C.H., White, T.A. (2018). Climbing the Data Mountain: Processing of SFX Data. In: Boutet, S., Fromme, P., Hunter, M. (eds) X-ray Free Electron Lasers. Springer, Cham. https://doi.org/10.1007/978-3-030-00551-1_7
That's definitely a non-trivial amount of data. Storage fast enough to read/write that isn't cheap either, so it makes perfect sense you'd want to process it and narrow it down to a smaller subset of data ASAP. The physics of it is way over my head, but I at least understand the challenge of dealing with that much data.
Thanks for the read!