My favorite way to end brushing my teeth.
green_light_stop
joined 1 year ago
There was an article recently about this (too lazy to search it). It's already starting to happen. If most of the content they train on is the internet and more internet content is created by LLMs without being tagged as AI generated content (can't be guaranteed by all actors), then it's inevitable. High signal training data is out the window.
There are also techniques where data centers do offline storage by writing out to a high volume storage medium (I heard Blu-ray as an example, especially because it's cheap) and storing it in racks. All automated of course. This let's them store huge quantities of infrequently accessed data (most of it) in a more efficient way. Not everything has to be online and ready to go, as long as it's capable of being made available on demand.
Edit: Clarifying that tape medium is typically used for the longest term storage with the caveat that read is slow, so that used for the stuff that is least likely to be accessed. For things that are accessed infrequently but still need to be available relatively frequently you can have a "caching layer" which is what I was referring to with the discs. It's a tradeoff between speed of access and information density. Here's an article from 2015 where Facebook/Meta is discussing their design: https://engineering.fb.com/2015/05/04/core-data/under-the-hood-facebook-s-cold-storage-system/
Instead of comp, here's a pool table. That'll keep everyone in the office!