Back

@eosfpodcast I had a conversation last week about the Internet Archive looking to hire an AI person, to explore what this tech could do for the management of their hoard of weird and obscure and often-boring videos.

There was a mild disagreement with someone that really was just about the fact that I was referring to Machine Learning, which has been around for a long time and of which LLMs are only one example, but they thought I was just repeating the hype of "wE CuD kUrE cAnS3r GaIzE!"

But my thinking was/is more that ML could indeed be a useful tool for curators and archivists at IA, because as I said they have amassed a large hoard of audio and video, a lot of which is extremely niche and would be suicidally boring to try to catalogue and cross reference it all with human brains.

Maybe if things get bad enough that monasteries come back. There's a Babylon 5 episode where Brad Dourif is a monk, and his monastery volunteers to go through thousands of hours of video footage to find a sus character who is up to no good. One of the better episodes in fact, highly recommend everyone slog through all that bad acting in the first season, it really does get very good.

But anyways, a GPT could go through all that video and it would certainly be able to very accurately do things like describe everything that appears onscreen (so if an archivist wanted to find every shred of footage they can find of '68 Camaros, for instance) and at least fairly accurately describe things discussed by people or voices, with a reasonable summation of the attitudes on display about the topic.

It would miss a lot of subtleties, for sure, like sarcasm and whatnot, but if the archivist starts with nothing and then has that imperfect list to search and cross-reference, that will absolutely save them a ton of time, and give them access to things that might otherwise have gone down the memory hole.

But in order for that to happen in a useful way, you can't just throw ChatGPT at it, and the results should never be presented to end users of the site; it's an expensive and imperfect tool for an expensive and imperfect job of curation/archiving.

For a useful and reasonable application, you need a competent programmer to create focused applications with specific training. In the case of IA, they would still probably need a GPT of sorts, if I've got my jargon correct, due to the generalized nature of their archive. In their case, a small onsite GPU bank will probably benefit them.

As a Sysadmin, when I did that job, my biggest challenge was conveying reality to developers with no interest in understanding the underlying systems, even when it would save the company enough money for ten more devs on their team. I tried but I'm bad at people, and so some really ridiculous and expensive vendor systems were put in place for very ignorant reasons. Company went under eventually.

America has done a similar thing, in terms of their stampede into the corral of Silicon Valley Life Management, and the AI hype bubble is just the biggest and most recent example. The dotcom crash should have wised people up. Maybe this will.

Because right now, the USA as a whole is experiencing the same thing I did at that company that is now dead: they went all in on a Silicon Valley VC scam, in fact they went all in on dozens to hundreds of them, each one taking an even bigger bite of the future.

The company going under came in a day, we didn't even get two weeks' notice, because things were stretched completely beyond their limits.

It takes more than a day for a giant economic superpower to have their bottom completely fall out, but it will look astoundingly fast when they write the histories in a few years. Faster than anyone believed possible. But the reason is that the country has already been hollowed out like the Hudson's Bay Company, there is nothing left to scrape away. Whatever the USA is going to be from now on, it will have to be rebuilt from the ground up or else put out of its misery.