So What? Getting Started Being a Data Prepper

Well, happy Thursday, folks. This is So What the Marketing Analytics and Insights Live Show. Uh Katie is gonna be along in just a few moments. She is just getting out of the Smarter X um, I think it's the master class. So she was teaching, she's actually just funny enough, teaching in two different places at the same time, one pre-recorded and one live.

But I'm here uh along with John. And this week we're talking about something kind of interesting. Uh we're talking about uh getting started being a data prepper. Uh which if you're not familiar, this is this has nothing to do with like traditional data preparation for like ETL and data science. This is actually like tinfoil hat territory prepper stuff, but for data and AI.

So John, before we get started, when I say when we say data prepper, what does that bring to mind other than like you know, a bunker? Right, yeah, exactly. Now, this is somebody who's like has some tape stored in a vault underneath Fort Knox and the salt mines, and you know, this is your you're going far and beyond so that no matter if you can survive an EMP, you're good to go. Exactly. Um, so what we're gonna talk about this week is if you wanted to do this, um, we're gonna talk about some of the different software and some of the different methodologies.

But let's let's start by talking about, and you know, uh we'll we'll channel Katie here and say, why would we there she is? Was I summoned? You're talking about the five Ps here. I was just about to start. I'll talk about the five Ps.

I mean, I don't know what you're talking about, but I can talk about the five Ps all day long. We're talking about like why we'd be talking about being becoming a data prepper. What is sort of the purpose of it in the five Ps? And there's a bunch of different reasons. One, stuff vanishes from the internet all the time.

Companies run out of funding, uh, websites go down, governments change, censorship changes and stuff, and stuff that was valuable just might go missing. Um, there's a number of uh different like web archives. Uh I just saw uh 538, uh, which was acquired by ABC, just gone. They just wiped ABC just wiped it from the internet. So all that history of you know Nate Silver stuff is just gone from the from their website.

Second reason, internet access is not always reliable. Uh as evidenced by how many hacks and other things are happening right now, the very fabric of the internet is potentially at risk. So having copies of important stuff is a good idea. Third, for all of us AI nerds, uh, AI is much faster if the data is local, if you know, clawed or whatever, doesn't have to go out and get data. If you've got it right on your machine somewhere, having this things around.

And fourth, backups. Follow the 321 rule. Three copies of your data in two different places on two different types of media, one of which is off-site. Because there's my uh my father who was in the army, always used to say, um, if you have two, you have one. If you have one, you have none.

So if you have two backups and one of them fails, you at least have one left. You have only one backup and it fails, you don't have any backups. So having so from a being a data prepper or a data hoarder is sometimes about let's just make sure we have copies of our stuff. So I guess the question that we would naturally get is when is that too much? Like two copies incl so there's three cops of your data, two backups and one live version.

You know, it why not three versions? Why not four versions? Like at what point is it too much? Oh, when you can't afford it anymore. Which is fair.

But I think it's you know, it and again, this comes down to again. I'm just jumping in. So of Chris and uh John already covered this, but it comes down to proper planning of like, do I have the space for it? Can I afford those backups? If I can, let me make the most use of them.

Um I think Chris, we have stuff on like I don't remember the name, it's like the frozen tundra of where data lives. I don't know necessarily where that is or how to access it, but I know we have stuff in there just in case. Glacier, right? Yeah, Amazon uh AWS Glacier. And we'll talk about that a little bit, but that's you know, one of the things I was putting together for today's episode is full-on prepper tinfoil hat.

I need to make a tinfoil hat. Um, but full on tinfoil hat, suppose all the the the cloud services, the big techs, whatever that suppose they just go kaboom, right? Maybe they were hacked, maybe they're rooted, maybe they ran out of money. What would you have? So there's three forms of media that are generally recommended for data prepping and data hoarding.

Um, SSDs or uh solid state drives, regular hard drives, which are mechanical drives, and then optical media like CDs and DVDs. In terms of shelf life, an SSD, if you were to just put it in a box somewhere, the data on that will last about five years. Uh, if you plug it in and you know, run diagnostics on it every so often, you'll you can get more life out of a hard drive. Typically, you can get five to ten years out of, especially if it's not on all the time. If you have it in a box secure somewhere and you power it up twice a year to and you know, run disk check on it just to make sure it's it's still fine.

You you can get 10 years out of that optical media, if it's a a disk that you burn, uh like with a CD burner, DVD burner, all of us who are like you know, 90s as kids and stuff like that. Remember like CD burners are so cool. Um off-the-shelf media will get about five years out of, but it is very sensitive to light, heat, and moisture. If you keep it in a cool, dry, dark place, you can get five to ten years out of that. There's some forms of archival um CDD uh burnable CDs and DVDs that are way more expensive, but you can get 25 to 100 years out of them properly stored.

Well, and so again, this goes into proper planning because like I live in Massachusetts, and despite the amount of humidifiers I run in my house, I can have uh dark places, but they're not cool, they're humid, um, regardless of where I am in my house. And so that's just something that I would have to factor in. Like, do I need to spring for you know, an external storage place, maybe like uh uh storage box somewhere at an external location because of the environment in my home. So again, that's sort of the thinking ahead. Like that's immediately where my brain goes, is I don't have any place dark and cool and temperature controlled in my house.

I have places that are dark, but they're gonna be warm and they're gonna be humid. Yeah. So it depends again, uh going back to the five Ps, it depends on the importance of the data, right? So if it's like, oh, these are all my favorite, you know, MP3s from when I was a kid, okay. You know, that you could you can put that in on and just check it every year and make sure it's still good.

Um if it is like the financials of your billion dollar company, you're gonna spring for the storage. Well, so gotta go check that. Uh yeah, no, it totally makes sense because you really do have to prioritize. Um, we again when you start to think about like data preppers, data hoarded, we tend to think everything is important. Not everything is important.

You really have to focus in on, you know, it's a if you're talking about your business, it's a good opportunity to do something like a KPI map. What data informs if we're meeting our goals or not? That's likely the data you want to start with in terms of what you want to be uh backing up the most often. Yep. I think we should get a Netflix special though out of data hoarders.

I think there's something there, definitely. Directed by John Wall. Like open up the drive, what's in there? That's a thing. That is a a genuine thing.

There are folks uh over on the Reddit uh data hoarder platform, they will buy old hard drives and stuff at like you know, yard sales and things to just check out to see what's on them. And boy, are there some surprises. Oh, I'm sure. I mean, I could probably give you at least three or four thumb drives out of the top drawer of my desk. I couldn't tell you what's on them.

Mm-hmm. Yep. I mean, I know guaranteed nothing salacious, but also nothing interesting. I picked up a flash drive after last year's Fourth of July fireworks that was uh in the projector, and it had, of course, you know, the cities, uh, whatever. They they left the flash drive behind, like on the lawn.

But also on there was budget documents because they didn't format the drive beforehand, which was fantastic. Um we're gonna go through a lot of stuff today. So uh this show will have an accompanying PDF. If you want to get it for free, uh pop on by our free Slack group. Go to shustinsites.ai slash analytics for marketers, because there's a lot to go through.

Uh any questions before I start just foaming at the mouth and and saying words. Um not a question, a comment. I will early and often interrupt you to ask my questions when I have them. Okay. First things first.

What software uh should you generally have to do data hoarding properly? And there are there's about 10 pieces of software that I think are essential for doing it well. Uh so the first one is a and some of these are command line tools, which if you are not comfortable with them, uh, this is a great time to learn. Uh, one the first one's called W3M. This is a text-based web browser that is you would use with a tool like Claude Code or Claude Cowork or whatever, and say, use this tool to go download this web page or this website, and it will know how to talk to it to grab all that data and store it somewhere safely.

So this is you know, if you if there's a website that you really like, or even just a page, this is a tool that will go and grab the actual HTML, store it and locally somewhere on your computer. Okay. Next, it built into virtually every uh system is a system called Git. Git is version control software. When it comes to data hoarding, one of the things you have to be aware of is that data can be corrupted uh online.

So, like, you know, for example, uh, if you were to go to certain pages about COVID-19, uh, those pages have changed in the last year and a half to have a completely different version of what happened. So, version control software lets you essentially have a timeline. So, as you download things over and over again, you can see how things change if you use something like Git. I strongly suggest if you're using any kind of agentic AI tool, please have uh Git installed because it will save your, I guarantee it will save your bacon because at one point or another, Claude or Open Code or Codex will do something, and you'll be like, wow, that is completely not what I asked you to do. And if it compresses the chat, it compacts the chat, you'll be like, Well, I can't get the working software back because you wrecked it.

Well, and that's something that we're seeing Claude do more often is if you have a long chat or a project that goes on over a series of multiple days, it's really kind of forcing you into compressing things or compacting things rather. Exactly. Next up on the list is uh transmission. So if you're if you do not have a BitTorrent client, this is put one of my favorites. Um, this allows you to get torrent files of all sorts of different media.

Uh and if you were to look, for example, like the Internet Archive, uh, you can download public free, open, open source, uh, public domain content, like entire archives of the internet using torrent. Torrent allows you to basically do peer-to-peer file sharing, and everyone shares the load of of certain types of data. Early this year, with the US Department of Justice released a whole bunch of files, um, they were available as one of the formats available is torrent, and which takes the load off of having one central file server that's trying to share the whole thing. It basically forms a peer community. So this is uh one of my favorites for that type of application.

Yeah, be mindful of your settings on that. Don't open it up wide to let the whole world have unlimited data if you're on some kind of data plan. Exactly. RSync and R clone are command line utilities. Uh Rsync uh will synchronize two directories, so one from the other.

So uh for example, every Monday and Friday I have a script on my computer that backs up my entire code directory to the trust insights uh Google Drive so that I no longer have to remember, oh, I got to back up my my uh mission critical code. It does it automatically now. Our clone does that with cloud-based services like AWS S3, etc. So if you have data that you want to back up, these scripts, which again, a tool like Claude Code can write you the code to do, will allow you to stuck sock away your data and put it on a scheduled task, and then you will never have to remember to run backups because it will just do it for you when you run it. Uh next up is one of my favorites for travel.

This is an app called Qix. QIX is a free open source set of applications that allow you to download uh big archives of data. There's a Qix reader, it's probably the best one to uh the device thing to use, and it will download like a two gigabyte file on your laptop or your phone or what have you, but then you can use that like all of Wikipedia inside of an app. You can get a nine gigabyte version of Wikipedia, which will easily fit in a modern phone, and then it's offline. You're on the plane, there's no Wi-Fi service, you still want to read some stuff, it's there for you.

Uh public domain things like uh Project Gutenberg, you know, you're again you're on the plane, you don't have anything to read. Up there's what 100,000 works on Project Gutenberg, which are all public domain, great fun tool. Not useful commentary. It looks like it has packages already, but like if you just want to grab all the TED talks that can do that. So does the UI it can handle?

Is it all HTML or is there other things that it can handle packages? HTML ebooks. Uh-huh, it's all ebooks. Okay. Yep.

Uh, the next one uh is an open source package called called believe it or not, this is pronounced Chakovka, uh, which is Polish for hiccup. Um this is deduplication software, which is super helpful if you've got big data archives. Say maybe you've got your own photo library from the last 20 years, and you're like, okay, which of these photos are actually duplicates? This is one of the best packages for doing it. It's super fast, it's built in Rust, it's open source, so it's it's free to use.

Speaking of which, if you want to operate your own local file server, Open Media Vault, uh, let's see, it looks like that one open. There you go. Is if you have hardware that you've bought, you know, you went to uh Best Buy or Walmart and you bought a couple of their eight terabyte drives that are on sale, you can install your own networking software at home to have your own data hoard uh ready to go that multiple devices on your on your intranet can access. If you are going to be doing like super heavy AI work, this is actually really helpful for being able to grab large files on your local network rather than store them all on your laptop, because your laptop will eventually run out of space. If you like ebooks, calibre is been a project's been around forever and ever that can convert pretty much anything to an ebook.

Um, so if you have HTML, web pages, RSS feeds, uh, Kindle books, PDFs, etc., it will tear them apart and reassemble them as ebooks. Uh, I use this software when we're making trust insights, um ebooks, because it does a great job of formatting it and making it look nice, but also it can manage your library for you. So if you give it an entire website worth of stuff, it can it can download you know entire stories and stuff and turn them into nice, nice, easy to use ebooks that are totally yours. Open code, if you are familiar with Claude Code, open code is very similar, except it's not run by a corporation, it's run by the community. It is a a command line coding agent that can hook into pretty much any LLM.

This is a good one to have because again, Claude Code is supported by Anthropoc. And while it's installed and works great today, if Anthropic runs out of money, uh then Claude Code stops working. Open code will continue to work for as long as the community supports it. You want to pair this with a tool like Olama. So Olama is another open source tool that serves AI models.

So you might download an open weights model like Google's Gemma or Alibaba Quinn, or depending on how much hardware you have and how wealthy you are, uh you could even put things like Deep Seat. But this open code plus Olama gives you generative AI on your machine that if the internet blows up or whatever, you still have AI capabilities at your fingertips. So those are the 10 core pieces that I think you should everybody should have if you want to get into data prepping. Before I go into multimedia, questions, comments, thoughts. Um first comment.

Uh, my dog is now attacking me because I've been sitting at my desk for too long. So apologies for any chaos that it looks. Second, so I mean, these are really technical tools. So, like, where if you had to narrow it down to one or two tools for a beginner to start with, like what would it be? Like, so I guess you would want to start to think about your use cases or your five Ps.

And so let's say I wanted to back up all of my web analytics data. Like, this is a lot of information. Where the heck would I start with that? So, web analytics data lives, it depends on whose system you have. Um, right.

If you have like Adobe uh or you have Google Analytics, you're gonna be using an AI tool of some kind, like Clawed code or Open Code, to write software that will connect to those systems and download all your data for you. Sure, but then where do I put it? So are these the sort of systems that you're choosing? This is to extract the data, or this is to store the data? This is uh mostly storage.

We're gonna talk about extraction in just a second. All right. So if I if I was just a brand new babe in the woods and I was like, hey, I probably need to back up my Google Analytics data. Uh you gave me 10 options. Like what how would I know where to start?

Um I would start with open code and an AI model. So that to write the software you need because there isn't out of the there is no out-of-the-box open free product that says, hey, press this button to get your analytics data. Right. No, I understand that, but you're talking these are storage. So take the AI out of it for a second.

You're saying that these are 10 storage systems. How do I know which one to choose? I guess that's really my question is like you just gave me 10 options. It's overwhelming. How do I know where to start?

So this goes back to you know the storage medium, hard drives, solid state drives, and optical media. It depends on your use case. Uh the the if you were just getting started, I'd say go on the retailer of your choice and look for who's got either a hard drive or a solid state drive that's cheap and is at least one terabyte. Because your analytics data, believe it or not, analytics data is really tiny. Uh like when I downloaded all of my our the Trust Insights Google Analytics 3 data when that went away, you know, five years worth of data was like 10 megabytes compressed.

Um, like that fits on the thumb drive. Uh when I downloaded my own data, which was 15 years worth, that was like 15 megabytes compressed, like super small data. Right. So, like web analytics data was just one example. I guess my question is, you know.

That's 10 options is a lot. They all seem really technical. Like, how would I, if I know nothing about any of any of these, which I don't, how do I know where to start? Like, that's a lot of options. It is.

And so you start with the five fees, and then based on that, you talk to your AI agent and say, here's what I want to do. Here's a bunch of different options. You can even take the PDF from this episode and say, what of this makes the most sense? Being clear in your in your purpose and your performance, like I don't want a cloud subscription, I don't want to pay more, I don't want to do this. You know, help me with all this, all these crazy different options, help me understand what are the best choices, or maybe we didn't have those best choices, and it could do some web search for you to help line up what is the best fit based on your needs.

It always comes back to the five fees. That makes sense. That's more helpful. Thank you. Okay, so that's a lot of the utility stuff, things like copying from one location to another, deduplicating your data, storing it, certain uh storing certain formats, and then having an AI tool that's always on hand regardless of what's happening in the world, um, so that you can do stuff with it.

Now, let's talk about getting how would you extract certain types of data? Um this is going to depend on what your needs are and how to organize it. One of the best tools for video is a project called YTDLP. And again, you can ask Claude Code, Open Code, Cowork, etc. to help you install this on your computer.

Uh, this is true of everything. This will download video, audio, or subtitles and captions from over a hundred different websites, not just YouTube, but if you can see it, pretty much it can grab data from it. Uh, if you have the appropriate credentials, like if you have the login and password, you can download from like Wistia, Vimeo, etc., as long as you have rights to view the content. Um, so this is super helpful when you there's a video that you want to save. Um, for example, maybe you want to back up all of the late show with Steven Colbert before CBS removes it from YouTube.

Like say, just go on that playlist, say, here's the playlist, go and grab all the things for folks using AI. This is a phenomenal tool because let's say there's a channel on YouTube that has all these cool uh AI tips and tricks. You know, maybe it's the uh in ear insights podcast from Trust Insights. You could give it that playlist URL and say, just grab the subtitles. I don't need the whole videos, just give me all the subtitles, and you'll get hundreds of these things that you can then put into something like Notebook LM uh or the AI agent of your choice and have that knowledge base that's yours so that even if you know Katie decides one day, you know what?

We are now a pizza company, we're removing all of our YouTube content. You still have all those episodes stored somewhere. Don't tempt me, that sounds great. I mean, honestly, it really does. It does.

Um if you have data that is rich media in one format and you want it in another format. This is a free tool called FFMPEG. It's again, it's a command line tool, pretty much anything to anything. So, for example, every week when I we work when I edit the trust Insights podcast uh from for video, I use this tool and immediately make the MP3. I don't have to uh it actually is written in a little script for my desktop.

So if you have a lot of data that you want to move from one format to another, this is a fantastic tool for doing it, especially if you want to take really big videos and maybe make them more compact. For example, you have maybe you you made a copy of a Blu-ray DVD, which is going to be like 28 gigabytes in this native format. You can use this tool to say convert it to the smallest available file format, which is probably gonna say, like, okay, you want AV1, and it will do that, and you'll end up with like a one gigabyte file that you can then put in your storage medium, and it will look great, it'll work on your hardware. This is the tool to do it totally free. Okay.

If you have DVDs and other media, handbrake will do exactly that. Say, okay, I've got my like I literally did this not too long ago. Yeah, have yet for those of you who are like 90s and 2000s kids, you know what this is. I got mine behind me. Exactly.

This is that binder of CDs and DVDs. Handbrake will convert them all into the digital formatter you're choosing, which again makes it a lot easier to archive so you don't have like 80 of these laying around. But am I even really living if I am not, you know, speeding down the highway with one hand on the wheel, trying to flip through my CD book, trying to find the right tune to like keep me going and then maybe not crash my car? Like, am I even living if I digitize everything? Honestly, you're better off that way than some of the people I see driving down the highway on their phone like this, just like not seeing the road at all.

Surfing spot. It's not the same as seeing it written on the CD with JMs with a Z. In Sharpie in Black Sharpie. Oh, of course, obviously. Um if you want to play some of that media that you've then ripped, jelly Thin is probably the single best local network tool that will serve up your content from your data hoard in video, audio, and image format.

It's completely local, totally open source. It works over tail scale even. So if you were to set up a tailnet with your uh local media server, and when you're on the road, you can still access your local home server that way without opening up holes in your internet firewall, which is super cool. Um free and open source. If you just want to manage the vast quantity of images that you've gone and downloaded, uh, there are uh this is a tool called Image, uh, which is again completely local, runs on a local server or your local laptop.

It's like Google Photos without the Google part. Uh which again, if you're trying to get rid of uh big tech things, very, very helpful to have to to uh have access to. The new flicker. That's flickr. Pretty much, yeah.

Um gallery DL is similar to you to YT DLP, except that this takes any page worth of images and grabs all the images from it. So if you go to go to say like the Smithsonian Institutes, uh, you know, uh galleries and and look at the images that they have in their museum things, you can say, I would like a copy of all of this, please. Obviously, uh for legal reasons, we have to remind you please use these tools only for things that you have permission to or that are licensed for you to do so. Please do not perform illegal activities with any of the things that we've we're showing you today. Um Spotify, if you will there's a downloader to if you get a Spotify playlist, it will attempt to find essentially the YouTube versions of those and grab those.

And then the most important for a lot of AI today, GitHub Desktop allows you to find GitHub code repositories of important projects and download them. Because again, if that project goes away or um the maintainer stop supporting it, you can still grab a copy of it, store it locally, and then with your own AI coding tools, make your own changes and stuff to it. There are so many really good pieces of software that I use that are open source projects that I grab a copy of it once a month just to make sure if the if ever went away, I would still have it and still be able to keep it up to date. Makes sense. All right.

Any questions on all the extraction tools and things before we move on to a quick discussion about formats and then popular data sources. No, let's just keep rolling. All right. In terms of formats, there's three formats you should care really care about. Number one is called Mbox.

Uh Mbox is the Unix mailbox format. Almost every tool like Outlook and whatever has their own weird wacky proprietary mailbox format for your email that makes it impossible to use like a couple years later because they keep changing it. So if you had email from say 2007, you can't open that today in Outlook. Um you have to do like three levels of conversion. Apps like Thunderbird, which is an open source project, store it in what is called Mbox format, which is a plain text archive.

I have Unix Mbox files from literally from college from 1995 that are viewable today because it's just plain text. So one of the things you want to do as a data prepper is have your email in a system like Thunderbird, it stores it locally, and those that for those files will be you could it'll be able to open them in 100 years. Like your great grandkids will be able to open them up and go, wow, that uh looks like grandpa forgot the cover on his TPS report. Second format you should know and love. We talked about this recently in Ireland Marketers, Markdown.

Markdown is a plain text format that has styling information in it, but um is still plain text. So for example, today's show notes. You can see on the left-hand side, this is markdown. It's text with you know a little text marks, and then you can see what it looks like when it's renders. Markdown, again, will be something you can open up in a hundred years and will still be usable.

And the third format, particularly for data when you're working and this is something that is both good for archiving and for AI, is YAML. YAML, depending on who you ask, either stands for YAML Ain't a Mark uh Markup Language or yet another markup language. It is a way to to represent tabular data in linear format so that AI can read it more easily because AI sucks at reading tables, or it allows you to um to store data in a very compact format. All right, finally, this the seven places I would like uh to have copies of data. Uh, number one, Wikipedia.

Um, you can download it. It works out. If you download all of Wikipedia, it works out to about 115 gigabytes of data. Which again, for you know, you can get a one terabyte flash drive now. So, this is not a burden, like it, yeah, it's not gonna fit on your phone, maybe, but you could get all of Wikipedia downloaded and then have that available offline and stored.

And it's probably not a bad idea to do that from time to time. The single biggest archive uh out there is archive.org, the internet archive and uh PSA or I guess public uh thing, please donate to them. They are community funded, uh there uh and they have a lot of uh needs for it for assistance with storage. But if there's any page that you want to go back and look up, you can you can see it. So, for example, in their archive, if I put in the Trust Insights website, uh inside the archive, you can see when that has changed.

And this you can download archives from their archive. Uh there's a tremendous amount of stored data. The third that John mentioned is the Kiwix archive. So these are modules that go in Kiwix Reader, like all of Wikipedia, uh all the TED Talks on certain topics. Uh, there's Wikipedia, there's uh the Bitcoin wiki.

There's so many different things that you can download into the Kiwix reader app on your mobile device and have that data available to you offline. Fourth is uh, as I mentioned, hugging face AI models. You want to find um the models that make the most sense for you. So there are 2.9 million AI models out there. Um all of the ones that are hugging face, you can download.

You can uh install on your computer. So we talked about open code earlier and OLAMA. This is where a lot of those models will come from. There's models for pretty much anything that you could ever want to do: image recognition, audio, video generation, music generation, OCR. The key thing to remember here is that once you download it, it's yours.

And if the company goes out of business or whatever, it's no longer the kind of thing that you know you have to worry, oh gosh, they don't they change this model on me. For we were talking this week uh in the Truste Insights podcast about uh enterprise AI, uh Enterprise AI 101. One of the things enterprises love uh is stability. One of the things that AI typically does not deliver is stability. Um, but if you're using open weights models, you can say, hey, we're gonna standardize on Gemma 4.

That's gonna be our model. We're gonna download it, we're gonna run it locally, and then Google can make as many changes as it wants. We don't have to change until we decide we're ready. Whereas if you're using the cloud-based version, you change when Google tells you to. And I feel like that if you take nothing else away from this episode, that in and of itself is maybe the best pro tip for companies that are exhausted by the constant functionality change of these large language models because you don't have control over how they change, when they change, why they change.

But if you find a version that is working really well for you, use it. Do this, what Chris is suggesting. Go to Hugging Face, download it, and use it. There's no reason that you have to always have the latest bells and whistles to be able to do your job effectively. Exactly.

The last two uh that are worth pointing out: uh Project Gutenberg, 75,000 books. You can download the entire archive if you want. These are all books that are in public domain. Uh, not a bad idea to have them because again, uh burning books has been a pastime of authoritarian uh governments in the past. And finally, OpenStreetMap, which is a free open weights alternative to Google Maps and Apple Maps.

OpenStreetMap allows you if you wanted to download the entire planet. Um, this is uh which is their service, uh, it's free and you can see it's 162 gigabytes. Again, not uh insignificant, but it you could download this and have a snapshot of and a map for any place on the planet uh on planet Earth. And again, if Google decided one day, hey, you know what, we're now gonna charge everyone three dollars for Google Maps. Like, okay, cool.

I'm just gonna use this instead. How does this stay updated though? So this is a open the OpenStreetMap database is a foundation. Uh and so people submit uh uh map corrections and things. There's a whole team behind it.

Um and they also use uh a lot of publicly available uh data from various governments and things like hey, here's where you know what's happening with this road, for example. Got it. Yeah, it's interesting. Um, you know, and again, it really this for me solidifies why you would want to start with something like the 5P framework, because I'm what I'm realizing through the course of this episode is I am not a data hoarder. I am not a data prepper.

Um, I'm happy to let a lot of stuff go. And so I only could I could I've used um the web archive, the way back machine one, um, but primarily when I forget to take the before shots of pages that I've changed. Um, so it is interesting. I do think again, sort of going through the 5P frame can really coming up with your why is essential because everything that Chris has just shown. Yes, there's some open source stuff, but it takes your time, your resources to set it up and maintain it, or you're buying a service that does it.

There's a lot of different ways to go. So make sure that you're running through the five Ps to really be clear about why you're backing up this data in the first place. You might think it's important, but if you really scrutinize it, is it really important? It could still be. I personally can't think of a lot of information, like in my life in general, that I would want to have backed up besides my wedding photos.

Mm-hmm. For yeah, again, it like you said, it goes back to the five Ps. Um, I think there is value as a private citizen to having something like a version, maybe the text-only version of Wikipedia, which is about nine gigabytes. It's not a bad idea to have like one of those that maybe you update because again, storage is relatively cheap. In the event that you end up with you know, people and places and media companies and stuff that start trying to rewrite history so that you can say, okay, this is canonically what the point of view was at this point in time.

And these things will be useful, you know, from a museum and library sciences perspective. These things will be useful in 10, 20, 30, 40 years when people look back at today and go, well, what was happening back then? Well, you have more data than ever in probably pretty good for uh format to to look back. One of the key messages that for me as someone who I do consider myself a data prepper, I'm surrounded by hard drives, is um at any point a cloud service can pull the rug on you and either say, now you have to pay to get all your data, or we're just gonna make it go away. Um, for folks who have uh been with companies like Flickr, for example, you recall very clearly about five, 10 years ago, they did a rug poll and said we're gonna charge a lot more now for the same level features because we're running out of money.

Like, okay, that's fair. You're a business, you have to run like a business, but I didn't want to pony up today. If you look at a true service like Google Photos, yes, today it is free. Will it stay free? Maybe, maybe not.

Depends on Google's fortunes. Uh, we had we're talking at the beginning of the episode about Amazon AWS Glacier. It's very low cost to put data in Glacier today. Will it stay that way? Maybe, maybe not.

Will Amazon be here in 10 years? Maybe, maybe not. We don't know. It's never a bad idea to have multiple copies of your data that are that's really important stuff in different ways you can go and access it. So you either are a data prepper or your friends with a data prepper and you let them do the data prepping for you, is what I'm hearing.

Yes, I think so. Yeah, it's the friend with a boat model. Yeah, much better. John gets it. Yeah.

So, like, you know, you're the friend who brings like the baked goods, and you know, you bring over like the bottle of wine or whatever, and then you let the other friend do all the data prepping. Do the run. It's good training, it's also part of you know, if you're gonna do any kind of insane data intensive, like from all the tools that you showed. I've played with a bunch of those because of you know, setting up a media server. Like, if you want to try and be able to access your music from all the rooms in the house and that kind of stuff, you start going down the rabbit hole of some of those tools.

And yeah, you know, sometimes it's a good thing, and other times you're like, uh, maybe I'll just switch back to CDs. I mean, when you think about it, this is something that I know a lot of uh Gen Z folks have been talking about, uh, and Gen General folks is is subscription overload, right? I have a subscription to everything, right? And this it's an entire generation that has realized you don't own anything. Like you've got your Apple Music subscription and your Spotify subscription.

When you stop paying, your stuff goes away. If you are doing the whole data hoarding and data prepping and you're making copies of stuff and things, yeah, then if it if you stop paying for Spotify or whatever, like all your music doesn't vanish because you've backed it up on local hardware. And you know, these are these are challenging economic times. If you have the ability to do this now, while the content is relatively available and while it's relatively easy to get a hold of and you're storing it, it pays off. Uh, I'll give you a real simple example.

Uh, there are some shows that are no longer on streaming services, and that you know, if so, if you bought the DVDs and then you ripped copies of them and you put them on your local media service, you can still watch those shows for everybody else who didn't do that, and you don't own anything, so those shows are just gone now from from your ability to watch them. Uh and so I find myself on a regular and frequent basis going, wow, I'm glad I made a copy of that. Mm-hmm. Yeah, no, I mean, I can definitely see use cases, you know, for it. Like, I'm certainly not saying like there's no use, there's no reason to do it.

It is interesting to start to hear what those use cases are. Um, because yeah, it's just it's a fascinating thing because you could prep everything, you could store all the data. But then, do you want to maybe look around and see like, are there organizations that are already doing that work that maybe you could participate in to help it that way versus trying to solely take it on yourself? I think there's a lot of, you know, again, going through the five Ps and really understanding like, is this already being done? So you mentioned a lot of these organizations, uh, maybe looking for donations.

Maybe there's a way to get involved so that you can help them versus trying to take on all the data storage yourself. Exactly. And finally, this is something we talked about for a while with AI. The more data you bring to AI within reason, the more data you bring, the less it hallucinates, less it makes things up. So if you have good quality data stored up and stockpiled, then when you need to use it with AI, uh, you don't have to go looking for it because you know you based on your own data prepping and data hoardings tendencies, you know where it is.

I have lost count of the number of times like I have just dragged a PDF out of one of my archives and just dropped it into Claude said this is how to do this. And this and the same thing is true of the information products you're making with generative AI. A lot of people have a tendency to say, Oh, I just did that, delete chat, whatever, and don't store the work. Well, you you burned tokens, which is electricity and water and stuff to make that output. Just don't just throw it away.

If it's useful, store it somewhere and maybe even share it with other folks in your organization so that the energy, the electricity and the water, the freshwater stuff that you used to make that thing can be reused without having to burn more resources or burn more tokens on your account. Um prior to this live stream, the reason I was late is I was doing a live event and talking about very similar things with. So I was giving the event uh talking about finding efficiencies and scalabilities with systems like Claude. And the big takeaway was making sure that you're doing those requirements, the five keys up front, but also um making sure you're taking the time on your roadmap to build all of those reusable skills that the team will use, those plugins, making sure you have all those connectors squared away before you unleash the whole team on the account, the system, so that you know I've already built the SOW creator. So John doesn't have to recreate that every time he goes into Claude.

He can just use the skill that's already in there, that's already been iterated and created. So he's gonna save the tokens, he's gonna save the usage, he's gonna save John. You're saving the planet. That's it. Free tokens.

That's what I'm all about. Ever since Chuck E. Cheese, I've known that's but yeah. Yesterday, Kelsey and I were working on Google App Script, and we built a rule set. This is how to this is how to tell Claude, this is how to write Google App Script.

That goes right into our Google Drive for the company, so that everyone going forward can use that to tell Claude this is how you do it, and you're not constantly reinventing the wheel. Reinventing the wheel is is like literally one of my biggest bugaboos when it comes to AI in particular, uh, because the tools do it a lot and the humans do it a lot too, and it's inefficient and it's wasteful. I agree. So that is getting started being a data prepper. Again, as a reminder, all the tools that we covered today, we're gonna put in our analytics for marketers Slack group because there was a lot of them.

They went by fast. Uh and uh you can grab that, you can just uh join for free at trustinsights.ai slash analytics for marketers. Any final parting thoughts. I mean, I think really just be clear on your use cases because it sounds, you know, pretty straightforward. But if you start trying to save every single piece of data, it's going to build up really quickly and get overwhelming.

And then, you know, what's the point if you're not maintaining it, if you're not checking on it? So make sure you're building that into your planning as well. Like it can't just sit there. And test your backups. That's mine.

Mm-hmm. Exactly. Thanks for tuning in, folks. We will see you all on the next one. Thanks for watching today.

Be sure to subscribe to our show wherever you're watching it. For more resources and to learn more, check out the Trust Insights Podcast at TrustInsights.ai slash TI podcast at our weekly email newsletter at TrustInsights.ai slash newsletter. Got questions about what you saw in today's episode? Join our free analytics for marketers Slack group at TrustInsights.ai slash analytics for marketers. See you next time.