How to archive your tweets permanently

A question frequently asked of me is, how do you archive your tweets? The answer is a free, open-source package called YourTwapperKeeper, from the folks who make the service of the same name. In order to use it, you must be familiar with how to install a server-side application, including setting up a database, editing a configuration file, running a MySQL script, and setting up a Twitter application using the developer’s interface. If you’re not able to do this, you may want to look for other options, but this solution is quite powerful and you can’t beat the cost.

The one thing that will trip up folks when installing is that unlike other PHP-based open-source packages, you must actually copy/paste or run the included MySQL configuration script separately. It’s a very manual install, which can be intimidating for novice developers. Follow the directions in the README file to the letter and you should be okay. If you’re not sure how to run a MySQL script, you may want to have someone else do it for you.

Once you’ve gotten the package installed, configured, and operational, you authenticate with Twitter and can begin to archive your tweets by any text string, including your username, hashtags, and more.

Here I’ve set up a couple of searches, for myself and for the recent Blue Sky Factory conference.

Once the searches pull in some results, you can do an incredible amount of slicing and dicing of the archive, excluding retweets, filtering based on users or text, looking at specific time periods, and more. Unlike regular Twitter search, you can dig back into the archives for as long as you have data collected, which can be handy for analyzing Twitter patterns over longer periods of time or in month over month/year over year periods, far more than the two weeks of history that Twitter provides.

The most powerful feature, however, is the ability to export to a variety of data formats, including XML, JSON, and CSV/Excel. This is valuable if you want to provide, say, a tweet-based event recap, or you want to do analysis of timestamps, user interactions, and networks in third party data tools.

Here’s a partial example from the Excel export. I can, for example, sort out what percentage of tweets and people are using what clients in my audience, get geographic coordinates for use in services like BatchGeo, and do time-based analysis of how tweets and retweets flow (hat tip to Gilad Lotan @gilgul for the idea).

YourTwapperKeeper isn’t for everyone. It’s decidedly unfriendly to install. In fact, for most marketers, you’ll want to have the IT guys do the heavy lifting for you entirely, but once you’ve got it up and running, you’ll wonder how you ever recorded Twitter data without it.

You might also enjoy:

• Almost Timely News: Recipes vs. Principles in Generative AI (2024-03-03)
• You Ask, I Answer: AI Works And Copyright?
• Almost Timely News, January 7, 2024: Should You Buy a Custom GPT?
• You Ask, I Answer: Reliability of LLMs vs Other Software?
• Mind Readings: You Need Passwords for Life in the Age of Generative AI Fraud

Take my Generative AI for Marketers course!

Join my Analytics for Marketers Slack Group!