A question frequently asked of me is, how do you archive your tweets? The answer is a free, open-source package called YourTwapperKeeper, from the folks who make the service of the same name. In order to use it, you must be familiar with how to install a server-side application, including setting up a database, editing a configuration file, running a MySQL script, and setting up a Twitter application using the developer’s interface. If you’re not able to do this, you may want to look for other options, but this solution is quite powerful and you can’t beat the cost.
The one thing that will trip up folks when installing is that unlike other PHP-based open-source packages, you must actually copy/paste or run the included MySQL configuration script separately. It’s a very manual install, which can be intimidating for novice developers. Follow the directions in the README file to the letter and you should be okay. If you’re not sure how to run a MySQL script, you may want to have someone else do it for you.
Once you’ve gotten the package installed, configured, and operational, you authenticate with Twitter and can begin to archive your tweets by any text string, including your username, hashtags, and more.
Here I’ve set up a couple of searches, for myself and for the recent Blue Sky Factory conference.
Once the searches pull in some results, you can do an incredible amount of slicing and dicing of the archive, excluding retweets, filtering based on users or text, looking at specific time periods, and more. Unlike regular Twitter search, you can dig back into the archives for as long as you have data collected, which can be handy for analyzing Twitter patterns over longer periods of time or in month over month/year over year periods, far more than the two weeks of history that Twitter provides.
The most powerful feature, however, is the ability to export to a variety of data formats, including XML, JSON, and CSV/Excel. This is valuable if you want to provide, say, a tweet-based event recap, or you want to do analysis of timestamps, user interactions, and networks in third party data tools.
Here’s a partial example from the Excel export. I can, for example, sort out what percentage of tweets and people are using what clients in my audience, get geographic coordinates for use in services like BatchGeo, and do time-based analysis of how tweets and retweets flow (hat tip to Gilad Lotan @gilgul for the idea).
YourTwapperKeeper isn’t for everyone. It’s decidedly unfriendly to install. In fact, for most marketers, you’ll want to have the IT guys do the heavy lifting for you entirely, but once you’ve got it up and running, you’ll wonder how you ever recorded Twitter data without it.
You might also enjoy:
- It's Okay to Not Be Okay Right Now
- What Content Marketing Analytics Really Measures
- The Basic Truth of Mental Health
- Is Social Listening Useful?
- How I Think About NFTs
Want to read more like this from Christopher Penn? Get updates here:
Get your copy of AI For Marketers
I’m surprised you didn’t mention a site like Storify: http://storify.com/. It’s great for grabbing certain posts if you want the manual filtering. Incredibly easy to use and a great alternative if someone doesn’t want to mess with MySQL script. Thanks as always!
Really like the idea of this, but I must be missing something so I gotta question a little bit… why not Rowfeeder (no need MySQL) to track a search term or Backupify (free for personal plans) to archive all Tweets? Just seems easier.
Here’s why, for me. Now, take this with the understanding that I’m a geek and a nerd. Rowfeeder and Backupify are services, which means that they have the potential (and these days it’s not insignificant) to go poof one day, taking all of my data with them. Yes, I’d still have the downloads, but I wouldn’t have the service or a data format that’s contiguous. With this software, it’s on my server, administered in my database, etc. so I have the data even if the TwapperKeeper project goes out of business. I can also run queries in MySQL that other services wouldn’t let me or that Excel would explode trying to execute.
Does that invalidate those services? No. Frankly, I’d sign up for them and use YourTwapperKeeper all at the same time so that you get the best of both worlds.