Total Recoll: Conducting Investigations Without Missing a Thing

Full text capture and search the easy way Dakota Nelson
Nov 08, 2017

One of the great underrated problems of OSINT is data overload. I don’t mean the fancy kind, the kind that requires graph databases and thousands of lines of code to solve - much more like “I think I read this on a website somewhere, but I can’t remember which one. I hope it hasn’t been taken down.”

Thankfully, there are ways to solve this - today we’re going to talk about a great free and open source tool that can help.

If you spend much time learning about OSINT, you already know about Hunchly - you should read through their page to get a great explanation (much better than I can provide) of the problem to be solved here. I haven’t used Hunchly myself, but I hear nothing but praise - if you want a professional-grade solution, this is probably the way to go.

Yet, for a lot of people, a commercial solution is overkill - or perhaps you just want more control over what’s going on. This year, at DEF CON 25’s Recon Village, I had the amazing opportunity to give a talk about an open-source alternative - Recoll.

Looking to get started fast? Download this recoll cheatsheet for a quick one-page guide to setting up and using Recoll in one convenient PDF.

Recoll bills itself as a desktop full-text search tool for Windows and Linux - which it is - but it’s also much more.

If this sounds like the kind of thing that would be useful for you, check out my DEF CON village talk:

I first started using Recoll as part of my SOURCE Boston talk on the NSA. It did an incredible job, allowing me to easily search through almost a gigabyte of PDFs with near-instant results. This is Recoll’s bread and butter - give it data (nearly any kind of data) and it will let you search through the full text of it all in nearly whatever way you wish. Here’s just a partial list of what formats Recoll can search through:

• text
• html
• maildir, mh, and mailbox
• Man pages
• Dia diagrams
• Excel and Powerpoint
• Tar
• Zip
• Konqueror webarchive
• Mimehtml web archive
• Microsoft Office Open XML
• OpenOffice
• SVG
• pdf
• Wordperfect
• CHM (Microsoft help)
• EPUB
• Rar archives
• 7zip
• iCalendar(.ics)
• Mozilla calendar
• postscript

Yep - that’s partial - there are more. The point is that Recoll can take almost anything you can throw at it. Want to know what else Recoll can do? Check out the video from the DEF CON Recon Village above and hear all about how Recoll can help you with web search and capture, and how to set everything up to get started in no time flat.

PLUS, get this recoll cheatsheet for a quick one-page guide to setting up and using Recoll in one convenient PDF.