Magic APIs, Exploits, and other fairytales of social media

Social Media Monitoring practitioners routinely bear the brunt of command level frustration over what can and cannot be found online. After years in the discipline it is easy to see where frustrations occur. Much of the conflict is based in misconceptions about SMM technologies. Some of them are actually humorous whereas others are downright aggravating. Below are some of the most commonly encountered:

1. Vaporware: (n) a program that despite what your friend told you, does not exist. I lost count of how many times a discussion has started with, “I want a program that…” and ends with a modern iteration of “SKYNET.” The fact that such a program does not exist has caused many a police executive and security professional to hesitate before buying a product. It seems they are holding out for that one perfect, all-knowing, all-doing, super crime solving software. This is commonly referred to as a “Unicorn” among practitioners. The solution? Go with a program that provides what you reasonably need.

2. “Why can’t you find it?” This is one of the single most aggravating questions any SM practitioner can hear. The easy answer is, “There are approximately 1.2 million terabytes of data available on the commercial web and you want me to find one missing tweet? From six days ago?” Of course this answer will not win friends or convert non believers to the fold. The truth is finding information like tweets and status updates can be extremely difficult. As a rule you need at least two of the four dimensions (User, Content, Location, Time) to vector in on a specific item.

3. Secret APIs- Let’s be clear, even SMM platforms have to abide by rules. As practitioners we know one of the risks in this discipline is the loss of an API. Savvy practitioners will know enough about the various social media sites to “mainline” searches if needed. Ethical practitioners stay away from SMM platforms who claim to have “secret access” to otherwise unavailable APIs. Rumors constantly swirl of platforms negotiating back-room deals with social media sites. In the end, if the data is acquired through suspicious means, it is no good for law enforcement, and may even lead to litigation.

4. Exploit-O-Rama- This is specific to SMM practitioners who gather information for prosecution. As of right now, information obtained via a script or exploit, which would normally not be accessible without a court order, is poisoned fruit. Many a speaker will stand in front of a room and deliver amazing speeches on the power of hidden exploits, but at some point even they must admit their methods will not stand in court. The best advice, “when in doubt, get a court order.”

5. Free is Key- There are plenty of free SMM platforms out there. Some of them like Tweetdeck and Topsy are actually really good. However, at the end of the day nothing beats a paid platform. Competition in this space has benefitted practitioners more than many can imagine. Whereas a few years ago a platform would display information once every 10 minutes, today’s platforms can monitor in real-time, build heat maps, conduct link analysis, and so much more. The old adage truly holds up, “you get what you paid for.”

Real Time Social Media Monitoring and Super Bowl XLIX

I’ve heard it referred to as the “Iron Circle.” A seemingly impenetrable ring of law enforcement, National Guard, fire services, the FBI, and dozens of other federal, state, and local agencies. At times it appears to be more secure than the White House. Despite dubious claims of circumventing security, I can attest, it is the most impressive display of temporary security in the United States. However, with the evolution of terrorism, criminality, and cyber intrusions, the next generation of security will not be featured in panoramic media shots or be populated with a sea of blue uniforms. It will be a small desk in a noisy room, somewhere not far from the Iron Circle, where one piece of information will have the power to move mountains.

This was the case during Super Bowl XLIX. While the media, fans, and critics took note of the army of law enforcement patrolling the stadium and related events, a small team of open source specialists were working 18 hour days pouring over an immeasurable amount of data. The information we discovered spanned from vague threats to specific situations, each one requiring some level of law enforcement response. Through all of this we developed some best practices I feel are worth covering. I must premise my observations with this; they are my observations. I’ve been conducting open source analysis since 2008 and I’ve logged well over a thousand hours. That being said, I constantly run into people in this field who challenge my perspective and cause me to see things in a new light. Rest assured, as you read the last line in this work, someone already hit “send” to let me know where I’m wrong. So be it.

The first best practice is finding the right people who can ask, “Why?” Anyone can be an analyst not everyone can be a good analyst. A good analyst is a person who looks at data for what it is, and then asks, “Why?” The same is true in open source analysis. I can sit you in front of the best media monitoring platform out there, but if you don’t know how and when to ask why, you may as well be an empty chair. I was fortunate enough to have two incredible analysts who were well trained and even more importantly, tenaciously inquisitive. They knew the difference between background “noise” and indications of trouble. In short, in order to be effective you need to ask why, a lot.

The second observation is the value of geolocation and its convergence with time. I’m not going to dive deep into special relativity or space time theory, but the addition of geolocation has added an incredibly useful dimensional element to open source analysis that deserves much more explanation. Simply put, when I first began open source analysis I was a prisoner of the past. Our rudimentary web crawlers could only find information after it had been in existence for several minutes (sometimes hours), and the location was practically nonexistent. Now we have media monitoring software like Media Sonar which captures information as it is being posted (time), and combines it with location (dimensions one, two, and three). Einstein would be proud! The combination has radically changed how we conduct real time analysis of social media and open sources.

I could list several examples, but that would take days. Here are a few that should make you stop and think about what you have been missing;

  1. A subject tweets, “My plans for tonight; get high as **** and break into the arena. Wish me luck.”
  2. A photo is posted to Instagram showing scantily clad women and the caption, “We need girls for the Super Bowl.”
  3. A selfie boldly states, “Thanks to security I now have to hide my **** in my boots. Try to find it now!”

Each one of these comments were tagged with a location and captured seconds after being posted. Each one required a law enforcement and/or security response. I venture to say that a few short years ago, none of these comments would have been seen let alone addressed.

The next observation may seem a little self-serving, but allow me to explain; Hire someone who knows how to do this. Law enforcement, government, and private security pride themselves on their “social media person.” I’ve personally been told numerous times that “so-and-so is a Subject Matter Expert and he/she can find anything on social media.” To that I say, congratulations! What you have is a digger and diggers are awesome! When I’m not doing real time, I’m a digger as well. But when you need information now, digging is only part of the job. You need someone who can read the real time data like reading a book. More precisely, like listening to a song. Like a single note in a song, the data you see is one element of a dynamic, emergent property. The difference is, that property could be a serious crime. So, either spend the time and money to train someone properly, or look outside for that extra help. (Incidentally you can find my email here).

Finally, the last observation I would like to make in this forum is the importance of diversity. Not in the sense you probably just imagined, but in the sense of tools of the trade. Real time analysis is hard. It takes a major investment in time and energy and can cause burn out really quickly. This is especially true when you’ve been watching social media feeds for 13 hours straight and suddenly someone from across the room who has nothing to do with real time analysis, logs-on to their Twitter account and finds a legitimate threat. The truth is, information slips through from time to time. In order to mitigate the slippage, use a few different tools. I would not suggest using more than three at a time, and to be honest, even using two at a time is hard. It is worth it however to have one person on the geolocation data, and another using keyword searches without the geolocation. The geolocation platform should always be primary. Key word, hashtag, and other searchable platforms are secondary, but in a pinch, may become the primary tool.

The Iron Circle will always be impressive, but as we gain an understanding of the synergy which exists between the virtual and real worlds, what happens in that small room not so far away, will be even more impressive!