January 14, 2009

If you stare at the lines in the road, you’ll never see where you’re going.

While the Devil might be in the details, if you want to know where he’ll be you need to look for the pattern. Pattern-recognition is one of the trickiest pieces of programming, partially because we know very little about how our brains work, and partially because humans make it look so damned easy. Our brains are designed to spot and record patterns.

Sure, it takes a long time. Evenutally, enough memories gel to enable us to look back and assign confidence of some correlation (if not causation.)

Red sky at night, sailor’s delight. Red sky at morning, sailor take warning.

That won’t hold up against the very best prediction engines we have, but it got us down that path. You go back and examine what happened before an event to determine what contributed.

Meanwhile, 200,000 people died at the end of 2005 because no one made the correlation that a big underwater earthquake might pose a problem. There was no warning for the tourists who were trapped by the wall of water. There was a warning for the Indonesian natives who knew that a sudden and unexpected ebb tide was a sign of a huge wave coming in. Many of the animals knew it, too.

It always seems far simpler when you know what the trigger is, instead of drowning yourself in a myriad of probably inconsequential details.

Mapping Ourselves

This post is more straightforward than I wanted it to be. It was originally envisioned as a short story describing the greatest effort in the fictional history of programming. Only I am not so sure it is fictional. It is too possible to not be probably, and world domination is potentially at stake.

I rush this essay into publication because of a fractured discussion with Steve Rubel, who tracks all things technology. Google has announced it is shutting down several of its free webservices, and Rubel mentioned in passing that Google Reader might not survive the next round without showing some value.

Reader is an RSS feed reading application. You ‘subscribe’ to blogs and content that you like online, and Reader ‘delivers’ it to you. Instead of clicking links and folders and bookmarks, the Web you like comes to you in one quick and tidy place. You can also subscribe to feeds of news searches, or mix and match your own sources.

Google Reader offers a couple of very useful features. One, all the feeds I subscribe to are searchable using the Google indexing and algorithm. I don’t have to ‘file’ things in my Reader, I can always find them later. If I choose, I can add whatever tags and descriptors I deem relevant. Second, I can share links with people I know. Real-life friends, business acquaintances, or those I network with online. I actually get ‘feeds’ of what my network of people found interesting, and we can add notes to each other pointing out key facts or summaries.

Soup to Nuts

What makes this interesting is Google’s role throughout. In the course of the content, Google plays several key roles. Not a monopoly, but it is a player in:

Content creation (Blogger, YouTube)
Content delivery (Feedburner)
Content aggregation (Reader)
Content discovery (Search)
Content sharing (Shared links in Reader)

No one else is a significant-enough player in all these aspects of the Information Age to track what is said to whom, and when it happens.

If you’re Google, you have Willy Wonka’s Golden Ticket: the map to the influencers. There’s a huge debate raging among the marketers and the public relations people and the politicos over just who has influence, and how you locate them.

Finding Influencers is important, because it allows you to target your message or your plea to only those people that really matter for a given function or moment. Those Influencers can change over time or given a different objective, but locating them is the key.

From the instant someone creates a video or a blog post, Google knows what is in it. (Again, there are other services, but Google gets enough of the video and blog business to make this scale.) Google also knows:

when it arrives in your Reader
when you read it
how you mark it
when you share it
when your friends read it
when they act on it…

…and the cycle continues. From Soup to Nuts, Google can know which people start the online tremors that lead to popularity of content. ‘Viral’ is no longer a marketing mystery – Google has the data to find the epidemiology.

The Ticket is the Beginning

Willy Wonka’s Golden Ticket wasn’t the prize itself – it was the step you had to clear to get to the prize.

Sure, Google could sell some of those results. It could offer up premium information to advertisers, or even offer direct targeted ads at the highest of most high Influencers. We’re not talking about the Pete Cashmores or Steve Rubels of the world – we want the people who are more likely to seed them with inspiration and information.

But even that isn’t the prize. Stay with me here.

Get several million people on Blogger, a large contingent of content producers. Get a couple of million more on Google Reader, and then sit back for about five years while they share data like no one’s business.

Except it is your business. You need to understand who the Nodes are, and how much time elapses between certain events. You need to learn how to look beyond the tiny pieces of data as individual bits, and instead look at the whole. Big picture, a bunch of water droplets becomes a cloud. And under certain atmospheric conditions, that cloud looks red.

Google isn’t going to drop Reader, because it needs us to keep feeding the data beast. It will take a good five years of collection (and maybe a couple of more concentrating on the data visualization to make it feasible, but isn’t that why Google hired all those engineers and algorithm people?)

Once you know what a ripple looks like, and the content of that ripple, you can track it. And you start to see the others. And eventually, you start to identify the ripples that preceded a discrete event instead of the ones that followed.

Making Waves

Google is building the world’s largest prediction engine. It’s now in a learning phase, and an early one at that. It’s building a new Vocabulary of Influence, not to sell us products but instead to tell the future. All you need is a series of similar events that you can compare, and look for correlating ripples that came before. Certain punctuated events would have no meaning, like outcomes of Super Bowls. But something like, say, a quarterly stock report, would be easy to parse.

It would be regular enough (and have a large enough data mine of its own) that you could put the most powerful computers to work just looking for the pattern. And as the owner of the ONLY data set that traces complete ripples of influence, there is no break in the chain to cloud the data.

Maybe the predictions will come with just a few hours notice, like a tsunami warning system. Maybe it will evolve into a longer-range forecasting tool for economics or finance. What could Larry and Sergey do with the Ginormous Google Gigawatt Crystal Ball? Other than promise us they Won’t Be Evil?

The truth is right there in your Google Reader, and the Devil is in your details. And that is why the High Holy Priests of Mountain View will never bring Reader to the sacrificial altar.

Let’s talk about identity for a little bit.

Do you know me? And if the answer is yes, let’s ask how you know me.

Do you know where I live?
Have we shared a meal?
Have we talked on the phone?
Have we swapped e-mails?
Have you followed me on Twitter enough to know when I am joking?
Did we go to school together?
Are you family?
Did I play a part in your wedding?
Do you know my passwords?

There are many degrees of knowledge, but when you get right down to it there are big cracks in the picture. Psychologically, we want to gloss over them because it’s uncomfortable to be with a stranger.

So we swapped an e-mail. Anyone can spoof an address that looks legit.
So we talked on the phone. Whose voice was that again?
Twitter? Good luck.
We went to school together? Think again.

Goldman’s Sacked

Enter John Goldman. John is about as off-the-grid as you can get. There are faint traces of him in my first high school yearbook: a picture here or there, and never any decent shot of his face. Some faculty members at Tuscaloosa County High School were quite worried when he didn’t show up for the 1984 graduation rehearsal. He never picked up his cap, gown, or any of his supplies.

John Goldman didn’t walk with his classmates. He couldn’t, because he was a creature of a total fabrication. Members of the yearbook staff made him out of whole cloth. (Full disclosure: I was not on the staff, but know the mastermind and those intimately involved.) It was a germ of a joke that sprouted legs, joined civic clubs, and failed to pick up its graduation paraphernalia. It was a deception and a conspiracy guided by an unusually light touch, and lasted longer than by rights it should if not for the discipline of those involved to keep the joke on the down-low.

It was identity theft, without an identity.

Circles and Rings

Which brings me to today.

I have several circles of friends in my Facebook. There is my high school crowd from Alabama. There’s my junior high crowd from Idaho (which has graciously granted me dual citizenship, if my alma mater once again forgets to invite me to a reunion.) There is a clique of former coworkers in Red Cross, my brothers and sisters in Kung Fu, and various communicators and marketers I’ve been privileged to connect with on various projects or to brainstorm. And there’s family.

These groups, these subtribes, are not created equal. Some know me as Ike, some know me as Isaac. Some have seen me cry, some have seen me bleed. Some have been over to my house for holidays, and some have been able to jump into a conversation with me as though 25 years had not flown by.

Some don’t know which foods I hate, nor which teams I root for, nor which movies I will never watch no matter the offered bounty. Some have no clue I grew up in Idaho, and many of them might have nominated me as least-likely to ever teach a martial art. It’s a good thing we’ve got profiles online to help us fill in the cracks, right?

Faceless Book

Unfortunately, we’ve now got these online profiles that help other people fill in the cracks.

Case in point: I have two friends on Facebook that I have worked with in the past. I worked with them in very different capacities on stories and projects years ago. As it happens, they are both affiliated with the same Bible college. Which means now my “People You May Know” box is filled with people that they know, and there is an assumption that I do as well. In my case, this is merely an annoyance.

Second case: Half of the graduating class in Idaho went to a different junior high, so I really never knew them at all. Seeing people with 31 mutual friends might be a strong indication that this indeed is one of my classmates, and with so many women using married names it can be really difficult to keep up. I find myself asking some rather rude questions, just to ensure I’m not adding a complete stranger and granting access to my private information. (Not that there’s anything salacious or dangerous there, but you see where this is headed…)

Which got me to thinking: What if John Goldman started friending people from the Tuscaloosa County High class of 1984? How many would add him? After all, he is in the yearbook, right?

And how many would add him after three mutual friends showed up? What if it were five? 12? 21?

The vast majority of online theft is not hardcore math or data-hacking. It’s human hacking. It’s gaining human trust, and abusing that trust to trick us into handing over the keys. The numbers game of social networking has made it even easier to exploit, because now you don’t need to scam but a couple of people to see the rest start to tumble like dominos. The inherent peer pressure, combined with the desire to not admit that you might have rudely “forgotten” someone can be a powerful motivation toward a single click of the mouse.

As the threshold of “friendship” continues to degrade, mostly through the abuse and dilution of the term by social networks, we need to be smarter about how we connect. We now compile vast banks of data with little regard to who might see it. We pretend like we’re surrounded by a wall of “friends,” but increasingly there are cracks in that perimeter, the banks are breached.

And even if Willie Sutton didn’t really say it, the banks will be targets because that’s where the money is.

Archives for January 14, 2009

The Engine