The Kabbalah of predictive journalism

Since this article is about predictive journalism, you may have guessed that Nate Silver would be mentioned. But while the statistical likelihood of mentioning Baye’s Theorem in an article that mentions Nate is high — as Nate is known for his creative use of the theorem — the outcome of this article is not as easy to predict (especially as I myself don’t know the outcome yet).

We corresponded our article about conscious journalism to the third intellectual sefirah of da’at (knowledge). But for this article we will be discussing the first intellectual sefirah of chochmah (wisdom) as the sages teach that the wise knows what will be born from their actions (ro’eh et hanoled).

In brief, Baye’s Theorem considers “Priors” (also called “signals”) – events that we have pre-existing knowledge of – and “Posteriors” – events that we want to predict. While the theorem has again been around for over 250 years, Nate adapted the theorem into what he calls the “Bayesian convergence,” or the method of dispelling myths and opposing opinions as evidence of the most likely outcome is uncovered.

This is the second reason why we corresponded Predictive Journalism to the sefirah of chochmah (wisdom). In addition to knowing what will be born from present-day actions–like the chess master who calculates the upcoming moves 1–this sefirah on the right side of the Tree of Life model is relates to the past, whereas binah (understanding) on the left side is relates to the future. 2

As in our previous articles, our subject of focus will be the concept, marketability, or point of attraction behind Baye’s Theorem. Specifically, how Nate approaches it different from others, so much so that he is now assembling a team to carry out his approach to journalism as part of the relaunch of his site, FiveThirtyEight.com.

But in order for his findings, and those under him, to be statistically accurate, even the most unique journalist pieces need to be based on what Baye’s Theorem calls “Priors.” But while the result is to dispel myths, these disruptive results need to be based on prior statistical findings.

First, this is the example that Wikipedia uses to explain Baye’s Theroem (you can read the equation part here:

Suppose a man told you he had had a nice conversation with someone on the train. Not knowing anything about this conversation, the probability that he was speaking to a woman is 50% (assuming the train had an equal number of men and women and the speaker was as likely to strike up a conversation with a man as with a woman). Now suppose he also told you that his conversational partner had long hair. It is now more likely he was speaking to a woman, since women are more likely to have long hair than men. Bayes’ theorem can be used to calculate the probability that the person was a woman.

A Game of Priors

The first finding, as a result of corresponding Nate’s work to the sefirah of chochmah, is that the primary task of a past-minded predictive journalist should be to continuously weigh and reanalyze Priors (pre-existing statistic) not Posteriors (event we want to predict).

For events that already occurred, as in the train example, instead of searching for the most probable Posterior, the approach of chochmah, the yiddishe kop, is to search for the one true Anterior (what actually happened).

As we will explain later, Nate still has come up with a good chap (catch) with how he approaches Baye’s Theorem for events that have not yet occurred. But for the meantime, since most examples used to explain Baye’s describe events that happened, these relate to an exercise in discovering the one true outcome, not the most probable.

This story aptly illustrates this point. Appropriately enough, another train story! (reprinted with permission from Jewlarious.com):

After months of negotiation with the authorities, a Talmudist from Odessa was finally granted permission to visit Moscow.

He boarded the train and found an empty seat. At the next stop, a young man got on and sat next to him. The scholar looked at the young man and he thought: This fellow doesn’t look like a peasant, so if he is no peasant he probably comes from this district. If he comes from this district, then he must be Jewish because this is, after all, a Jewish district.

But on the other hand, since he is a Jew, where could he be going? I’m the only Jew in our district who has permission to travel to Moscow.

Ahh, wait! Just outside Moscow there is a little village called Samvet, and Jews don’t need special permission to go to Samvet But why would he travel to Samvet? He is surely going to visit one of the Jewish families there. But how many Jewish families are there in Samvet? Aha, only two — the Bernsteins and the Steinbergs. But since the Bernsteins are a terrible family, so such a nice looking fellow like him, he must be visiting the Steinbergs.

But why is he going to the Steinbergs in Samvet? The Steinbergs have only daughters, two of them, so maybe he’s their son-in-law. But if he is, then which daughter did he marry? They say that Sarah Steinberg married a nice lawyer from Budapest, and Esther married a businessman from Zhitomer, so it must be Sarah’s husband. Which means that his name is Alexander Cohen, if I’m not mistaken.

But if he came from Budapest, with all the anti-Semitism they have there, he must have changed his name.

What’s the Hungarian equivalent of Cohen? It is Kovacs. But since they allowed him to change his name, he must have special status to change it. What could it be? Must be a doctorate from the University. Nothing less would do.

At this point, therefore, the Talmudic scholar turns to the young man and says, “Excuse me. Do you mind if I open the window, Dr. Kovacs?”

“Not at all,” answered the startled co-passenger. “But how is it that you know my name?”

“Ahhh,” replied the Talmudist, “It was obvious.”

Woman with Long Hair

Taking the initial Wikipedia example. Someone had a conversation on the train who we now know had long hair. Now let’s say this person is an observant Jewish male, so now there are two options. Either he is related to this long-haired woman, or he’s engaging some spiritually-minded male soul in a discussion about Judaism. But we know this man’s family is in another city and he is travelling for business needs, so the latter option is more likely, especially as he is Chabad.3 But then we can start wondering about why this long-haired man is travelling on the same train, a train going to a very unique destination. By the end of it, given a few key “Priors,” the astute will not only be able to figure out the gender of this person, but some personal details as well.

Andrew Mason’s New Company

Now just for fun, let’s put our version of predictive analysis to the test, but this time for an event that has not yet happened. The intent is not that we are trying to predict the outcome (something which can easily enter murky waters according to the Torah) but to deduce based on present facts, the most probable outcome.

A few days ago, former CEO of Groupon, Andrew Mason, was in the news again. This time it was because he sold more than half of his Groupon stock. As a predictive journalist we’d like to write about what Andrew plans to do next as a result of this sale. But in order to predict, we need to go back in time.

In a May 2013 blog post, Andrew begins with the following:

I feel very lucky to be alive at a time when someone like me can have a simple idea like Groupon that ends up impacting millions of people. If there’s a silver lining to leaving Groupon, it’s the opportunity to start something new. I’ve accumulated a backlog of ideas over the last several years, my favorite of which I’ll be turning into a new company this fall.

It is now nearing the spring of 2014, and aside from a motivational business song album entitled “Hardly Workin,” seemingly, we are still waiting for that business announcement.

As explained in “Groupon: Turning Daily Deals into Tipping Points,” Groupon was a commercialized version of Andrew’s prior non-profit venture, ThePoint.com.

Here is some of the CrunchBase.com bio for ThePoint.com:

The Point is a website for organizing group actions. Its goal is to help people congregate around the issues they care about and combine forces to make things happen.

But to travel conceptually back another step, in science the “tipping point” is called a phase transition (e.g., the degree at which water becomes ice). Since Andrew moved from Chicago to San Francisco (what many consider the extended version of Silicon Valley), and as at the time of that announcement was planning on volunteering time at the start-up engine Y-Combinator, the prediction (again from an analysis of the past) is that Andrew is planning a new version of ThePoint.com geared to assist start-ups and other innovators.

But whereas that site relied on the “wisdom of the crowd” to select and support new social initiatives, what would we expect to be new about Andrew’s upcoming company?

Recent tweets indication that Andrew is still interested in implementing music in his new venture. But whereas ThePoint.com gave full access for new initiatives to state their case before the public, seemingly Andrew is looking for some way to tip or change the world, through music. For instance:

Gamifying Productivity

Similar to music, games (also mentioned in this tweet) have also been shown to increase productivity in the workplace. One of the leading proponents for this way of thinking, called the “gamification” of reality, is Jane McGonical.5

Given the above, the probability is that the new project is something related to gamifying tipping points with an emphasis on start-ups and soothing music. For instance, to gamify ideas to see which are more likely to be successful and which not (like the chess master who calculates the upcoming turns in advance).

This platform would also have implications for the workplace, where new ideas are suggested in board meetings on a weekly basis. By gaming new ideas, with creative music and engaging graphics, this could help start-ups and creative minded individuals to work through the possible outcomes of their ideas, before hiring a staff, fundraising, etc…

From Probability to Outcome

Especially since Quantum Mechanics, the world has become less deterministic. What we just presented was the convergence between three concepts called “phase transitions (tipping points),” “music” and “games.” But while each of these three represents a line in our conceptual three-dimensional mapping (i.e., classical music on one end of the music line, neo-modern music on the other), the point of convergence is defined as the intersection of these three lines. While the particular outcome can only be determined if the event already occurred, what can be determined is the conceptual mapping of the topic at hand.

This distinction we are now making is what this article about Nate and Baye’s Theorem calls the ability to navigate “big data” by means of “cloud computing.” But instead of relying on computers to do the work for us, as mathematics also relates to the sefirah of the chochmah, we prefer to speak about the unbounded limits of human potential.

The first step is to begin to conceptually map out Big Data, as explained in “The Question Behind the Quora Brand Concept;” the inner hope and attraction behind quora was that they would become a platform for doing this. Then once we have our cloud of interconnected concepts to travel among, the great connectedness between all “lines” of interest will become readily visible.

Morality, Light and Truth

How does a person choose which way to navigate through the content cloud of Big Data? It depends on which line the traveler most wants to travel on at that moment. While in space the three dimensions are up-down, right-left and front-back, according to Kabbalah they correspond to the lines of morality (moral-immoral), light (light-dark) and truth (true-false).6

Given that Andrew began with the tipping point concept, then went to music, we placed the convergence point or axis as gaming, with the other two lines intersecting it. As truth is always the outcome of the prior two–morality and light–then according to this correspondence, if a game was chosen as the next project, then it was also perceived as the project that would promote the most truth in the world. These three lines also inter-include within one other; which means that are nine primary formations, with an infinite numbers of spatial coordinates in between to choose from.

Bayesian Convergence

From the above, we can begin to appreciate why Nate’s takes on the Baye’s Theorem is so marketable. By using it to determine the probability of future events, in a method Nate calls “Bayesian Convergence,” Nate is alluding to the hidden potential behind the theorem that has laid dormant for over 250 years. The concept behind the theorem is best expressed as a method to navigate through the three-dimensional cloud of future possibilities and probabilities. Specifically how groupings of concepts converge with one another, and how travelers decide which line to travel on and in which direction.

As the sum of the probabilities of all possible outcomes is always one, we would like to end with the blessing that all our journeys lead us to come closer to the One, God.

If you are interested in mathematics and statistics, I encourage you to explore this further. As with previous articles, the above was written as an introduction to begin rethinking the world of predictive journalism.

Inspired by Rabbi Yitzchak Ginsburgh’s recent Monday night class in Efrat. Please check back for a link to the transcript.

1. A game invented by King Solomon, the wisest of all men.

2. This can be understood as relating to the present right-left or conservatism-liberalism differentiation in politics. For an entire class on this subject, watch: http://malchuty.org/2010-02-23-15-41-08/1084–q.html

3. While Chabad men speak to women when doing outreach, instead of physically sitting next to the woman, he would lean over from another seat or talk while standing.

4. The returning nature of chochmah is also returning to a greater awareness of the Divine nothingness or truth to be found in every particle of creation (or every story to be told).

5. My friend Rabbi Asher Crispe has written both about the tipping property of music and the ability to change reality by first gamifying it. The music article is called: Rock and Roll Redemption. Gaming: A World Taken by Games: Jane McGonigal at TED and: Upgrading the (Video) Game of Life

6. This was explained by Rabbi Ginsburgh in a class in Crown Heights a handful of years ago. If someone is interested, I could try and locate the master for you.

Photo Credit: CC-SA (Baye’s Theorem spelt out in blue neon at the offices of Autonomy in Cambridge.” by mattbuck, Creative Commons).

–

Read Start-Up Israel to keep your finger on the pulse of Israeli high-tech and innovation!