Justify Overview

Christopher Fry, cfry@media.mit.edu
started Feb, 2009; edited Aug, 2013.

Justify helps people make the best decisions possible under the circumstances.

The Circumstances

Complex problems confuse us. Our ability to hold lots of thoughts in our head and once, weight them comparatively and choose the most important is limited at best. Thus we often end up making difficult decisions based on the last point we heard, or the first, or what our uncle did in a similar situation, or what's popular. We will buy products because they are the most expensive figuring "You get what you pay for". Other times we buy the cheapest because "its a bargain". Perhaps we buy because "we always buy that" even in the face of new products in the same category.

These techniques are plagued with logical fallacies, inaccurate weighing of pros & cons, and subject to invisible holes in our knowledge that prevent us from making optimal choices. Committees formed to make important decisions sometimes perform worse than any individual in the committee would if acting alone. (The US congress can't balance a national budget but their individual household budgets probably are balanced.)

Justify is a tool to alleviate these mental shortcomings by giving users a tool for isolating and recording points, classifying them into types, organizing them with respect to each other, drawing conclusions automatically and presenting results in a concise format. Justify attempts to get the benefits of the greater knowledge of a group of individuals, while minimizing the mis-coordination common when more than one person is in control. We encourage you to continue reading about the design and usage of Justify, but if you're desperate to get started, you can skip to the tutorial.

Humans

Humans have a far greater capacity to reason than any other species. However, even casual historians know that we use this capacity poorly. By structuring our thought processes better, we humans can leverage our reasoning ability into something much more useful. Humans have another capacity that can help us here: the ability to use tools to amplify our capabilities. What sort of tool can help us reason?

Language

Language competes with reason as our most significant skill. Language and reason are intimately tied to one another. Roughly speaking we have a word associated with each important concept. Without that word, concepts are difficult to communicate. Language is not just a tool to communicate with others. Language is a tool for reasoning about concepts within an individual.

Natural languages (English, Spanish, etc.) were and continue to be designed by the rather chaotic process of evolution. Over eons, evolution can create amazing results, yet rarely are they optimal for any given task. Language is no exception. Natural languages are great at generating large and ambiguous vocabularies. Synonyms fill the infosphere with redundancies. Near synonyms may provide artistically interesting shades of meaning but shades are by definition, difficult to distinguish and therefore cloud clear thinking. Worse, many words have more than one meaning, meanings that can be discerned only through context. Unfortunately, context is often ambiguous, thus so is the intent of the speaker. Making matters worse, the meanings assigned to words change not just with their context of other words. Meanings also change with the people you're with, physical reality and time.

Words are put together in a grammar, which is a set of rules describing how words are interpreted based on their juxtaposition with other words. Grammar in a language like English is so complex that probably no one can articulate it definitively and completely. Attempting to reason with natural language is doomed to be frustrating at best.

Artificial Language

By replacing, or at least augmenting, natural language with an artificial language, we can overcome most of the detriments of natural language. However, any artificial language won't do. For the domain it expresses, a good artificial language must be:

Justify aspires to fulfill those goals for the domains of reasoning, decision making and argumentation. Justify is first and foremost a language. It contains very specific constructs such as 'prioritize' and 'inconsistent'. It also contains more generally applicable ones such as 'idea' and 'question' to lend coverage to broad concepts.

Justify is an extensible language. New vocabulary can be added to its simple, yet flexible, grammar to encompass a rather wide range of thought. The Justify user interface is a "reader" for the language, permitting the user to dynamically vary the level of detail presented. It is also an "editor" that guides people into making syntactically and at least somewhat semantically correct points. Justify simplifies complex reasoning by providing:

Process Guidance

People often dislike being forced to follow a particular process. They want to short-cut the red tape and get to what matters most to them. However, anyone who attempts to enforce a streamlining process may be seen as a dictator and thus disliked. We have sympathy for this sentiment as many processes are poorly suited to the task at hand and are thus viewed as 'getting in the way'. Particularly pathological processes such as the US Senate's filibuster can render any meeting counter-productive.

Justify provides a definitive framework that none-the-less accommodates flexibility. It allows 'points of information' at any juncture, permits off-topic points at any time, and the easy creation of new topics while subtly encouraging focusing on the topic at hand. Even more important for quality discussions:

By using Justify to organize a discussion, no one has to be the 'rules enforcer bad guy'. Once participants get the basic idea, these rules are so natural that they become invisible.

Identifying Holes

One of the most difficult things in a decision-making process is to notice information that is missing. Justify discussions are, at least in some sense, always incomplete, because they are infinitely extendable. Nonetheless, there are a few ways Justify helps you achieve a relative completeness.

  1. Certain points will 'conclude' to 'unfinished' and the yellow 'unfinished' icon propogates up the point hierarchy to make this obvious. For instance, questions must have at least one answer or they will be unfinished.
  2. You can protect against many common omissions by using a 'checklist'. The book "The Checklist Manifesto" explains that even experts forget steps in complex processes. In Justify, you can use the menus of rationale types to test whether you've left out an important argument. When a point is made, say "let's build a house", you can use the subtypes of "con" for a list of reasons not to do it. Some are more obvious such as 'extravagant', but might it also be 'unhealthy' or even 'disrespectful'? Considering a list of possible objections helps build confidence that you've done a thorough job.
  3. The Justify interface allows the user to vary the detail presented at each point in a discussion. If its a long discussion you may not care about or have expertise in some branches of the discussion. With mouse clicks you can hide those branches and focus on those parts you do care about, checking for completeness.
  4. The assessments that Justify computes from user-entered points allow users to gauge the 'result' of any branch at any level of a discussion. If you agree with the current assessment, you may leave well enough alone. But if you disagree, examine that branch for missing rationale. Perhaps others don't understand what you do and you can help them with a well-placed point. Or perhaps they do. By examining a particular issue more deeply, you may learn something and change your mind.

When Not To Use Justify

Justify requires you to think formally about your problem. This cognitive 'overhead' is not worth it for simple problems. Additionally a problem may be complex and hard to solve yet ultimately unimportant. Unless you enjoy problem solving as a academic exercise, we don't recommend that you use Justify for insignificant problems. Its OK to practice with simple problems, but normally, use Justify forhard problems that you really care about.

Who Should Not Use Justify

Justify discussions require participants to reason. Reasoning requires patience and a willingness to listen openly to other points of view. We do not recommend that unreasonable, impatient, non-listeners be allowed to edit Justify discussions. Such traits will harm most collaborations and Justify is no exception. Die-hard adversaries are unlikely to have productive discussions unless they can check their anger at the door. Justify may provide a sufficiently new environment where adversaries can step out of their traditional roles and make an extra effort to understand each other. We suggest that authors give each other the benefit of the doubt and ask questions about ambiguities rather than make quick assumptions. Justify encourages very specific criticisms about precise ideas. It discourages broad criticisms or attacks on authors as opposed to specific ideas.

We have not performed tests with large numbers of people in a discussion. We suspect it will not work well. This is because the more people there are discussing something, the more content there will be. This makes it less likely that each person will carefully review 100% of that content. This increases the likelihood of misplaced and redundant content which further increases the likelihood of lower quality content. A person who enters a point without understanding all existing points risks polluting the discussion. Although Justify makes it much easier to locate 'where' within a discussion a particular point should be, there is a limit to even the best of organizational techniques.

Usage Scenarios

Justify can be used by an individual to help sort out thorny issues. You get not just a reasonable assessment but a structured record of your thought process.

A couple of people sitting around a laptop is another fine usage scenario. You may be predisposed to collaborate, to argue or anywhere in between It works best if users have, at least at a high level, some shared goals and values.

A meeting of a bunch of people and a projector on a big screen is another fine way to use Justify. Participants that speak without listening carefully can be formally accused of being redundant or irrelevant.

Because Justify runs in a web browser, it can be used over the net with a bunch of people, all who connect at different times. This "different times" is not simply a convenience for those who hate meetings or live in a far away time zone. Real time meetings tend to be dominated by those that can deliver superficially good sounding points quickly. More considered points may be more valuable but often take 'think time' that would be too time consuming for everyone in a real time meeting. Permitting any-time point making, is just one of the ways Justify can collect better ideas from the same group of people.

Finally, you might productively use Justify for a single discussion in all of the above ways. It can start out with an individual capturing the main points that they foresee for answering a question. They could then rope in a colleague to 'check their work' and add some missing pieces. If that is beneficial, grab a few more people and hold a structured meeting to gain the advantages of more diversified experience. If the meeting runs beyond the available time, it can be continued over the web.

Semantic Continuity

Justify guides users to enter their comments in relevant places in a structured way. Thus it is not necessary that they "follow the current thread". Participants of a given issue may enter points in different sections of the issue and they will still be perfectly organized for view by whoever wants to understand a particular area of an issue in the future. This "structured record for the future" can capture the knowledge gleaned in making a decision for use in making other related decisions. It also provides an ideal memory for "picking up where you left off" and getting new members "up to speed" on the progress of an existing team.

Once a decision has been made, often times unforeseen circumstances bring into question the wisdom of the decision. With a perfect record of the rationale for the original decision, decision makers can enter in the new information and Justify will derive new assessments that encompass both the old and the new points.

Decision-Making Strategies

There is no 'one format fits all' when it comes to problem solving and making tough decisions. Justify provides a number of techniques:

Organization

Writing down arguments in a structured fashion can help solve many problems. By providing a flexible outline format, Justify keeps your points orderly and easy to summarize. Justify gives special support for meetings via agenda and action items, automatically percolating undone action items up the hierarchy.

Point Categorization

The type of each point must be declared at least to one of several broad categories. The main categories are:

Within each category there are many kinds of points that can be made, but the key here is that you can't make a point without declaring what type of point it is and what it is directly relevant to. Imagine how much more productive meetings would be if all points came with those characteristics!

Assessment Computation

In most meetings, the myriad of ideas articulated often leaves us so confused that we gravitate towards the latest points rather than the totality of what's been said.

By using pros and cons, Justify computes which cons have themselves had cons raised against them and thus now ignored, which ideas have no remaining support and which pros have survived the onslaught of criticism. We are free to focus on the point at hand, and let Justify keep track of history and its logical conclusions. Justify does not suffer the human cognitive bias of "latest is most important". Humans, aware of this bias will often repeat themselves, drawing out the meeting while adding nothing. Using Justify, not only are repetitions useless, they can be explicitly pointed out, which leads to a decrease in repetition.

Question

Most discussions in Justify are headed by a question. This frames the issue or problem to be considered.

Justify supports questions of type: categorize, clarification, how much, prioritize, and pro or con and a few more. The type indicates the expected type of the assessment as well as the algorithm for computing the question's assessment. Questions usually have subpoints of candidate answers, each of which may have subpoint rationales of pro and con, amongst others.

The most important type of question is prioritize. It allows you to rank candidate answers utilizing criteria to score various aspects of the candidate solutions. Using criteria to 'divide and conquer' complex questions can be done as many levels deep as you like. There's a quite flexible way to 'weight' these criteria. Justify does the math and presents you with an ordered (and scored) list of answers.

Cancel_out

When weighing the importance of various points, it's quite difficult to "add up" all the pros and cons to arrive at a solution. One mechanism Justify uses is called cancel_out. By declaring that a pro point cancels out a con point, they are both ignored. You can have multiple cancel_out points within a given discussion, each of which may compare several pros to perhaps a different number of cons. By reducing the number of remaining rationales, Justify helps you better understand the most favorable option. We have Ben Franklin to thank for this insightful technique. In Justify, meta/cancel_out points can be made moot like other points, allowing you to debate the merit of such matches in a formal way. I hope Ben would appreciate this extension.

Math

Spreadsheets are marvelous tools for keeping track of numbers. But what if we are uncertain or even disagree about some of those numbers? Spreadsheets don't provide a forum for debate about those numbers. Justify provides the most common math functions of a spreadsheet in a hierarchical format and gives users the ability to challenge each number and/or operation. For example, when summing the cost of a list of parts, Justify will exclude those parts that have been deemed unnecessary by the automatic assessment computed from the rationale against a part entered by users.

Consensus

Consensus is a process by which all participants must agree for a decision to be made.In practice, it is often a rather loose process. Justify helps tighten it up by requiring that each blocking concern be expressed definitively followed by rationale for and against it. Each of those rationales can themselves have rationale until the issue is approved or blocked for good reason.

Consensus requires that all individuals in a meeting agree for a resolution to pass. It is nice to achieve this, but Justify doesn't require that participants know or believe in the consensus process. It does, however, help to achieve consensus by forcing points made to be discrete, typed, placed accurately in context and have assessments. This can help users find common ground, narrow though it may be. Consensus is but one of many processes you can use in modeling decisions.

Vote

In addition to rational decision making, Justify also has voting. But rather than degrade to the most familiar kind of voting, Justify also supplies several more sophisticated options which help select more desirable outcomes for more voters. Voting is considered to be the decision-making tool of first resort in modern democratic societies. Justify considers voting the tool of last resort, to be used only when reasoning fails.

Mix and Match

The above techniques need not be used to the exclusion of others. Justify provides a framework where all of these may be employed in a single issue. Employing the most appropriate techniques for a given situation takes some skill. Justify supports learning while doing via focused on-line help.

Copper Bullet

Justify seeks to efficiently harness the collective intelligence of the people in a discussion. It is not a silver bullet, but if you use it well, it can organize the best thoughts of users more effectively than other tools. Using it well means NOT treating it as a typical web forum or blog where casual 'conversational' style prose is rapidly entered. Create concise, carefully considered, accurately arranged bullet points, not paragraphs.

Justify relies on the intelligence of its participants. Therefore having the right people in a discussion can make the difference between a commonly acceptable result and a truly innovative solution.(Please observe that a roomful of disorganized geniuses rarely translates into a product worthy of the individuals.) At the outset of a discussion, you might not even know what expertise is needed. As the discussion unfolds, it may become clear that there are 'knowledge holes' that the team can't fulfill. But the team can use Justify to define those holes and choose who would be the best individuals to fill them. Picking new team members is, after all, just another decision to make that Justify can help with.

Conclusion

Making decisions is the single most important activity humans do. Justify is a programming language that helps people make optimal decisions. It augments reasoning by providing a framework for organizing typed points. From these points, Justify is able to automatically draw logical conclusions, permitting humans to focus on entering correctly typed points with corresponding rationale.