Daljit Dhadwal

Ideas, books, software, and other useful tools

Summarizing values from multiple columns into one column in Excel

When extracted, data collected using radio buttons sometimes comes out as a TRUE/FALSE column for each option.

For example, data from the following question:

Radio_Button_Example

may look like this in the extract:

Radio_Button_Extract

Data that is spread over multiple columns like this can be hard to analyze (e.g., pivot tables don’t work well with this kind of data). If you have data that looks like this, you can easily determine which option was selected and collapse the three columns of data into one column. This is assuming that one value is TRUE (this will be the case if a response is required). Once the data is in one column, it’s easy to summarize the information using a pivot table.

We’ll create a new column called Q1_ eaten_cereal that will have a value of Q1_eaten_cereal_yes, Q1_eaten_cereal_no, or Q1_eaten_cereal_dont_remeb.

Radio_Button_Extract_Collapsed

The following Excel formula in cell E2 will determine which option was selected:

=INDEX($B$1:$D$1,1,MATCH(TRUE,B2:D2,0))

The MATCH(TRUE,B2:D2,0) part of the formula looks for the TRUE value in cells B2 to D2. It returns a 1 because TRUE is in the first cell. The outer part of the formula becomes INDEX($B$1:$D$1,1,1) and this will return the item in row 1 and column 1 from cells B1 to D1 (the row containing the text for the radio button options).

Revising your work

As DeLillo matured as a writer, his detritus increased . . . . in the early eighties, he began . . . to type each paragraph over and over, often on its own page, so that within a draft a paragraph may appear a dozen times on a dozen sheets, as he works it to his satisfaction. . . . The process gives DeLillo’s drafts a highly deliberate pace, like a blind man tapping his way forward. . . .

The painstaking nature of DeLillo’s method can be seen in his drafts for “Underworld” . . . . He goes through a dozen pages to settle on the language of the opening two paragraphs, in which a Harlem teen-ager named Cotter Martin gets ready to jump the turnstile at the Polo Grounds to see the famous 1951 Dodgers-Giants playoff game. . . . “It’s a school day, sure, but he’s nowhere near the classroom. The longing to be here, standing in the shadow of this old rust-hulk of a structure, is too hard to resist—this metropolis.” DeLillo breaks off and starts again: “It’s a school day, sure, but he’s nowhere near the classroom, the box of forty blank faces.” He pauses, then alters the image to “the box of forty mismatched heads.” He returns to his original riff: “It’s a school day, sure, but he’s nowhere near the classroom and it’s not a matter of midweek blues.” Then he drops “midweek blues,” but introduces the idea of melancholy in a lovely pair of sentences: “Most longings go unfulfilled. This is the word’s wistful implication.” He transforms these two sentences into one: “Longing on a large scale is what makes history.”

Five years later, DeLillo turns to these words again . . .

An excerpt from the 2007 New Yorker article Why do the archives of so many great writers end up in Texas?

Putting in the time

What [Hendrik] Bode was saying was this: “Knowledge and productivity are like compound interest.” Given two people of approximately the same ability and one person who works ten percent more than the other, the latter will more than twice outproduce the former. The more you know, the more you learn; the more you learn, the more you can do; the more you can do, the more the opportunity – it is very much like compound interest. I don’t want to give you a rate, but it is a very high rate. Given two people with exactly the same ability, the one person who manages day in and day out to get in one more hour of thinking will be tremendously more productive over a lifetime. I took Bode’s remark to heart; I spent a good deal more of my time for some years trying to work a bit harder and I found, in fact, I could get more work done.

The above is an excerpt from You and your research, a talk Richard Hamming gave in 1986. Hamming, who passed away in 1998, was a mathematician and computer scientist. One among the many things he did was develop techniques for minimizing errors when transmitting data. You can see him in the documentary Logic by Machine (Part 2)

Managing your time

Under time pressure

Credit: ©iStockphoto.com/parema

One of the core tenets of organizational behaviour is that there is no single best way to do something. The best way to do something quite often depends on the specifics of the situation. What works under one set of circumstances may not work under a different set of circumstances.

For example, a highly directive management style might work well with a team where everyone is new and inexperienced but probably wouldn’t be effective with a team where everyone is already an expert (it would be perceived as micromanaging).

This idea—that the best thing to do depends on the specific situation—is the key idea in contingency theory.

Time management is another area where there is no single best method.

For example, some jobs consist of working on a single or very small number of projects over a long period of time with few interruptions, while other jobs consist of having to quickly shift from working on one urgent request to another, leaving very little uninterrupted time to work on longer term projects. A time management system that works in the first situation may not be very useful in the second. I think part of the reason there are so many books on time management (searching for “time management” on Amazon.com returns over 87,000 results) is because there is no single best method. Different authors have discovered systems and techniques that have worked for them in their particular environments.

In recent years, David Allen’s Getting Things Done, usually abbreviated as GTD, has come to dominate the time management category. A vibrant internet community has developed around discussing and expanding on the ideas presented in the book. At the heart of the book is a simple flowchart that describes how to record all of the things you need to do. One of the big ideas in GTD is that everything you need to do shouldn’t just be dumped into one big to-do list. In the GTD system, there are many different kinds of lists (and places) to record what you need to do. For example, in GTD you keep separate lists for phone calls you need to make, books you need to read, and groceries you need to buy. Stuff that doesn’t need to be acted on but may be useful in the future (such as office supply catalogs) gets filed. The idea is to get everything out of your mind and on paper (or some sort of electronic list). There’s a good summary of GTD on the WikiSummaries site. The following 46 minute video is a very high level introduction to GTD by David Allen.

Along with having different kinds of lists, the other big idea in GTD is to hold (minimally) a weekly review of all the stuff you’re working on. Time management systems are very fragile. Sometimes it just takes a few hours of a large number of e-mails and phone calls coming in to overwhelm the system and cause it to collapse. The weekly review is the ongoing repair and maintenance that is needed to keep the system functioning. Here’s an MP3 recording of David Allen and his staff discussing the importance of the weekly review (here’s a list of free podcasts available from the David Allen website).

The idea of having different kinds of lists for different types of things is taken to an extreme by Michael Linenberger. Linenberger has created an entire system to manage a very specific type of to-do list. Linenberger’s system is called Master Your Now (MYN). If most of the things you need to do come via e-mail and most of your time is spent jumping from one urgent thing to another, then the MYN system is for you. The MYN system is a method for managing a to-do list that only deals with urgent ad-hoc tasks.

Most people think of Microsoft Outlook as a program used to read e-mail. In reality, it’s a database system that can be used to manage all kinds of information.  One of the most underused features of Outlook is its ability to convert one type of item into another type of item. For example, Outlook can turn e-mails into calendar invites or tasks and vice versa. Linenberger’s Total Workday Control Using Microsoft Outlook exploits this functionality to create an incredibly powerful and easy-to-use implementation of the MYN system. The following video gives a quick overview of the MYN system.

In some ways, Linenberger is the anti-Steven Covey. Covey, the author of The Seven Habits of Highly Effective People, popularized the idea of the mission statement (Habit 2: Begin with the End in Mind), and then spending most of your time working on those things that are aligned with your mission statement (Habit 3: Put First Things First) rather than spending most of your time doing whatever happens to be urgent at the moment. Linenberger takes the opposite approach. The MYN system is about working on whatever happens to be urgent at the moment. The focus on urgency leads to two important ideas: the planning horizon is very short in workplaces where most things are urgent (Linenberger says that at most things can only be planned two weeks ahead); and since new urgent things are always overtaking whatever was previously urgent, managing start dates becomes more important than managing deadlines. Linenberger’s system is essentially a method for managing when you’ll actually be able to start things. When new urgent tasks come in, the start dates for less urgent tasks get pushed out further into the future.

In their respective books, Getting Things Done and Total Workday Control Using Microsoft Outlook, David Allen and Michael Linenberger present many other ideas on time management in addition to the ideas discussed here that are sure to be useful. Available for less than $20 each, both books are worth reading.

Do performance measurement systems actually improve performance?

Performance measurement systems are widely used by organizations in both the public and private sectors, with the objective of improving decision making and performance. The cost associated with building and implementing these systems is high, yet many fail to deliver the expected results. This paper proposes an evaluation method and framework using an evidence‐informed theory of change. The proposed theory of change can be used to guide the evaluation of the effectiveness of implemented performance measurement systems or to inform the design of new systems.

This is from the abstract to A framework for evaluating the effectiveness of performance measurement systems by Lori Criss Powers [via EvalTalk mailing list].

Copying tables from Word to Excel

Every once in a while, a situation comes up where there’s a need to copy a table made in Microsoft Word to Excel. If the Word table has multiple lines of text within a cell that have been separated by a carriage return (i.e., the Enter key has been typed at the end of a line of text), then the table won’t properly copy over. When Excel encounters a carriage return, the cursor will move into a new row.

For example, in the following Word table, the Enter key has been typed at the end of every line in the address column.

Table in Word

If you copy the above into Excel, you’ll get this:

Table in Excel

The city and postal codes are copied into their own cells.
Data like this is hard to work with (e.g., you can’t sort this data).

Here’s how to solve this problem.

In Word replace the paragraph markers (i.e., the Enter key character) with some other symbol. For example, use ZZ.

In Word, type Ctrl H to open the Replace dialog box. Make sure to press the more button to see the advanced options.

Word Search and Replace

The table in Word will now look this:

Final Word Table

Copy this table into Excel.

Table in Excel_2

Now, each person record only takes up one row. Next we just have to replace the ZZ with line breaks. In Excel type Ctrl H to open the Replace dialog box. You’ll need to type in the code for the new line character using the numeric keypad. Make sure the Num Lock key has been pressed.

Excel Search and Replace

Here’s what the final result looks like in Excel.

Final Excel Table

The planning fallacy

From the Wikipedia article on the planning fallacy:

The planning fallacy is a tendency for people and organizations to underestimate how long they will need to complete a task, even when they have past experience of similar tasks over-running. The term was first proposed in a 1979 paper by Daniel Kahneman and Amos Tversky. Since then the effect has been found for predictions of a wide variety of tasks, including tax form completion, school work, furniture assembly, computer programming and origami. The bias only affects predictions about one’s own tasks; when uninvolved observers predict task completion times, they show a pessimistic bias, overestimating the time taken. In 2003, Lovallo and Kahneman proposed an expanded definition as the tendency to underestimate the time, costs, and risks of future actions and at the same time overestimate the benefits of the same actions. According to this definition, the planning fallacy results in not only time overruns, but also cost overruns and benefit shortfalls.

You can read the entire article at Wikipedia.

The autosum keyboard shortcut in Excel

Pressing the Alt key and equal sign key (i.e., first press the Alt key and then while keeping it pressed, press the equal key) will autosum a contiguous range of cells. In the following example, the numbers 10, 20, 30 are typed into cells B2, B3, and B4 respectively. Then in cell B5, first the Alt and Equal sign keys are pressed and then the Enter key is pressed.

Autosum Shortcut in Excel

Ray Dalio’s Principles

Recently, both New York and The New Yorker profiled Ray Dalio. Dalio is the founder of Bridgewater Associates. Bridgewater runs the world’s largest hedge fund. Both articles provide fascinating and quite entertaining glimpses into the corporate culture at Bridgewater. Both articles, but the one in New York more so, focus on a document Dalio wrote explaining his management (and life) philosophy.

From New York magazine:

A corporate gospel isn’t novel—The Bloomberg Way was required reading for Bloomberg News employees years before Dalio set down his ideas. But Principles is no mere training manual. The first half of the book, “My Most Fundamental Principles,” contains Dalio’s abridged autobiography, an apologia for radical transparency, and a step-by-step guide to “personal evolution.” The precepts in this section—many of them written in a digressive, self-serious style that reads as if Ayn Rand and Deepak Chopra had collaborated on a line of fortune cookies—are never about making money, at least not openly. They’re about following rules, learning from mistakes, reaching your goals. Doing things in the Bridgewater way is thought to be the best—and possibly only—way of achieving these objectives.

Links to the articles:

New York: The Billion-Dollar Aphorisms of Hedge Fund Cult Leader Ray Dalio
The New Yorker: Mastering the Machine: How Ray Dalio built the world’s richest and strangest hedge fund

Dalio has posted his Principles (a 123 page PDF) on the Bridgewater Associates website.

The J programming language

Glowing J

Credit: ©iStockphoto.com/Franck Boston

J is an obscure, but potentially very useful programming language for certain types of tasks. It’s a calculator language, and its specialty is manipulating arrays. J was created by some of the same people who made APL. Here’s a great primer on getting started with J, and here are some books on learning the language. You can download J for free, and it recently became an open source project.