On Good Judgment and Decision-Making: The Science and Practice

Imagine you’ve made two decisions. One has resulted in a loss of 10 thousand dollars, while the other resulted in a gain of 10 thousand dollars. Would you say that the former was a bad decision, and the latter was a good one?

The first response that usually comes to everyone’s mind is “of course, the second decision was better”. But this is not necessarily the case. Can you think of a scenario where your response would be the opposite?

I’ve been thinking and reading about decision-making for many years while trying to put everything I learned into practice. Below, I summarize the strategies and mental models that I personally found most useful.

The Process vs. The Outcome

We all have a natural tendency to judge decisions based on their outcomes. This is not the worst heuristic, as there is a correlation between the quality of decisions and outcomes. But this heuristic has a major flaw — it doesn’t account for luck and incomplete information.

Continue reading

Learning How to Learn (And 20+ Studies)

I’ve been interested in cognitive science and effective learning methods for years. I’ve read multiple books and articles and put many ideas to test. So I thought I’d synthesize my notes into the blog post, add references to scientific studies, and share it with you.

TL;DR

Effective Learning Strategies

  • Distributed learning: study less in each session but more frequently
  • Active recall: actively test your knowledge and skills
  • Distributed recall: space the tests in time and adjust the intervals based on performance
  • Interleaving: practice multiple related yet different skills/concepts simultaneously
  • Elaborative interrogation (quiz-and-recall): ask yourself questions and use the material you’ve learned to answer them
  • Self-explanation and the Feynman technique: explain what you’ve just learned in simple terms

Physiology and Brain’s Health

  • Sleep
  • Exercise
  • Nutrition

Disclaimer and Introduction

I have no formal background in cognitive science or neuroscience and this has been more of a side interest. My understanding is limited and I still need to learn how to effectively and consistently apply all these ideas to practice.

That being said, I found some of the methods described in this article very useful. For example, I’ve used them to learn foreign languages, the basics of programming, and various disciplines covered during the two-year MBA program.

Effective Learning Strategies

Strategy #1: Distributed (Spaced) Learning Practice

In short, it’s better to distribute one’s practice over a period of time than cram it into one day.

In one study elementary school students were asked to study in one of the three ways: massed, clumped, and spaced.

  • Massed = four lessons at a time
  • Clumped = two lessons on one day and two lessons on the next day
  • Spaced = one lesson per day for four days

The “spaced” group performed best, followed by the “clumped” group:

Another study compared comprehension scores under three different conditions:

  1. Read a text once (“single”)
  2. Read a text twice (“massed”)
  3. Read a text twice with a week-long gap (“distributed”)

When tested immediately, the second group performed best. But when tested with a delay of two days, the third group performed best.

This method is also superior for learning motor skills.

How to apply this in practice:

Create a learning schedule or find time to practice a little bit every day or every few days instead of cramming all your learning into one or just a few days.

If you’d like to learn more, read the Wikipedia article on distributed practice.

Strategy #2: Active Recall (Retrieval) Practice

It might be more effective to actively retrieve the information you’ve already learned than passively re-read or try to learn it once again.

One study that compared a method that emphasized study sessions with a method that emphasized tests and found the latter to be more effective for delayed recall.

  • SSSS = four study sessions
  • SSST = three study sessions, followed by one test
  • STTT = one study session, followed by three tests

Even imagining that you might be tested on the material you’re learning might help improve the recall.

How to apply this in practice:

If a few days ago you learned how past tense works in the Spanish language, try to remember the rules or even test yourself on your knowledge — instead of simply re-reading the same material once again.

You can read more about the active recall practice on Wikipedia.

Continue reading

Book Notes: How Google Works by Eric Schmidt

How Google Works” by Eric Schmidt, despite being quite self-congratulatory and maybe even prone to confirmation bias, is full of inspirational ideas and bids of practical wisdom to learn from. I took a few (ok, quite a few) notes on smart creatives, decision making, hiring, innovation, strategy, career, management and even managing email.

On smart creatives:

“And who, exactly, is this smart creative? A smart creative has deep technical knowledge in how to use the tools of her trade and plenty of hands-on experience. In our industry, that means she is most likely a computer scientist, or at least understands the tenets and structure of the systems behind the magic you see on your screens every day. But in other industries she may be a doctor, designer, scientist, filmmaker, engineer, chef, or mathematician. She is an expert in doing. She doesn’t just design concepts, she builds prototypes. She is analytically smart. She is comfortable with data and can use it to make decisions. She also understands its fallacies and is wary of endless analysis. Let data decide, she believes, but don’t let it take over.

She is business smart. She sees a direct line from technical expertise to product excellence to business success, and understands the value of all three. She is competitive smart. Her stock-in-trade starts with innovation, but it also includes a lot of work. She is driven to be great, and that doesn’t happen 9-to-5. She is user smart. No matter the industry, she understands her “get it right the next time around. She is self-directed creative. She doesn’t wait to be told what to do and sometimes ignores direction if she doesn’t agree with it. She takes action based on her own initiative, which is considerable.

She is open creative. She freely collaborates, and judges ideas and analyses on their merits and not their provenance. If she were into needlepoint, she would sew a pillow that said, “If I give you a penny, then you’re a penny richer and I’m a penny poorer, but if I give you an idea, then you will have a new idea but I’ll have it too.” Then she would figure out a way to make the pillow fly around the room and shoot lasers.

She is thorough creative. She is always on and can recite the details, not because she studies and memorizes, but because she knows them. They are her details. She is communicative creative. She is funny and expresses herself with flair and even charisma, either one-to-one or one-to-many.”

 

HIPPOs:

“Hippopotamuses are among the deadliest animals, faster than you think and capable of crushing (or biting in half) any enemy in their path. Hippos are dangerous in companies too, where they take the form of the Highest-Paid Person’s Opinion. When it comes to the quality of decision-making, pay level is intrinsically irrelevant and experience is valuable only if it is used to frame a winning argument. Unfortunately, in most companies experience is the winning argument. We call these places “tenurocracies,” because power derives from tenure, not merit. It reminds us of our favorite quote from Jim Barksdale, erstwhile CEO of Netscape: “If we have data, let’s look at data. If all we have are opinions, let’s go with mine.”

When you stop listening to the hippos, you start creating a meritocracy, which our colleague Shona Brown concisely describes as a place where “it is the quality of the idea that matters, not who suggests it.” Sounds easy, but of course it isn’t. Creating a meritocracy requires equal participation by both the hippo, who could rule the day by fiat, and the brave smart creative, who risks getting trampled as she stands up for quality and merit.”

 

On technical insights as a driver of innovation:

“Bet on technical insights, not market research. Product leaders create product plans, but those product plans often (usually!) lack the most important component: What is the technical insight upon which those new features, products, or platforms will be built? A technical insight is a new way of applying technology or design that either drives down the cost or increases the functions and usability of the product by a significant factor. The result is something that is better than the competition in a fundamental way. The improvement is often obvious; it doesn’t take a lot of marketing for customers to figure out that this product is different from everything else.

For example, at that time Google was experimenting in applying some of our expertise from online advertising to other advertising markets, including print, radio, and TV. These were clever efforts, supported by smart people, but they lacked that fundamental technical insight that would shift the cost-performance curve non-incrementally and provide significant differentiation. All three ultimately failed. And when we look back at other Google products that didn’t make it (iGoogle, Desktop, Notebook, Sidewiki, Knol, Health, even the popular Reader), they all either lacked underlying technical insights from the outset, or the insights upon which they were based became dated as the Internet evolved.”

Continue reading

Stanford Class: How to Start a Startup by YC

Free, open to everyone and highly educational Stanford class “How to Start a Startup” has just ended. But all the materials, including talks by star speakers, such as Paul Graham, Peter Thiel, Ben Horowitz, Sam Altman, Brian Chesky and others are going to be available online. For quick reference, here is the complete collection of all course materials:

Lectures

Date Speaker Topic
9/23/14 Sam Altman, President, Y Combinator
Dustin Moskovitz, Cofounder, Facebook, Cofounder, Asana, Cofounder, Good Ventures
Welcome, and Ideas, Products, Teams and Execution Part I
Why to Start a Startup
9/25/14 Sam Altman, President, Y Combinator Ideas, Products, Teams and Execution Part II
9/30/14 Paul Graham, Founder, Y Combinator Before the Startup
10/2/14 Adora Cheung, Founder, Homejoy Building Product, Talking to Users, and Growing
10/7/14 Peter Thiel, Founder, Paypal, Founder, Palantir, and Founder, Founders Fund Competition is For Losers

Continue reading

Book Notes: What Technology Wants Notes

I enjoyed reading Ben Casnocha’s notes from What Technology Wants by Kevin Kelly and took a couple of notes myself:

On inter-dependancies:

Each new invention requires the viability of previous inventions to keep going. There is no communication between machines without extruded copper nerves of electricity. There is no electricity without mining veins of coal or uranium, or damming rivers, or even mining precious metals to make solar panels.

On evolution of the scientific method:

The classic double-blind experiment, for instance, in which neither the subject nor the tester is aware of what treatment is being given, was not invented until the 1950s. The placebo was not used in practice until the 1930s. It is hard to imagine science today without these methods.

On games:

The cybernetician Heinz von Foerster called this approach the Ethical Imperative, and he put it this way: “Always act to increase the number of choices.” The way we can use technologies to increase choices for others is by encouraging science, innovation, education, literacies, and pluralism. In my own experience this principle has never failed: In any game, increase your options.

On sacrifices as a form of  investment:

As Suketu Mehta, author of Maximum City (about Mumbai), says, “Why would anyone leave a brick house in the village with its two mango trees and its view of small hills in the East to come here?” Then he answers: “So that someday the eldest son can buy two rooms in Mira Road, at the northern edges of the city. And the younger one can move beyond that, to New Jersey. Discomfort is an investment.

On using older technology as a statement:

Who would have guessed anyone would burn candles when lightbulbs are so cheap? But burning candles is now a mark of luxuriant uselessness. Some of our hardest-working technology today will achieve beautiful uselessness in the future. Perhaps a hundred years from now people will carry around “phones” simply because they like to carry things, even though they may be connected to the net by something they wear.

Continue reading

How Do Top Students Study

Planning to give this approach to taking notes and learning new subjects a try at my MBA classes.

Quoting Hooman’s response to Quora topic “How do top students study?“:

I maintained a 5.0/5.0 GPA in one of my MIT masters and 4.9/5.0 in my other MIT masters until my very last semester when I had to fly to job interviews and couldn’t attend all my classes. I achieved these grades while doing a double research load. That is I worked simultaneously for an MIT professor and two Harvard Medical School professors.

And here is his take on how he did it:

Write your notes in a way where you can test your retention and understanding. Simply put my notes can be used like flashcards because I write them in a form where I separate a “stimulus” from a “response.” The stimulus are cues or questions (think: front side of flashcard), while the response is the answer to the cue (think: back of flashcard). But the stimuli are to the left of a margin, while the responses are to the right. The key advantage of this is that just by putting a sheet of paper on top of your notes, you can hide the responses, while leaving the stimuli visible. You can have multiple margins and multiple levels of stimuli and response for greater information density. When you get good at this you can write notes in this form in real-time. To get some idea of what I’m talking about google for “Cornell Notetaking method” My notetaking method is a variant of this.

Why Berkeley-Haas

People sometimes ask for my reasons behind choosing Berkeley-Haas among other schools.

Berkeley. Is MBA worth it

Shortly, these are the reasons:

  • It is a part of a larger top university.
    • For instance, I do not think I would enjoy going to LBS or INSEAD as much. These are great business schools, but these business schools are not part of larger universities.
    • There is something special about being on campus with ~30 thousand smartest students from all over the world who also major in fields other than business.
    • For example, UC Berkeley is currently top 1-2 in Computer Science, top 1-5 in Physics and top 1-3 in Mechanical Engineering.
  • Location, location, location. Not only because it is sunny California, but because being close to Silicon Valley opens many career opportunities that are highly germane to my vision.
  • Focus on technology and entrepreneurship.
  • Believe it or not, but the I find the defining principles personally appealing.
  • Great brand and good rankings do not hurt as well.
    • But in particular, I liked the fact that Haas ranks well on the “Student evaluation” axis, it is #2 in “Students’ evaluation of career services” and quite high overall “Student rating of the program (out of 5)”, 4.56, according to The Economist 2013 rankings.
  • I really want to add “people” to this list of Berkley-Haas strengths, but this would be probably unfair as I cannot compare to other schools as I could not know it for sure before starting the program.

The list could go on and on and on, but these are probably the main ones top-of-mind. Of course, there are drawbacks as well, but no one schools is perfect.

Let me know what you think!

Memrise: learning languages mnemonically using spaced repetition

You might remember my recent post about “Moonwalking with Einstein” book about memory and mnemonics.

What you probably did not know though is that one of the characters, winner of multiple memory championships, Ed Cooke, launched his own web startup Memrise.com

I first read about it in The Guardian article “How I learned a language in 22 hours” by the book author, Joshua Foer. Although, the title is somewhat misleading, the approach is very interesting. So, what is so special about Memrise? They have an interesting learning model. Primarily, the focus is on languages, but there is a whole range of secondary courses, ranging from “How to say I love you in 100 languages” to HTML5.

memrise

The first principle: spaced repetition. The words you are asked to remember are spaced in time in a precise manner with intervals calculated based on your past performance. By the way, the same principle is employed by simple app Anki which I once briefly mentioned in my post about GMAT.

The second principle: mnemonic. You are usually shown a picture or an idea that helps you remember a given word using associations. These so-called mems are added by members of community in the best crowd-sourcing traditions. The most voted-up are displayed.

And of course, they also use gamification, who does not?

Overall, it appears to be an interesting tool. However, it remains to be seen whether it is really a good idea to invest one’s time in it without a firm intention to actually learn a language. But learning to read a Chinese menu sounds like a fun thing to try anyway, all serious goals aside.

By the way, Memrise is also discussed on Quora which I recently wrote about.

Venture Lab: free Stanford University online courses

I devoted some of my time to self-education in September-December using the opportunity provided by Stanford University Venture Lab‘s online courses. In this post I would like to briefly describe my impressions and share some useful materials from the courses. Venture Lab courses are somewhat different from other online courses in that they take place during a specific time with specific deadlines, and you need to enroll in order to participate.

I tried three courses: Technology Entrepreneurship, Crash Course on Creativity, and Finance. As they are very different, I will tell you about each one separately.

Technology Entrepreneurship

This is probably the best one. Although, there are multiple opinions on whether entrepreneurship can be taught, there are a lot of useful materials.

Lectures certainly vary in content. Some are quite theoretic and contain, for instance, large charts showing, how probability of team’s success depends on the number of members or on their background. Others are more about anecdotal. There was, for example, a story about history of skateboarding culture. I didn’t know that it originated from dry Californian summer during which many Californian pools were empty and people started using them to ride inside instead of swimming.
Aside from theory, there is also a team work that is highly encouraged and is in fact an integral part of learning process. Working in teams you are supposed to do market research, create and test the business model. Participants are very diverse. There are people from Pakistan, Belarus, Serbia, Russia, Canada and many other countries. You can pick any of already existing teams (hundreds of them) or create your own and invite other students to join.

If it is too late for you to participate, you might still benefit from these materials:

I think it takes about an hour a week or so to keep up with the lectures. But if you want to do more, for example communicate on forum, read recommended books, and actually try to apply ideas to some project, it will take longer, of course.

By the way, if you are interested in this topic, I highly recommend you these recent notes from Peter Thiel’s lectures. They also happen to be in Stanford. Computer Science dep. It will save another hundred thousand dollars. 😉

Continue reading