Social reasoning about two clusters of smart people

Here’s a sketch. All of the following are generalizations, and some are wrong.

There are rationalists.

The rationalists are unusually intelligent, even I think, for the tech culture that is their sort of backdrop. But they are, by-and-large kind of aspy: on the whole, they are weak on social skills, or their is something broken about their social perceptions (broken in a different way for each one).

Rationalists rely heavily on explicit reasoning, and usually start their journeys pretty disconnected from their bodies.

They are strongly mistake theorists.

They have very very strong STEM-y epidemics. They can follow, and are compelled by arguments. They are masterful at weighing evidence and coming to good conclusions on uncertain questions, where the there is something like a data-set or academic evidence base.

They are honest.

They generally have a good deal of trust and assumption of good faith about other people, or they are cynical of humans and human behavior, using (explicit) models of “signaling” and “evo pysch.”

I think they maybe have a collective blindspot with regards to Power, and are maybe(?) gullible (related to the general assumption about good faith). I suspect that rationalists might find it hard to generate the hypothesis that “this real person right in front of me, right now, is lying to me / trying to manipulate me.”

They are, generally, concerned about ex-risk from advanced AI, and track that as the “most likely thing to kill us all”.

 

There’s also this other cluster of smart people. This includes Leverage-people, and some Thiel people, and some who call themseleves post rationalists.

They are more “humanities” leaning. They probably think that lots of classic philosophy is not only good, but practically useful (where some rationalists would be apt to deride that as the “rambling of dead fools”).

They are more likely to study history or sociology, than math or Machine Learning.

They are keenly aware of the importance of power and power relations, and are better able to take ideology as object, and treat speech as strategic action rather than mere representation of belief.

Their worldview emphasizes “skill”, and extremely skilled people, who shape the world.

They are more likely to think of “beliefs” as having a proper function doing something other than reflecting the true state of the world, for instance, facilitating coordination, or producing an effective psychology. The rationalist would think of instrumentally useful false beliefs as something that is kind of dirty.

They tend to get some factual questions wrong (as near as I can tell): one common one is disregarding IQ, and positing that all mental abilities are a matter of learning.

These people are much more likely to think that institutional decay or civilizational collapse is more pressing than AI.

 

It seems like both these groups have blindspots, but I would really like to have a better sense of the likelyhood of both of these disasters, so it would be good if we could get all the virtues into one place, to look at both of them.

 

 

A view of the main kinds of problems facing us

I’ve decided that I want to to make more of a point to write down my macro-strategic thoughts, because writing things down often produces new insights and refinements, and so that other folks can engage with.

This is one frame or lens that I tend to think with a lot. This might be more of a lens or a model-let than a full break-down.

There are two broad classes of problems that we need to solve: we have some pre-paradigmatic science to figure out, and we have have the problem of civilizational sanity.

Preparadigmatic science

There are a number of hard scientific or scientific-philosophical problems that we’re facing down as a species.

Most notably, the problem of AI alignment, but also finding technical solutions to various risks caused by bio-techinlogy, possibly getting our bearings with regards to what civilization collapse means and how it is likely to come about, possibly getting a handle on the risk of a simulation shut-down, possibly making sense of the large scale cultural, political, cognitive shifts that are likely to follow from new technologies that disrupt existing social systems (like VR?).

Basically, for every x-risk, and every big shift to human civilization, there is work to be done even making sense of the situation, and framing the problem.

As this work progresses it eventually transitions into incremental science / engineering, as the problems are clarified and specified, and the good methodologies for attacking those problems solidify.

(Work on bio-risk, might already be in this phase. And I think that work towards human genetic enhancement is basically incremental science.)

To my rough intuitions, it seems like these problems, in order of pressingness are:

  1. AI alignment
  2. Bio-risk
  3. Human genetic enhancement
  4. Social, political, civilizational collapse

…where that ranking is mostly determined by which one will have a very large impact on the world first.

So there’s the object-level work of just trying to make progress on these puzzles, plus a bunch of support work for doing that object level work.

The support work includes

  • Operations that makes the research machines run (ex: MIRI ops)
  • Recruitment (and acclimation) of people who can do this kind of work (ex: CFAR)
  • Creating and maintaining infrastructure that enables intellectually fruitful conversations (ex: LessWrong)
  • Developing methodology for making progress on the problems (ex: CFAR, a little, but in practice I think that this basically has to be done by the people trying to do the object level work.)
  • Other stuff.

So we have a whole ecosystem of folks who are supporting this preparadgimatic development.

Civilizational Sanity

I think that in most worlds, if we completely succeeded at the pre-paradigmatic science, and the incremental science and engineering that follows it, the world still wouldn’t be saved.

Broadly, one way or the other, there are huge technological and social changes heading our way, and human decision makers are going to decide how to respond to those changes, possibly in ways that will have very long term repercussions on the trajectory of earth-originating life.

As a central example, if we more-or-less-completly solved AI alignment, from a full theory of agent-foundations, all the way down to the specific implementation, we would still find ourselves in a world, where humanity has attained god-like power over the universe, which we could very well abuse, and end up with a much much worse future than we might otherwise have had. And by default, I don’t expect humanity to refrain from using new capabilities rashly and unwisely.

Completely solving alignment does give us a big leg up on this problem, because we’ll have the aid of superintelligent assistants in our decision making, or we might just have an AI system implement our CEV in classic fashion.

I would say that “aligned superintelligent assistants” and “AIs implementing CEV”, are civilizational sanity interventions: technologies or institutions that help humanity’s high level decision-makers to make wise decisions in response to huge changes that, by default, they will not comprehend.

I gave some examples of possible Civ Sanity interventions here.

Also, think that some forms of governance / policy work that OpenPhil, OpenAI, and FHI have done, count as part of this category, though I want to cleanly distinguish between pushing for object-level policy proposals that you’ve already figured out, and instantiating systems that make it more likely that good policies will be reached and acted upon in general.

Overall, this class of interventions seems neglected by our community, compared to doing and supporting preparadigmatic research. That might be justified. There’s reason to think that we are well equipped to make progress on hard important research problems, but changing the way the world works, seems like it might be harder on some absolute scale, or less suited to our abilities.

 

 

 

 

Why is the media consumption of adult millennials the same as it was when they were children?

[Random musings.]

Recently, I’ve seen ads for number of TV shows that are re-instantiations of TV shows from the the early 2000s, apparently targeted at at people in their late twenties and early thirties, today.

For instance, there’s a new Lizzie Mcguire show, that follows a 30-year-old Lizzie as a practicing lawyer. (In the original show, she was a teenager in high school.) In a similar vein, there’s a new That’s So Raven Show, about Raven being a mom.

Also, recently, Disney released a final season of Star Wars the Clone Wars (which ran from 2008 to 2014).

These examples seem really interesting to me, because this seem like a new phenomenon. Something like, Millennials unironically like and are excited about the same media that they liked when they were kids. I think think this is new. My impression is that it would be extremely unusual for a 30 year-old in 1990, to show similar enthusiasm for the media they consumed as a 12 year old. I imagine that for that person there is a narrative that you are supposed to “grow out of childish things”, and a person who doesn’t do that is worthy of suspicion. (Though I wasn’t there in 1990, so maybe I’m miss-modeling this.)

My impression (which is maybe mistaken), is that Millennials did not “grow up” in the sense that earlier generations did. Instead of abandoning their childhood interests to consume “adult media”, they maintained their childhood interests into their 30s. What could be going on here?

  • (One thing to note is that all three of the examples that I gave above are not just Disney properties, but specifically Disney+ shows. Maybe this is a Disney thing, as opposed to a Millennial thing?)

Some hypotheses:

  • One theory is that in the streaming era, demographics are much more fragmented, and there is an explosion of content creation for every possible niche, instead of aiming for broad appeal. So while there always would have been some people who are still excited about the content from their childhood, now media companies are catering to that desire, in order to capture that small demographic.
  • Another possibility is that the internet allowed for self-sustaining fandoms. In the past, if you liked a thing, at best you could talk about it with your friends, until that content ended and your friends moved on. But with the internet, you could go on message boards, and youtube, and reddit, and be excited about the things you love, with other people who love those things, even decades after they aired. The internet keeps your childhood fresh and alive for you, in a way that wasn’t really possible for previous generations.
  • Maybe being a geek became destigmatized. I think there is one group of adults in 1990 that would be unironically excited about the content that they enjoyed as kids and teen-agers: Nerds, who still love Star Wars, or Star Trek, or comic books, or whatever. (I posit that this is because nerds tend to like things because of how natively cool they seem, which is pretty stable over a lifetime, as opposed to tracking the Keynesian beauty contest of which things are popular with the zeitgeist / which things are cool to like, which fluctuates a lot over years and decades.) For some reason (probably related to the above bullet point), being a geek became a lot less socially stigmatized over the early 2000s, and there was less social backlash for liking nerdy things, and for being unironically excited about content that was made for children.
    • I feel like there is deeply related to sex. I posit that the reason that most young men “grow out of childish things”, is that when they become interested in girls, they start to focus near-exclusively on getting laid, and childish interests are a liability to that. (Nerds either 1) care more about the things that they like, so that they are less willing to give them up, even for sex or 2) are more oblivious of the impact that their interests have on their prospects for getting laid). But I have the sense that unironically liking your childhood media is less of a liability to your sex-life in 2000, than it was in 1990, for reasons that are unclear.
    • (Again, maybe it is because the internet allows people to live in communities that that also appreciate that media, or maybe because nerds provided a ton of social value and can get rich and successful, so being a nerd is less stigmatized on the dating market, or maybe because special effects got so good that the things that were cool to nerds are now more obviously cool to everyone (eg superhero movies have mass appeal).
  • Maybe the content from the early 2000s is just better, in some objective sense, than the content of the 1970s – 1980s. Like maybe my dad grew out of the content that he watched as a kid, because it was just less sophisticated, where as the content that my generation watched as kids, is more interesting to adults?
  • Maybe the baby boomers had an exciting adult world to grow into, which was more compelling than their childhood interests. Millennials feel adrift in the world, and so default to the media they liked as kids, because they don’t have better things to do?

 

Impressions on what is happening in the US in this decade

Epistemic status: wild-eyed inside view impressions, based on narrative and stereotype, mostly devoid of hard facts. I expect hard facts to change my view. [I also note that this story basically accords with my grey-tribe ideology / worldview.]

Over the past couple of weeks, I’ve started following twitter for the first time, and (relatedly) I’ve been reading a lot of Venkatesh Rao’s writing, which is mostly new to me. This combined with the fact that history in 2020 seems to be “moving faster” than it used to, has caused me to start thinking about society, and social class, and some other topics that I’ve not thought much about before.

This shitty post represents an outline of my current, tentative, view of what the heck is going on in the United States these days. As noted in the epistemic status, this synthesis is based on my subjective impressions watching the world unfold, more than rigorous analysis, and accordingly I’m missing a lot of complexity, at best, and totally off base at worst. But nevertheless, this kind of unrigorus making sense of things seems like a good starting point, to guide my empiricism.

Optimism and social stability

There’s a lot of talk about inequality, and the social unrest that income inequality foments. But I think that inequality of income, or of wealth, is a little bit of a red herring. I posit that social unrest mostly stems from pessimism about one’s personal life outcomes.

When folks expect their lot in life to improve, decade by decade, and when they have an expectation that if they work hard, their children can enjoy a better standard of living than they do, they’re pretty content. So long as that is true, they don’t care that much if other people are wealthier than them (especially people who live far away, but I think this is also true of literal neighbors).

Upsetting the social order is risky and dangerous, and is therefore an action of last resort. Most people don’t actually like violence and will avoid it if they can. If there are opportunities for social advancement, they’ll take them.

However, if those opportunities are lacking, for some reason or another, and people feel like they can’t get ahead, they’ll feel frustrated. They have a sense that something unfair is happening, and are likely to adopt the mindset of the pie fallacy: there’s only so much to go around, and one person’s wealth implies another person’s poverty.

In fact, while the pie-fallacy is a fallacy in the general case, the 0-sum mindset is a pretty accurate summary of one’s situation when there are not opportunities to improve your lot in life. Conditioning on that premise, your wellbeing does actually tradeoff against other people’s resources. If theres some blocker to creating new value, the only thing left is redistribution, either by legal means like taxation, or illegal, like outright revolution.

I think that it is this frustration, of a lack of upward social mobility, much more than inequality specifically, that leads to revolution or attempts at redistribution.

 

Unfortunately, I think that a huge swaths of Americans actually find themselves in this position, of being unable to improve their lot. The American dream, that you can work hard for most of your life, so that your kids can have a better life than you feels like an empty promise.

In the red-tribe side of things, this looks like the factory worker in the rust belt, whose job was outsourced to a foreign country, leaving him without a clear away to support himself. And as the economic centers of his world dried up, he watched as his community became a shadow of its former vibrant self. For him, the American dream seems to have gone wrong somehow. Somehow he’s worse off than his father was

On the blue-tribe side of things, this is embodied by the millennial in one of the costal mega-cities, working as part of the gig economy or in dismal retail, who is saddled with enormous college debt,  which somehow didn’t give her much career capital, and who is paying exorbitante rent to for a small room in a shared apartment, or alternatively still living with her parents. A child of the nineties, she thought, and her parents thought, that if she worked hard and went to a good college, the world was her oyster. But somehow it doesn’t seem to be working out that way.

I imagine that both of these people feel like they are running in place: they were promised that if they worked hard, they could have a good life. But instead they’re treading water.

Why

Why are we here? It seems like for a number of decades standards of living were rising, and people were doing better than their parents. Why did the American dream stop working?

I think that it is the result of a number of factors.

First, we need to keep in mind that “having a better standard of living than your parents did” was much easier for the Baby Boomers, because that was a lower bar to beat. The Boomer were born right after the Great Depression, and many of them were first or second generation immigrants. If your mom and dad walked off the boat with literally $10 in their pocket, and worked their way up from there, or if they were impoverished farmers whose crops failed in the dust bowl, it is a lot more likely that you’ll end up better off then them.

Second is the great stagnation. The boomers benefited from one of the biggest surges in economic growth in history, driven by the mass deployment of new technology, which created new industries. Riding that wave, life got better for a lot of people. If true growth is much slower now, less benefit will accrue to individuals.

But the biggest thing is that the new economy is exclusionary. Information technology and it’s derivatives, the industries in which almost all the growth we do have is concentrated, is fundamentally about scale, which means that there a.

In one of my favorite essays of all time, How to Make Wealth, Paul Graham says, in order to make a lot of wealth, you need to have leverage, in the sense that your decisions scale. And that one of the best ways of getting leverage, is by developing technology.

What is technology? It’s technique. It’s the way we all do things. And when you discover a new way to do things, its value is multiplied by all the people who use it. It is the proverbial fishing rod, rather than the fish. That’s the difference between a startup and a restaurant or a barber shop. You fry eggs or cut hair one customer at a time. Whereas if you solve a technical problem that a lot of people care about, you help everyone who uses your solution. That’s leverage.

Doing work with computers, means that you have 0-marginal cost, which means that if you solve a problem once, you’ve solved it an arbitrarily-large number of times, so you can create a huge amount of value in one go.

But there’s a dark side of leverage, which is that it tends to shift the world towards winner-take all dynamics. A few people who are well suited to this new world (possessing an extreme entrepreneurial mindset, or technical chops, or excellent sales skills, etc.) thrive, producing huge amounts of value. But for the Americans that don’t fit that template, the work-a-day opportunities are becoming less and less attractive, because those roles provide less and less marginal value.

When the marginal costs of goods was non-0, that had a flattening effect on the power law, because even if you couldn’t design a ford car, you provided value to the final product by working on the assembly line. But today, the marginal cost of most everything in the new economy is 0.

Approximately speaking, everyone has been thrust, whether they like it or not, from Taleb’s Mediocristan into Extremistan. More of progress is dominated by a few super-winners, compared to a generation ago, when progress was, though still on a power law, more equitable.

[I feel like this doesn’t quite explain it. Like sure, wages are stagnating. But if costs were falling faster than wages, then in wouldn’t matter: you would be doing better year by year, just by dint of falling cost of living.

So part of the story is that, for some reason, the rents are too damn high, such that people have trouble just getting by.

Why’s that?

Maybe it has something to do with the fact that almost all of the economic opportunities are concentrated in a few mega-cities? You either have to live there, or be resigned to watching the world decay around you?

Does that mean that if zoning were different, so that there were ten times the number of apartments in SF, NYC, Chicago, DC, and Austin, this problem would basically go away, and people could basically rise with the rising tide of the economy? Or if the pandemic causes the world to decentralize, and people can pretty much live and work from anywhere in the world?

I guess I would be surprised if that were the result of either of those changes. Maybe because there are forces that are preying on people and therefore driving net income to some set point above subsistence, but below personal net-growth?

It might be that the high cost of living is a consequence of the huge disparities in production power: there are the heroes / robber barrons of the new economy, who are mostly living in a few mega cities, and almost all the the wealth flows from there, so you have to be in the mega-cities to have access to any of the opportunities?

Also, don’t forget about the college debt which is part of what makes the cost of living high, and which has been driven up by other factors.]

Reaction

Whatever the cause, I now suspect that there are a lot of people that feel like whatever they do, there’s no way for them to get ahead. They feel, justifiably, disenfranchised.

It’s natural, in a situation where you feel trapped, to try and make sense of what is happening to you. There are a number of narratives that people adopt to orient to the situation that they find themselves in. But I think the two biggest ones are Trumpism, and Wokeism.

The archetypical disenfranchised red-triber is frustrated and confused that somehow things are getting worse for him and his peers. The literal person of Donald Trump exploits those feelings, and provides a narrative that America has been taken advantage of by other countries (for instance, Mexico, feeding us illegal immigrants, and China, who is cheating in trade deals), due to something like weakness on the part of the establishment and the costal elites (who in any case, seem to alienate him or condescend to him at every turn). Under this narrative, these trends need to be reversed, to make America Great Again ™.

The archetypical millennial blue-triber, in contrast, feels frustrated and confused about how she can’t seem to get ahead. She buys into a narrative that the reason for this is systemic oppression: the problem is racism, the patriarchy, and capitalism, and broadly systems that serve their own interests by exploiting weaker groups. Under this story, what is needed is an overhaul of those oppressive systems.

Note of course that this sort of disenfranchised frustration isn’t the only thing feeding these two ideologies. Lots of people who, personally, have an optimistic trajectory might buy into or act in alignment with either one, for all the usual reasons, from virtue signaling, to tribal mimesis, to thinking seriously about the problems and their origins and coming to the conclusion that the Trumpist or the Wokists are basically correct.

But, I posit that the energy that is driving both movements comes from large swaths of people who feel like, whatever they do, they can’t get ahead. If there were ample individual economic opportunity, people wouldn’t care nearly as much about those problems.

Social media

On top of this, we introduced a new technology to the masses over the past decade and a half. Our social and political lives are organized by twitter and facebook.

This has a divisive, echo-chamber enforcing, effect. The Liberals and the Conservatives have always been at each others throats, railing against the other as an outright evil. But I imagine that there was something very different about the world when everyone watched Walter Cronkite on the news every night. There were strong, emotional disagreements, but everyone was, to a large extent, living in the same world.

No longer. Now, the media landscape is fractured, and virtually every event is spun to support one ideology or another. And now almost everyone has a voice in the conversation, due to social media, where before, there were defacto gate-keeper institutions.

Much more than we used to I think, we live in one of a number of parallel worlds that are layered on top of each other. My guess is that this aggravates tensions, by reinforcing one’s existing narrative. Then, because of the natural memetic incentives, this results in extreme othering and demonization of anyone with a meaningfully different set of priorities.

[It seems like there’s a lot more here, that it might be worth understanding in detail.]

Also, our institutions are falling apart, in the sense of loosing their legitimacy, and in the sense of losing their ability to function in pretty basic ways, for related reasons. Basically, it seems like in this cultural battlefield, every institution has to pick a side. This lack of space for “neutrality” edges out the possibility of non-ideological sense-making. (I’m not satisfied with that explanation. I feel like there is a much more detailed story of the incentives that have torn apart news papers and the universities, for instance.

Covid

On top of all this, we have a pandemic, itself litteraly-insanely politicized, forced a lot of people into enclosed spaces with minimal social contact, for months, and waited we went collectively stir-crazy.

Powder keg

Overall, this reads like a pretty concerning setup. I don’t have a good intuition for how robust the substructure of our world is, but it certainly seems like it is being tested more than any other time in my extremely short lifetime.

Seems maybe bad?

 

 

Values are extrapolated urges

[Epistemic status: a non-rigorous theory that will turn out to be wrong or incomplete in the light of future philosophical work. But also, this is how I think it works.]

Related to: Value Differences As Differently Crystallized Metaphysical Heuristics

In this post I want to outline my understanding of what “values” are, at least for human beings. This idea or something very like it may already have standard terminology in academic philosophy, in which case, I would appreciate being pointed to the relevant references. This may be obvious, but I want to say it to lay the groundwork for the for a puzzle that I want to talk about in the next post.

Basically, I posit that “values”, in the case of human beings, are crystallization or abstractions of simple response patterns.

[Google doc version for commenting]

Abstracting values from reactions

All animals, have a huge suite of automatic reactions to stimuli, both behavioral and affective, and both learned and hard coded.

When thirsty and near water, a lion will drink. When a rabbit detects a predator, they’ll freeze (and panic?). When in heat (and the opportunity presents itself) a giraffe will  copulate. When his human comes home, a dog will wag its tail and run up in greeting, presumably in a state of excited happiness. [I note that all of my examples are off mammals.]

Some of these behaviors might be pretty complex, but their basic structure is TAP-like: something happens, and there is some response in physiology of the animal. I’m going to call this category of “contextualized behaviors and affects”, “urges”.

Humans understand language, which means that the range of situations that they can respond to is correspondingly vaster than most animals. For instance, a human might be triggered (have a specific kind of fear response) to another human making a statement.

But that isn’t the main thing that differentiates humans in this context. The big difference between humans and most other animals, is that humans can abstract from a multitude of behaviors, to infer and crystallize the “latent intentionality” among those behaviors.

For instance, an early human can reason,

When I see a tiger, I run, and feel extreme overriding panic. If the tiger catches me, I’ll try to fight it. When a heavy rock falls from a cliff, and I hear it falling, I also have a moment of panic, and duck out of the way.

When I am hungry, I eat. When I am thirsty, I drink.

When other people in my tribe have died, I’ve felt sad, and sometimes angry.

…Guess I don’t want to die.

From each of these disparate, contextualized, urges-to-action and affective responses (which by the way, I posit are not two distinctly different thing, but rather two ends of spectrum), a person notices the common thread, “what do each of these behaviors seem to be aiming towards?”

And abstracting that goal, from the urges, he/she then “owns” it. He/she thinks of him/herself as an entity wanting, valuing, caring about that thing (rather than a bundle of TAPs some of which are correlated).

My guess is that this abstraction operation is an application of primate (maybe earlier than primate?) social-modeling software to one’s self. It is too expensive to track all of the individual response behaviors of all of the members of your band, but fortunately, you can compress most of the information by modeling them not at adaption-executors, but as goal directed agents, and keeping track of their goals and their state of knowledge.

When one applies the same trick to one’s own behavior and mental states, one can compress a plethora of detail about a bunch of urges into a compact story, about what you want.

This is the origin of “values.” Values are compressions / abstractions / inferences based on / extrapolated from the a multitude of low level reactions to different situations.

I think that most animals can’t and don’t do this kind of inference. Chipmunks (I think) don’t have values. They have urges. Humans can, additionally, extrapolate their urges into their values.)

I’m pretty sure that something like this process is how people come to their values (in the conventional sense of “the things they prioritize”) in real life.

For instance, I am triggered by claims and structures that I perceive as threats to my autonomy. I flinch away defensively. I think that this has shaped a lot of my personality, and choices, including leading me into prizing rationality.

Furthermore, I posit that something like that process is how people tend to adapt political ideologies. When someone hears about the idea of redistribution, and their visceral sense of that is someone taking things from them, they have a (maybe subtle) aversion / threatened feeling.* This discomfort gives rise to an urge to skepticism of the idea. And if such a person hangs out with a bunch of other people that have similar low-level reactions, eventually, it becomes common knowledge, and this becomes the seed of an ideology, that gets modified and reinforced by all the usual tribal mechanisms.

I think the same basic thing can happen when someone feels (probably less than consciously) threatened by all kinds of ideologies. And this + social mimesis is how people end up with “conservative values” or “liberal values” or “libertarian values” or what have you. (Of course what one’s associations

* – I have some model of how this works, the short version being, “stimuli trigger associated (a lot of the action here is in the association function) mental imagery, which gives rise to a valence,  which guides immediate action, modulo further, more consequential deliberation. In fact, you can learn to consciously catch glimpses of this happening.

Of course all of this is a simplification. Probably this process occurs hierarchically, where we abstract some goals from TAP-like urges, and then extrapolate more abstract goals from those, and so on until we get to the “top” (if it turns out that there is a “top”, as opposed to a cycle that has some tributaries that flow into it).

For that reason, the abstraction / crystallization / triangulation process is not deterministic. It is probably very path dependent. Two people with the exact same base level pattern of urges, in different contexts will probably grow into people with very different crystallized values.

Values influence behavior

Now a person might abstract out their values from their behavior in a way that is largely non-consequential. They model themselves, and describe themselves, in terms of their values, but that is just talk. The vast majority of their engagement in the world is still composed of the behaviors stemming from their urges different situations.

But, it also sometimes happens that abstracting out values, and modeling one’s self as an optimizer (or something like an optimizer) for those values, can substantially effect the level of behavior.

For one thing, having a shorthand description of what one cares about means that one one can use that description for deliberation. Now, when considering what to do in a situation, a person might follow mental process that involves asking how they can achieve some cashed goal, instead of reflexively acting on the basis of the lower level urges that the goal was originally abstracted from.

This means that a person might end up acting in a way that is distinctly in opposition to those low level reactions.

For instance, a person might want status and respect, and they can feel the tug to go drink and socialize “with the guys” of their age group, but they instead stay home and study, because they reason that this will let them get a good job, which will let them get rich, which they equate with having a lot of status.

Or a person might take seriously that they don’t want to die, and sign up for cryonics, even though none of their urges recommended that particular action, and in fact, it flies in the face of their social conformity heuristics.

Furthermore, in this vein a person might notice inconsistencies between their professed values and the way they behave, or between multiple diverse values. And if they are of a logical turn of mind they may attempt to modify their own behavior to be more in line with their values. Thus we end up with moral striving (though moral striving might not be the only version of this dynamic).

Bugs

Just to say this explicitly, humans, uniquely (I think? maybe some other animals also abstract their values), can examine some particular behavior or reaction and consider it to be a bug, a misfiring, where the system is failing to help them achieve their values.

For instance, I’m told that a frog will reflexively flick out it’s tongue to ensnare anything small and black that enters its field of vision. From the perspective of evolution, this is a bug: the behavior is “intended” for catching and eating flies, and eating bits of felt that human researchers throw in the air (or whatever) is not part of the behavior selected for.  [Note: in talking about what evolution “intended”, we’re executing the same mental move of abstracting goals and values from behavior. Evolution is just the fact of what happened to replicate, but we can extend

But, I claim here, that asking “is this behavior a bug, from the frogs perspective?” is a miss-asked question, because the frog has not abstracted its values from its behaviors, in order to reflect back on its behaviors and judge them.

In the parallel case of a human masturbating, the human can abstract its values from his or her behavior, and could deem masturbation as a 1) bug, a dis-endorsed behavior that arises from a hormonal system that is partially implementing his or her values, but which misfires in this instance, or 2)  as an expression of what he/ she actually values, part of a life worth living.

(Now it might or might not be the case that only one of these options is reflexively stable. If only one of them is, for humans in general, there is still a meaningful sense in which one can be mistaken about which things are Good. That is a person can evaluate something as aligned with their values, but would come to think differently in the limit of reflection.)

 

 

 

 

 

A taxonomy of Cruxes

[crossposted to LessWrong]

This is a quick theoretical post.

In this post, I want to outline a few distinctions between different kinds of cruxes. Sometimes folks will find what seems to be a crux, but they feel some confusion, because it seems like it doesn’t fit the pattern that they’re familiar with, or it seems off somehow. Often this is because they’re familiar with one half of a dichotomy, but not the other.

Conjunctive, unitary, and disjunctive cruxes

As the Double Crux method is typically presented, double cruxes are described as single propositions, about which, if you changed your mind, you would change your mind about another belief.

But as people often ask,

“What if, there are two propositions, B and C, and I wouldn’t change my mind about A, if I just changed my mind about B, and I wouldn’t change my mind about A if I just changed my mind about C, and I would only if I change my mind about A, if I shift on both B and C?”

This is totally fine. In this situation would would just say that your crux for A is a conjunctive crux of B and C.

In fact, this is pretty common, because people often have more than one concern in any given situation.

Some examples:

  • Someone is thinking about quitting their job to start a business, but they will only pull the trigger if a) they thought that their new work would actually be more fulfilling for them, and b) they know that their family won’t suffer financial hardship.
  • A person is not interested in signing up for cryonics, but offers that they would if a) it was inexpensive (on the order of $50 a month and b) if the people associated with cryonics were the sort of people that he wanted to be identified with. [These are the stated cruxes of a real person that I had this discussion with.]
  • A person would go vegetarian if, a) they were sure it was healthy for them and b) doing so would actually reduce animal suffering (going a level deeper: how elastic is the supply curve for meat?).

In each of these cases there are multiple considerations, none of which is sufficient to cause one to change one’s mind, but which together represent a crux.

As I said, conjunctive cruxes are common, I will say that sometimes folks are too fast to assert that they would only change their mind if they turned out to be wrong about a large number of conjunctive terms.

When you find yourself in this position of only changing your mind on the basis of a large number of separate pieces, this is a flag that there may be a more unified crux that you’re missing.

In this situation I would back up and offer very “shallow” cruxes. Instead of engaging with all the detail of your model, instead look for a very high level / superficial summary, and check if that is a crux. Following a chain of many shallow cruxes is often easier than trying to get into the details of complicated models right off the bat.

(Alternatively, you might move into something more like consideration factoring.)

As a rule of thumb, the number of parts to a conjunction should be small: 2 is common, three is not that common. Having a 10 part conjunction is implausible. Most people can’t hold that many elements in their head all at once!

I’ve occasionally seen order of 10 part disjunctive arguments / conjunctive cruxes in technical papers, though I think it is correct to be suspicious of them. They’re often of the form “argument one is sufficient, but even if it fails, argument 2, is sufficient, and even that one fails…” But, errors are often correlated, and the arguments are likely not as independent as they may at first appear. It behooves you to identify the deep commonality between your lines of argument, the assumptions that multiple arguments are resting on, because then you can examine it directly. (Related to the “multiple stage fallacy‘).

Now of course, one could in principle have a disjunctive crux, where if they changed their mind about B or about C, they would change their mind about A. But, in that case there’s no need to bundle B and C. I would just say that B is a crux for A and also C is a crux for A.

Causal cruxes vs. evidential cruxes

A causal crux back-traces the causal arrow of your belief structure. They’re found by answering the question “why do I believe [x]?” or “what caused me to think [x] in the first place?” and checking if the answer is a crux.

For instance, someone is intuitively opposed to school uniforms. Introspecting on why they feel that way, they find that they’re expecting (or afraid that) that kind of conformity squashes creativity. They check if that’s a crux for them (“what if actually school uniforms don’t squash creativity?”), and find that it is: they would change their mind about school uniforms if they found that they were wrong about the impact on creativity.”

Causal cruxes trace back to the reason why you believe the proposition.

In contrast, an evidential crux is a proxy for your belief. You might find evidential cruxes by asking a question like “what could I observe, or find out, that would make me change my mind?”

For instance, (this one is from a real double crux conversation that happened at a training session I ran), two participants were disagreeing about whether advertising destroys value on net. Operationalizing, one of them stated that he’d change his mind if they realized that beer commercials, in particular, didn’t destroy value.

It wasn’t as if he believed that advertising is harmful because beer commercials destroy value. Rather it was that he thought that advertising for beer was a particularly strong example of the general trend that advertising is harmful. So if he changed his mind in that instance, where he was most confident, he expected that he would be compelled in the general case.

In this case “beer commercials” are serving as a proxy for “advertising.” If the proxy is well chosen, this can totally serve as a double crux. (It is, of course, possible that one will be convinced that they were mistaken about the proxy, in a way that doesn’t generalize to the underlying trend. But I don’t think that this is significantly more common than following a chain of cruxes down, resolving at the bottom, and then finding that the crux that you named was actually incomplete. In both cases, you move up as far as needed, adjust the crux (probably by adding a conjunctive term), and then traversing a new chain.)

Now, logically, these two kinds of cruxes both have the structure “If not B, then not A” (“if uniforms don’t squash creativity, then I wouldn’t be opposed to them anymore.” and “if I found that beer commercials in fact do create value, then I would think that advertising doesn’t destroy value on net”). In that sense they are equivalent.

But psychologically, causal cruxes traverse deeper into one’s belief structure, teasing out why one believes something, and evidential cruxes traverse outward, teasing out testable consequences  or implications of the belief.

Monodirectional vs. Bidirectional cruxes

Say that you are the owner of a small business. You and your team are considering undertaking a major new project. One of your employees speaks up and says “we can’t do this project. The only way to execute on it would bankrupt the company.”

Presumably, this would be a crux for you. If you knew that the project under consideration would definitely bankrupt the company, you would definitively think that you shouldn’t pursue that project.”

However, it also isn’t a crux, in this sense: if you found out that that claim was incorrect, that actually you could execute on the project without bankrupting your company, you would not, on that basis alone, definitively decide to pursue the project.

This is an example of a monodirectional crux. If the project bankrupts the company, then you definitely won’t do it. But if it doesn’t bankrupt the company then you’re merely uncertain. This consideration dominates all the other considerations, it is sufficient to determine the decision, when it is pointing in one direction, but it doesn’t necessarily dominate when it points in the other direction.

(Oftentimes, double cruxes are composed of two opposite bidirectional cruxes. This can work totally fine. It isn’t necessary that for each participant, the whole question turns on the double crux, so long as for each participant, flipping their view on the crux (from their current view) would also cause them to change their mind about the proposition in question.)

In contrast, we can occasionally identify a bidirectional crux.

For instance, if a person thinks that public policy ought to optimize for Quality Adjusted Life Years, and they’ll support whichever health care scheme does that, then “maximizing QALYs” is a bidirectional crux. That single piece of information (which plan maximizes QALYs), completely determines their choice.

“A single issue voter” is a person voting on the basis of a bidirectional crux.

In all of these cases you’re elevating one of the considerations over and above all of the others.

Pseudo cruxes

[This section is quite esoteric, and is of little practical relevance, except for elucidating a confusion that folks sometimes encounter.]

Because of the nature of mono-directional cruxs, people will sometimes find pseudo-cruxes, propositions that seem like cruxes, but are nevertheless irrelevant to the conversation.

To give a (silly) example, let’s go back to the canonical disagreement about school uniforms. And let’s consider the proposition “school uniforms eat people.”

Take person who is in favor of school uniforms. The proposition that “school uniforms eat people” is almost certainly a crux for them. The vast majority of people who support school uniforms would change their mind if they were convinced that school uniforms were carnivorous.

(Remember, in the context of a Double Crux conversation, you should be checking for cruxy-ness independently of your assessment of how likely the proposition is. The absurdity heuristic is insidious, and many claims that turn out to be correct, seem utterly ridiculous at first pass, lacking a lot of detailed framing and background.)

This is a simple crux. If the uniform preferring person found out that uniforms eat people, they would come to disprefer uniforms.

Additionally, this is probably a crux for folks who oppose school uniforms as well, in one pretty specific sense: were all of their other arguments to fall away, knowing “that school uniforms eat people” would still be sufficient reason for them to oppose school uniforms. Note that doesn’t mean that they do think that school uniforms eat people, nor does it mean that finding out that school uniforms don’t eat people (duh) would cause them to change their mind, and think that school uniforms are good. We might call this an over-determining hypothetical crux. It’s a bidirectional crux that points exclusively in the direction that a person already believes, and which furthermore, the person currently assumes to be false.

A person might say,

I already think that school uniforms are a bad idea, but if I found out they eat people, that would be further reason for me to reject them. Furthermore, now that we’re discussing the possibility, that “school uniforms don’t eat people” is such an important consideration such that it would have to be a component of any conjunctive crux that would cause me to change my mind and think that school uniforms are a good idea. But I don’t actually think that school uniforms eat people, so it isn’t a relevant part of that hypothetical conjunction.

This is a complicated series of claims. Essentially, this person is saying that in a hypothetical world where they thought differently than they currently do, this consideration, if it held up would be a crux for them (that would bring them to the position that they actually hold, in reality).

Occasionally (on the order of once out of 100?), a novice participant will find their way to a pseudo crux like that one, and find themselves confused. They can tell that the proposition “school uniforms eat people” if true, matters for them. It would be relevant for their belief. But it doesn’t actually help them push the disagreement forward, because, at best, it pushes further in the direction of what they already think.

(And secondarily, it isn’t really an opening for helping their partner change their mind, because the uniform-dispreferring person, doesn’t actually think  that school uniforms eat people, and so would only try to argue that they do if they had abandoned any pretense of truth-seeking in favor of trying to convince someone using whatever arguments will persuade, regardless of their validity.)

So this seems like a crux, but it can’t do work in the Double Crux process.

There is another kind of pseudo crux stemming from bidirectional cruxes. This is when a proposition is not a crux, but it’s inverse would be.

In our school uniform example, suppose that that in a conversation, someone boldly, and apropos of nothing,  asserted “but school uniforms don’t eat people.” Uniforms not eating people is a monodirectional crux that dominates all the other considerations, but school uniforms not eating people is so passé, that it is unlikely to be a crux for anyone (unless the reason they were opposed to school uniforms was kids getting eaten). Nevertheless, there is something about it that seems (correctly) cruxy. It is the ineffectual side of a monodirectional crux. It isn’t a crux, but its inverse is. We might call this a crux shadow or something.

Thus, there is a four-fold pattern of monodirectional cruxes, where one quadrant is a useful progress bearing crux, and the other three contain different flavors of pseudo cruxes.

Proposition: “If school uniforms eat people, then I would oppose school uniforms”

Suppose school uniforms eat people Suppose school uniforms don’t eat people
I am opposed to school uniforms Overdetermining hypothetical crux:  “I would oppose school uniforms anyway, but this would be a crux for me, if (hypothetically) I was in favor of school uniforms.” Non-crux / Crux shadow: “Merely not eating people is not sufficient to change my mind. Not a crux.”
I am in favor of school uniforms Relevant (real) monodirectional crux: “If school uniforms actually eat people, that would cause me to change my mind.” Non-crux / Crux shadow: “While finding out that uniforms do eat people would sway me, that they don’t eat people isn’t a crux for me.”

And in the general case,

Proposition: X is sufficient for A, but Not X is not sufficient for B

X is true X is false
I believe A Overdetermining hypothetical crux Non-crux / Crux shadow
I believe B Relevant monodirectional crux Non-crux / Crux shadow

Note that the basic double crux pattern avoids accidentally landing on pseudo cruxes.

.

 

A conversational pattern I observe

[This is super rough]

[I’m pretty sure that there is prior work on this question that I don’t know about yet.]

Here’s a conversational pattern that I’ve noticed recently. I observed this specifically in conversations with my parents, but on further reflection, I do this too, and postulate that pretty much everyone does.

Basically, one person will say something, either sharing an opinion, or (maybe this is an atypical rationalist version?) sharing a fact, or something in between.

Then the other person will share an opinion or a fact that is at least somewhat related.

Basically the people go back and forth, sharing what they know about / what they think / what they’re interested in.

For instance, a few weeks ago, someone was telling me about how it wasn’t accidental that Hitler lost the war, which I agreed with, and I shared some anecdotes from my recent reading of the Rise and Fall of the Third Reich, particularly junctures where the NAZIs could have won the war if things had gone slightly differently (like if the Luftwaffe had continued bombing the areas in which the RAF command bunkers, from which they were using RADAR and radio to coordinate their planes, during the battle of Britain).

This may not be a great example, since I might have been making a point about historical contingency, and hindsight bias. I’m very confident that this pattern expressed itself in my most recent conversation with a friend to “catch up” though.

She would say something about something, and I would say something vaguely related.

(Ok. In both these cases, I had phenomenology of “desire to impress” and “pressure to show sophistication.” This is less evident in my conversations with my parents, but it isn’t implausible that there was a less vivid current of the same. That maybe the driving force of this weird thing.)

In any case, this conversational pattern is common, but it is particularly evident when talking with my parents, I’ll often share something, and (misunderstanding what I’m saying) will say something else that is only barely or vaguely related. (I think this might be offered as if it what is being said is in agreement with me.) This particular situation seems to highlight how much conversation is just an opportunity to share things that we want to talk about with someone else, who is likewise taking the opportunity to talk about things that they want to talk about.

What’s going on here? Why do we like doing this?

Some thoughts:

  • Maybe we just like the feeling of being listened to, because it validates our opinions and perspectives. This is pretty interesting, given that it’s fake: the other person isn’t actually engaging with our opinions except to riff on them. It’s as if we’re masturbating each-other.
  • Maybe this is about intelligence or sophistication signalling? We get to show off our thoughts?
  • Maybe it’s a kind of bonding / tribe affirming thing: we pseudo agree with each-other’s propositions (without engaging with them critically)
  • We just like talking about what we’re interested in for some reason, and the real question is why we like that.

Some musings on historical contingency, randomness, and my own desire to be an optimizer

It just hit me how the fact that this pandemic probably escaped from a lab in China means that this event, with global and personal consequences, is so contingent. There are other Everett branches where it didn’t happen, and I am going about my life as I would have.

I have a weight in my stomach about this. Especially in so far as the quarantine is good for me, and I am learning things or building skills / meta-skills that will have a permanent impact on my life, it is unsettling to think that they depend on fate. Am I going to be consistently less effective in those other worlds, because this “lucky” break didn’t happen? (Interestingly, that is the horrifying thing to me, not the thought of all the luck that I’ve missed out on that went to other worlds.) My growth and power is so fragile.

Of course there have been other events like this: WWII was contingent on Hitler’s birth, and that massively shaped the world I live in now. Most earths don’t look like this one for that reason. There’s still an intact European Jewry, the balance of power looks very different, and science may not have been institutionalized and may not be state-funded. How were Atomic bombs discovered in those world? How did the computer revolution happen?

And of course my personal life is incredibly contingent. If I hadn’t come across Grimoire for the Apprentice Wizard, would I be on a path of ambition? If I hadn’t gone to U Chicago, and met Stefan, would I have have even encountered LessWrong. Reading the sequences, at that time in my life, radically, radically transformed my trajectory.

But this is a huge contingent event occurring after I came into myself as a conscious entity, and after I had “gotten my footing” in the world. This pandemic has changed, and is apt to change a lot for me, but only by luck.

I think that this one stings in particular (at least in part) because, in principle I could have taken time and space to do the things that I’m doing now. But I predict that I wouldn’t have, unless forced. I can feel how that’s a bias. How across the multiverse, I’m less strong than I might otherwise be, because I’m not resisting that temptation. And here, I needed random chance to save me, instead of bootstrapping by my own power.

This is a reminder to me that I want to be an optimizer: I want to be the sort of thing that, whatever sort of universe it finds itself in, convergently steers that world towards the same state. Obviously, the worlds don’t actually converge, because of how much is determined by the randomness, and because I am small. But I want it to be the case that my situation defines the shape of the problems I have to solve, and the tools that I have to work with (and build myself from), but that my situation never sways my basic ability to steer.

I want to be the sort of thing that systematically moves in right direction, no matter where in the multiverse I find myself.

Holding both the positive aspects of a problem and reality of there being a problem

[Probably obvious to many (most?) of my readers, but the obvious often bears saying aloud.]

This morning, as part of my rest day, I did the first exercise of Claudia Altucher’s book, Become an Idea Machine, which is mostly a series of prompts for generating 10 (or more) new ideas, one prompt a day.

The first prompt is to take 10 complaints that you have, and for each one, turn it into a positive expression of gratitude. She gives the example of hating being stuck in traffic every day, and generating from that, “I am grateful that I live in a city that has so much traffic because that means there [are] plenty of opportunities here, and I can meet lots of interesting people.” You’re supposed to do this ten times, on your own problems.

I think that this is a great exercise. I imagine that I tend to be too rigid in my assessment of circumstances, and there are in fact a lot of good things resulting from a “problem”, which I fail to apprehend, because I’m stuck in my existing mindset. Done well, I think this mental motion shouldn’t feel like a self-trick, whereby you convince yourself that “the grapes were probably sour anyway.” Instead it feels like uncovering actual value, henceforth unnoticed, all around you.

As an example, my fourth most pressing problem is the lack of satisfying romantic partnership. And it is natural to focus on the pain or longing of that situation. But also, not having a partner means that I have a lot of solitary time, to think and work and do what I want, which I really value. By default, I think I wouldn’t notice how much I value that alone-time unless I did have a partner, at which point I might start to feel stifled. Hedonically, it seems like a bug if I only pay attention to the aspects of a situation that I dislike.

So I think appreciating what’s good about things that we might reflexively label as bad.

However, I think that it is crucial, if we want to be balanced and sane, that we not lose track of the problem being bad in this process.

It is right and good for the paraplegic to savor the way his disability forces him to slow down and be reflective, and “stop and smell the roses.” But that doesn’t mean that he can’t assess the situation, the pros and cons, and determine that on net, it would be better to be able to walk (and climb and run).

Likewise, it is extremely appropriate to look around at the way the recent pandemic has brought communities together, and the way that it has brought out heroism in many, and feel pleased and proud. But that doesn’t mean that we have to say that the pandemic is good, or that we shouldn’t wish that it hadn’t happened.

The claim “there is often unrecognized good in what seems to be bad” does not entail that “everything that seems bad is actually good”, (or the related “everything happens for a reason”). When we conflate the first with the second(s) them we’re  setting ourselves up for a kind of Stockholm syndrome of the present, a massive status quo bias.

There is more good in the world than we often appreciate by default. And the world is complex, and it is often hard to tell what the ultimate consequence of something will be. But that does not mean that we abdicate our sacred right to weigh up the pros and cons,  make the best assessment we can of whether a thing is good or bad, and then try to steer the world towards the good. When we do otherwise the consequences can be literally catastrophic.

(Of course this is not to say that everything that we reflexively consider bad is in fact bad on net. We might take something that we hate, consider all the benefits that it gives us and all the costs, and conclude that actually, our initial impression was wrong, and we on reflection, prefer the world with the supposed-to-be-bad thing. Just as we might reconsider our reflexive attitude to something that is supposed to be good, and conclude that on-net, we dis prefer it. [For instance, Brienne’s comment in this thread has had me reevaluating my attitude towards romantic love, lately.])

 

 

Engineering vs. Valence

Despite often being maligned, the contemporary “spiritual” worldview has some things going for it. I emphasizes love, compassion, and gratitude, which are probably close to the most important thing to focus on for individual and community eudaimonia. (Though emphatically not for safely or sanely steering civilization.). New-agey types, despite their woo, do have a number of very effective tools, frameworks, and insights, like NVC, Focusing, and circling. The metaphysics is very confused, but metaphysics is really hard to get right (and if anything like panpsychism is true (which my inside view currently puts a lot more mass on than most of my peers), then they will turn out to have been not that far of the mark).

But, I suggest that the thing that is most wrong in the “spiritual” worldview is the general propensity to posit “things with intrinsic valence”. In my caricature of the spiritual worldview (which I more-or-less believed  from the ages of 15 to 20), many things (actions, energies, materials) are seen as fundamentally “good” and others as fundamentally “bad”, and that “good” things have good effects and “bad” things have bad effects.

  • For instance, according to David Hawkins, meditation, prayer, optimistic thoughts, and positive emotions,  “raise your vibration“, attract good things into your life, and provokes peace locally, and across the world. (Negative thoughts, negative emotions, artificial supplements, etc.  lower your vibration.)
  • Or, orgonite, which is a substance which is supposed to transmute “negative energy” into “positive energy.” Cellphone towers supposedly emit “negative orgon energy” (which “promote drought, negativity, fear, and so on”),
  • “Toxins” is a sort of generic term for “bad things that are in your body”. You want to get rid of them.
  • The phrases “good karma” and “bad karma” speak for themselves.

The list goes on and on. All of these invoke a sense of a thing that is good and safe vs. a thing that is bad and harmful. Obviously, you want more of the good, and less of the bad.

The problem is that almost nothing in the world seems to work like this. Things aren’t intrinsically bad or good. They just have effects. And whether those effects are good or bad depend on mechanistic details of the systems involved.

For example there is no substance that is fundamentally “healthy” or “unhealthy” for humans. The effects depend on the dosage, and location.

Air is obviously good for people, but an air bubble in the the blood stream can be fatal. Potassium is critical to the functioning of our neurons, but too much potassium disrupts that functioning, so we use potassium solutions as a lethal injection . Water is prototypically “good for you”, but drinking too much causes health problems and, of course, having water in your lungs is a good way to drown.

These substances aren’t fundamentally “bad.” The harm comes from too much of them in the wrong place.

I claim that (almost) everything works like this.

Facebook, or the internet, isn’t good or evil, it just has effects, some of which are positive and some of which are negative.

Smoking  will kill you, but its stimulant effects can boost your effective intelligence very slightly.

Even compassion and Christly forgiveness, are harmful in some systems.

The world is a bunch of complicated systems, and in order to get good effects, we don’t just mash the “good” button. Rather, we mostly need to figure out which precises elements go in exactly the right places.