Kaj_Sotala

Ap­ply now: Get “un­stuck” with the New IFS Self-Care Fel­low­ship Program

• Kaj_Sotala 12 Jul 2024 20:27 UTC

  3 points

  0 ∶ 0

  in reply to: AsherS’s comment on: Aban­don­ing func­tion­al­ism: Some in­tu­ition pumps

  Right, most functionalist theories are currently not physically constrained. But if it turns out that consciousness requires causal properties implemented by EM fields, then the functions used by the theory would become ones defined in part by the math of EM fields. Which could then turn out to be physically constrained in practice, if the relevant causal properties could only be implemented by EM fields. (Though it might still allow a sufficiently good computer simulation of a brain with EM fields to be conscious.)

  But “there are multiple ways to physically realize any (Turing-level) computation, and multiple ways to interpret a physical realization as computation, and no privileged way to choose between them”

  This argument feels somewhat unconvincing to me. Of course, there are situations where you can validly interpret a physical realization as multiple different computations. But I tend to agree with e.g. Scott Aaronson’s argument (p. 22-25) that if you e.g. want to interpret a waterfall as computing a good chess move, then you probably need to include in your interpretation a component that calculates the chess move and which could do so even without making use of the waterfall. And then you get an objective way of checking whether the waterfall actually implements the chess algorithm:

  … it seems overwhelmingly likely that any reduction algorithm would just solve the chess problem itself, without using the waterfall in an essential way at all! A bit more precisely, I conjecture that, given any chess-playing algorithm A that accesses a “waterfall oracle” W, there is an equally-good chess-playing algorithm A0 , with similar time and space requirements, that does not access W. If this conjecture holds, then it gives us a perfectly observer-independent way to formalize our intuition that the “semantics” of waterfalls have nothing to do with chess. [...]

  Interestingly, the issue of “trivial” or “degenerate” reductions also arises within complexity theory, so it might be instructive to see how it is handled there. [...] Suppose we want to claim, for example, that a computation that plays chess is “equivalent” to some other computation that simulates a waterfall. Then our claim is only non-vacuous if it’s possible to exhibit the equivalence (i.e., give the reductions) within a model of computation that isn’t itself powerful enough to solve the chess or waterfall problems.

  Likewise, I think that if a system is claimed to implement all of the functions of consciousness (whatever exactly they are) and produce the same behavior as that of a real conscious human… then I think that there’s some real sense of “actually computing the behavior and conscious thoughts of this human” that you cannot replicate unless you actually run that specific computation. (I also agree with @MichaelStJules ’s comment that the various functions performed inside an actual human seem generally much less open to interpretation than the kinds of toy functions mentioned in this post.)

• Kaj_Sotala 11 Jul 2024 20:17 UTC

  6 points

  3 ∶ 0

  on: Aban­don­ing func­tion­al­ism: Some in­tu­ition pumps

  Interesting post!

  One thing that confused me was that there were sections where it felt like you suggested that X is an argument against functionalism, where my own inclination would be to just somewhat reinterpret functionalism in light of X. For example:

  I’ve yet to see a satisfactory functionalist account of how binding can happen (and I’ve come to believe that it’s not even possible in principle). At the same time, possible solutions positing that binding happens, for example, at the level of the electromagnetic fields produced by neurons strike me as elegant, parsimonious, and rigorous.

  If it turns out that the information in separate neurons is somehow bound together by electromagnetic fields (I’ll admit that I didn’t read the papers you linked, so don’t understand what exactly this means), then why couldn’t we have a functionalist theory that included electromagnetic fields as its own communication channel? If we currently think that neurons communicate mostly by electric and chemical messages, then it doesn’t seem like a huge issue to revise that theory to say that the causal properties involved are achieved in part electromagnetically.

• Kaj_Sotala 1 Jul 2024 22:48 UTC

  3 points

  0 ∶ 0

  in reply to: Linch’s comment on: Re­view of Past Grants: The $100,000 Grant for a Video Game?

  I think that some of the bits in that essay were too strong, in particular this line

  All, or at least most, of education could be done via games that were as addictive and enjoyable as traditional games.

  was probably wrong, for reasons Andy Matuschak outlines:

  Games are designed first and foremost to be fun—or beautiful, or engrossing, or exhilarating. Games are an aesthetic medium, and (generally speaking) they compel our participation insofar as they compel us aesthetically. It’s true that in some games, players end up developing certain skills or understandings along the way. But that doesn’t mean we can make a great game that teaches anything. You’re seeing the survivors. These games’ designers tried and discarded dozens of gameplay ideas in search of something aesthetically compelling. Then, only after they’d satisfied the primary constraint of making something fun, or beautiful, or whatever, the designers figured out how to ensure people would learn what they need as they play. Most mechanisms are not fun. Good games come from a demanding selection process which works the other way around: first, find the fun. There’s no reason at all to believe that for any arbitrary abstract topic, one can always “find the fun” which implicitly teaches it.

  On the other hand, in principle it still seems to me like you should be able to make games that significantly improve on current education. Even if an edugame wasn’t as fun as a pure entertainment game, it could still be more fun than school. And people still watch documentaries because they value learning, even though documentaries can’t compete with most movies and TV shows on pure entertainment value.

  But then again, for some reason DragonBox seems to have been an exception rather than the rule. Even the company that made it mostly just made games for teaching simpler concepts to younger kids afterward, rather than moving on to teaching more complicated concepts. The fact that I haven’t really heard of even reasonably-decent edugames coming out in the 11 years since that post seems like strong empirical evidence against its thesis, though I don’t really understand the reason for that.

• Kaj_Sotala 7 Jan 2024 9:01 UTC

  8 points

  0 ∶ 0

  in reply to: Mitchell Laughlin🔸’s comment on: Effec­tive al­tru­ism as the most ex­cit­ing cause in the world

  Thanks for noticing that! These days I would always use “their” in this kind of context, but I guess I didn’t yet have that habit back in 2014. Edited.

Most peo­ple should prob­a­bly feel safe most of the time

• Kaj_Sotala 25 Apr 2023 17:03 UTC

  10 points

  0 ∶ 0

  on: EA might sys­tem­at­i­cally gen­er­ate a scarcity mind­set that pro­duces low-in­tegrity actors

  (Upvoted.)

  Some of the attempts within EA to solve this seem to be to push even more towards just being a professional network. I think that’s dangerously wrong, because it doesn’t remove the informal networks and their power. It just makes access to them harder, and people more desperate to get in.

  Somewhat relevant counterpoint:

  For everyone to have the opportunity to be involved in a given group and to participate in its activities the structure must be explicit, not implicit. The rules of decision-making must be open and available to everyone, and this can happen only if they are formalized. This is not to say that formalization of a structure of a group will destroy the informal structure. It usually doesn’t. But it does hinder the informal structure from having predominant control and make available some means of attacking it if the people involved are not at least responsible to the needs of the group at large. [...]

  … an elite refers to a small group of people who have power over a larger group of which they are part, usually without direct responsibility to that larger group, and often without their knowledge or consent. [...] Elites are nothing more, and nothing less, than groups of friends who also happen to participate in the same political activities. They would probably maintain their friendship whether or not they were involved in political activities; they would probably be involved in political activities whether or not they maintained their friendships. It is the coincidence of these two phenomena which creates elites in any group and makes them so difficult to break.

  These friendship groups function as networks of communication outside any regular channels for such communication that may have been set up by a group. If no channels are set up, they function as the only networks of communication. [...]

  Some groups, depending on their size, may have more than one such informal communications network. [...] In a Structured group, two or more such friendship networks usually compete with each other for formal power. This is often the healthiest situation, as the other members are in a position to arbitrate between the two competitors for power and thus to make demands on those to whom they give their temporary allegiance.

  What links here?

  • Severin's comment on EA might sys­tem­at­i­cally gen­er­ate a scarcity mind­set that pro­duces low-in­tegrity actors by Severin (25 Apr 2023 22:17 UTC; 1 point)

Here, have a calm­ness video

• Kaj_Sotala 23 Nov 2022 20:29 UTC

  3 points

  0 ∶ 0

  in reply to: WilliamKiely’s comment on: Meta AI an­nounces Cicero: Hu­man-Level Di­plo­macy play (with di­alogue)

  gwern on /​r/​machinelearning:

  There’s no comparison to prior full-press Diplomacy agents, but if I’m reading the prior-work cites right, this is because basically none of them work—not only do they not beat humans, they apparently don’t even always improve over themselves playing the game as if it was no-press Diplomacy (ie not using dialogue at all). That gives an idea how big a jump this is for full-press Diplomacy.

• Kaj_Sotala 22 Nov 2022 19:15 UTC

  6 points

  0 ∶ 0

  in reply to: WilliamKiely’s comment on: Meta AI an­nounces Cicero: Hu­man-Level Di­plo­macy play (with di­alogue)

  There’s a commentated video by someone who plays as the only human in an otherwise all-Cicero game, which at least makes it seem like the dialogue is doing a lot.

• Kaj_Sotala 22 Nov 2022 18:22 UTC

  8 points

  1 ∶ 0

  on: Meta AI an­nounces Cicero: Hu­man-Level Di­plo­macy play (with di­alogue)

  Worth noting that this was “Blitz” Diplomacy with only five-minute negotiation rounds. Still very impressive though.

• Kaj_Sotala 20 Oct 2022 16:58 UTC

  5 points

  2 ∶ 1

  on: The her­i­ta­bil­ity of hu­man val­ues: A be­hav­ior ge­netic cri­tique of Shard Theory

  Some behavioral traits such as general intelligence show very high heritability – over 0.70 – in adults, which is about as heritable as human height.

  I’m very confused about what numbers such as this mean in practice, since the most natural interpretation (“70% of the trait is genetically determined”) is wrong, but there aren’t very many clear explanations of what the correct interpretation is. When I tried asking this on LW, the top-voted answer was that it’s a number that’s mostly useful if you’re doing animal breeding, but probably not useful for much else.

  You mention a lot of heritability numbers, could you clarify what it is that we’re intended to infer from them? (It seems to me that the main thing we can infer from heritability numbers is that if a trait has heritability above zero, then there’s some genetic influence on it, but since you mention some traits having “very high” heritability, I presume that you find there to be some other information too.)

• Kaj_Sotala 19 Sep 2022 7:15 UTC

  7 points

  1 ∶ 0

  in reply to: That's Confidential’s comment on: Many ther­apy schools work with in­ner mul­ti­plic­ity (not just IFS)

  I’m not sure if there is any reason that should be strongly persuasive to a disinterested third party, at the moment. I think the current evidence is more on the level of “anecdotally, it seems like a lot of rationalists and EAs get something out of things like IFS”.

  But given that one can try out a few sessions of a therapy and see whether you seem to be getting anything out of it, that seems to be okay? Anecdotal evidence isn’t enough to strongly show that one should definitely do a particular kind of therapy. But it can be enough to elevate a therapy to the level of things that might be worth giving a shot to see if anything comes out of it.

• Kaj_Sotala 18 Sep 2022 20:18 UTC

  8 points

  2 ∶ 1

  in reply to: That's Confidential’s comment on: Many ther­apy schools work with in­ner mul­ti­plic­ity (not just IFS)

  If they are better, why haven’t they been more widely adopted by mainstream medicine?

  Part of it is that the effectiveness of therapy is often hard and slow to study, so it’s hard to get unambiguous evidence of one being better than another. E.g. many therapists, even if working within a particular school of therapy, will apply it in an idiosyncratic style that draws upon their entire life/​job experience and also knowledge of any other therapy styles they might have. That makes it hard to verify the extent to which all the therapists in the study really are doing the same thing.

  My impression is that CBT got popular in part because it can be administered in a relatively standardized form, making it easy to study. But that means that it’s not necessarily any better than the other therapies, it’s just easier to get legible-seeming evidence on.

  Another issue is that therapy can often take a long time, and lots of the quantitative measures (e.g. depression questionnaires, measures of well-being) used for measuring its effects are relatively crude and imprecise and it can be hard to know exactly what one should even be measuring.

  So overall it can just be relatively hard to compare the effectiveness of therapies, and particular ones get more popular more slowly, e.g. by therapist A hearing that therapist B has been doing a new thing that seems to get better results that A does and then getting curious about it. Or by word of mouth through therapy clients when a new approach is found that seems to work better than conventional therapies, as has been happening with IFS.

• Kaj_Sotala 12 Jul 2022 20:34 UTC

  2 points

  0 ∶ 0

  in reply to: CounterBlunder’s comment on: Wheel­ing and deal­ing: An in­ter­nal bar­gain­ing ap­proach to moral uncertainty

  Oh wow, that is a really great paper! Thank you very much for linking it.

• Kaj_Sotala 9 Jul 2022 21:51 UTC

  8 points

  0 ∶ 0

  in reply to: howdoyousay?’s comment on: Doom Circles

  why anyone would choose one big ritual like ‘Doom circles’ instead of just purposefully inculcating a culture of opennes to giving /​ receiving critique that is supportive and can help others?

  These don’t sound mutually exclusive to me; you can have a formal ritual about something and also practice doing some of the related skills on a more regular basis.

  That said, for many people, it would be emotionally challenging if they needed to be ready to receive criticism all the time. A doom circle is something where you can receive feedback at such a time when you have emotionally prepared for it, and then go back to receiving less criticism normally.

  It might be better if everyone was capable of always receiving it, but it’s also good to have options for people who aren’t quite there yet. A doom circle is a thing that can be done in less than an hour, whereas skill-building is much more of a long-term project.

  low-context feedback is often not helpful;

  That’s true, but also: since people generally know that low-context feedback can be unhelpful, they might hold back offering any such feedback, even when it would be useful! Having an explicit context for offering the kind of critical feedback that you know might be incorrect gives people the chance to receive even the kinds of impressions that they otherwise wouldn’t have the chance to hear.

  feedback is ultimately just an opinion; you should be able to take and also discard it.

  Yes, in any well-run doom circle, this exact thing would be emphasized in the beginning (levin mentioned this in their comment and it was probably done in the beginning of the circles I’ve participated in, as well; I don’t remember what the exact preliminaries were, but it certainly matches my general sense of the spirit of the circle).

• Kaj_Sotala 9 Jul 2022 21:17 UTC

  2 points

  0 ∶ 0

  in reply to: Amy Labenz’s comment on: Doom Circles

  That’s correct, most of the people in the circle (including the person with the wizard line, I think) I’d only met a couple of days before.

• Kaj_Sotala 8 Jul 2022 22:47 UTC

  3 points

  2 ∶ 0

  in reply to: Amy Labenz’s comment on: Doom Circles

  That’s great, thank you :D

• Kaj_Sotala 8 Jul 2022 21:33 UTC

  2 points

  0 ∶ 0

  in reply to: Amy Labenz’s comment on: Doom Circles

  Yes please :D

• Kaj_Sotala 8 Jul 2022 21:28 UTC

  4 points

  0 ∶ 0

  in reply to: Jakub Stencel’s comment on: Doom Circles

  It’s my impression that in writing workshops where people bring their writing to be criticized, it’s also a common rule that the writers are not allowed to respond to the feedback. I believe the rule exists exactly because of what you say: because another person’s feedback may be off or biased for a variety of reasons. If there was a discussion about it, the recipient of the feedback might get defensive and want to explain why the feedback was flawed. That would risk the conversation taking an unpleasant tone and also any correct feedback not being properly heard.

  When the rule is instead “you are required to listen to the feedback, but it’s then totally up to you what you do with it and can choose to ignore it as totally deluded if you wish”, that gives people the full license to do so. Unfair-feeling criticism won’t put them in a position where they have to choose between defending themselves and losing face—not defending yourself won’t make anyone lose face since nobody is allowed to defend themselves. That helps make it easier to consider whether some of the unfair-seeming criticism might actually have a point. And on the other hand, if the criticism really was unfair, then someone defending themselves might give the other people the temptation to argue against the defense in order for them to save face and justify their criticism as correct… and someone trying to argue to you that no really, you really are doomed (or “your writing really is bad”), would totally break any atmosphere of mutual kindness and support.