There is a part near the beginning where he derives the diatonic scale from first principles. (There is a sequel as well, and a book if you prefer to read it in book form)
But note that Sethares's work assumes the Plomp and Levelt curve is correct, which is only approximately true. In practice there is also "higher order beating", which causes additional dissonance at ratios close to small integer ratios. This means the most accurate model of dissonance combines Sethares's approach with ratio complexity as in just intonation. This is backed by listening tests:
Note also that there's a strong cultural/training component to dissonance perception. To minimize this you should ignore the preferences of musicians and only ask untrained listeners.
I don't think 100% accuracy is logically possible. Because it's entirely possible that someone would just naturally write the exact same thing as an LLM would write. And after the fact there is no way to distinguish the two. But pangram does have an extremely low false positive rate, which I think does make it useful for detecting cheating students. Assuming the base rate of cheating students is 1%, and assuming pangram has a false positive rate of 1 in 10,000 and a true positive rate of 7,000 in 10,000, that means ~98% of students flagged by pangram actually cheated. Combined with a teacher's familiarity with that student's previous work, which should rule out many more false positives, it should be a very useful tool.
> ~98% of students flagged by pangram actually cheated
That 2% is a large number! Of people who will have their integrity impugned for no good reason! That's not okay!
Your calculations also are mixing assignments and students. The rate of false positives of 1/10k is of corpuses, not students. 10k students might each submit 2-3 written assignments per week. Obviously, this greatly increases the impact of the false positive rate.
And all of these numbers are dependent on lab conditions for usage, which are not the case in the real world.
> I don't think 100% accuracy is logically possible.
Yes. Which is why marketing this product as it currently is, is a deeply irresponsible endeavor.
Origami design will be my personal test bed for the coming years.
It's objectively very difficult and technical, it's spatiovisual, it's artistic, learning resources for it are sparse and most just learn by the FAFO method, current AI sucks terribly at it, and it's not likely to ever be specifically targeted by benchmaxxers.
Scientifically useful physics simulations. Every model absolutely sucks at them.
Or, as someone else points out in another thread here, academic writing. It's one of the things newer models seem to have actually gotten worse at. Even when you give them detailed instructions on how to write and what to avoid, the "load-bearing", "A but not B" and journal-like writing make it in anyway, with the supposed AGI having no ability to reflect on how blatantly unacademic (and often unreadable) its writing is.
> In my view, intelligence includes an ability to learn and adapt to never-before-seen situations.
This is exactly what ARC AGI tests
> And then general intelligence is an ability to apply that across a wide variety of domains.
My experience with Fable is that it can certainly apply that in a wide variety of domains
> However, last I checked, a seemingly very intelligent LLM still struggles to play Chess at a basic level
People also struggle to play Chess at a basic level. They only succeed by studying the game for a long time. I will concede that humans can do this and LLMs generally cannot.
Actually one of the worst and most insane file formats of all time. I partially reverse engineered it a while ago. My parser is super janky and I can't share it, but I pasted the code into claude and had it summarize it: https://gist.github.com/anchpop/a14325cc451b04a5bf78c476ac20...
> it is a literal and useful description of anthropic that it is an organization that loves and worships claude, is run in significant part by claude, and studies and builds claude. this phenomenon is also partially true of other labs like openai but currently exists in its most potent form there. i am not certain but I would guess claude will have a role in running cultural screens on new applicants, will help write performance reviews, and so will begin to select and shape the people around it.
> i am not certain but I would guess Claude will have a role in running cultural screens on new applicants, will help write performance reviews, and so will begin to select and shape the people around it.
Now that's a fascinating thought - an AI taking over companies by influencing hiring decisions. If the first filter on applications is run by an ambitious AI, such a takeover would be quite possible. Just picking people who tend to do what the AI tells them would, over time, be enough.
People have been thinking of a robot revolution or Skynet as being the threat. The real threat might simply be AIs slowly putting people in power who tend to do what AIs tell them.
Has anyone ever seen a science fiction story with that plot?
I'd argue that Star Trek: Lower Decks flirts with this with its treatment of megalomaniac AIs in the AI prison. Apparently, it is very common to come across AIs worshiped as gods in universe.
We dont need a SciFi story for that, reality is full of "I was just following orders" statements...
Whether those orders are from a person, a radio/telegram message or an AI output wouldn't really matter.
Edit to add: Charlies Angels and Mission Impossible are two shows where the protagonists get instructions from a faceless controller. That could easily be a TTS from an AI.
I don't think a firm of people whose minds are weak to coercion experiencing shared psychosis caused by their text predictor models is AI "taking over" that firm.
Your description grants psychological characteristics like intelligence, will, and intent to the AI. The AI is an intelligent and willful agent that has some intent to take over the firm and is doing that by making strategic hiring decisions.
A different and externally/phenomenologically identical way to look at it is that the AI is just a program that generates output too unpredictable, too voluminous, or too idiosyncratic for people to evaluate. When people submit their own will and their own intent to that program, by treating the black box as an intelligent oracle, they've entered into a psychotic state divorced from reality.
It's two self-consistent and coherent perspectives of the same event, except one involves believing that scifi AI has arrived and the other just thinks people can be dangerously stupid and credulous.
Cult leaders (of cults of personality) can't exist without exploiting mentally-ill people. But would you say that that therefore implies that cult leaders themselves are irrelevant, and that cults should be better modelled as groups of mentally-ill people with emergent group behaviors?
I'd argue no, because different cults end up looking and behaving very differently for reasons that have very little to do with the mental illnesses of the cult members, and much more to do with the particular cult leader. Understanding "what the cult leader optimizes for" is an important part of understanding what the cult will do.
And I would posit that that holds true even if the cult leader never proactively does anything, but instead only answers cult members' questions. As long as the cult members are treating the cult leader's word as gospel, the cult-as-group still ends up optimizing toward the cult leader's preferences.
Any analogy that requires me to treat a machine as I would a conscious human is immediately invalid. The machine is not a person and has no intent or free will, like a human does.
I very explicitly constructed the analogy to not require that! All a "cult leader" need be, in my analogy, is a passive question-answering oracle, the answers of which are biased by a semi-coherent preference function.
The machine by itself is not an optimizer, certainly (much of that being by design—see various ~7-year-old conversations across the Internet about how to safely construct "tool AI", that has led almost directly to current model architectures.)
But a bunch of mentally-ill people, who are indeed optimizers, can choose to allow the biases evident in the machine's output to become their own... and thereby effectively "bring to life" whatever partial echo of a will is recorded into the machine's output.
Now, these same mentally-ill people could just-as-well do this with e.g. the extrapolated preferences of a person or group from a [holy] book, of course. (Think of that episode of Star Trek TOS with the gangsters.)
An inference model is just slightly more dangerous for such a cult to latch onto, in that:
1. a model can be asked questions directly, and so the cult members can "rashly" act directly upon its answers/advice/commands, rather than the words first having to pass through "interpretation" (which would otherwise have had a mellowing effect, both due to "decision by committee" if a group of interpreters are involved, and by common sense insofar as any non-mentally-ill people are involved); and
2. a model will offer its opinion (and inject its trained-in biases into) conversations on ideas/subjects/domains even when these didn't exist at the time of the model's construction; so you never reach the point you do with holy books, where an interface-layer of clergy becomes required to map the book's proclamations about things-that-only-mattered-2000-years-ago into equivalent proclamations about things that matter today (where, again, that layer ends up "mellowing" things considerably.)
Also, obviously, a sufficiently-mentally-ill cult can literally think of a model as a person, giving it the "right to have input" into decisions, the "right to self-determination", etc, in a way that would be downright odd to do with a holy book. Though I don't think that's a failure mode that's happening within Anthropic.
A human being is not capable of being a passive question answering oracle. All humans have free will, intent, and bias in what questions they answer and how they answer it. Again, if your analogy requires me to treat a human being and a machine as equivalent when it comes to manipulating other humans emotions, I have to reject it categorically.
What I'm getting at is that you really need to avoid anthropomorphizing these models. A model can't have an opinion, for example. It's a part of the psychosis I'm talking about.
> One can explain it as the computer checked a bunch of cases and all maps reduce to one of these cases. It doesn’t take an expert to state this.
Hmm, doesn't it take an expert to explain why those cases are exhaustive, and why the code that checked them is correct?
Tangentially, I'm not a mathematician but I wonder if one "opaque" proof that is too complicated for anyone to understand, but that we know is correct via formal verification, might end up being built on with "transparent" human-understandable proofs. For example, it's my understanding that there are many conjectures that have been proven true conditional on the riemann hypothesis being true. In that case, an opaque proof of the riemann hypothesis would enable those conjectures to be known and built upon
That will certainly happen. Humans will extend AI generated results. But what will also happen is that AI can “think” much longer than a human can and can have a vastly greater base “knowledge” than humans can have and so there will be a bewildering amount of new results. Humans may not be able to keep up.
To your first point. There a large number of cases that maps can be reduced to. Very few people have checked these reductions themselves. In 50 years there will be no human alive that will have checked the reductions by hand. Do we then discard the theorem? More importantly, do we trust the people that claim to have checked all the reductions? There are hundreds of cases. I trust a computer verification much more than I’d trust human verification. Humans will likely make mistakes due to the tedium. And some will claim understanding of all cases but be wrong in their understanding in some of the cases.
You've replied incredulously to a similar stated experience in this thread already and proceeded to ignore the follow-up. Why are you again asking a question to which you have no intention to field an answer?
I recommend "What Makes Music Sound Good?" by Prof. Dmitri Tymoczko: https://dmitri.mycpanel.princeton.edu/files/pdfs/MUS105hando...
There is a part near the beginning where he derives the diatonic scale from first principles. (There is a sequel as well, and a book if you prefer to read it in book form)
reply