[HN Gopher] Anki SRS Algorithm : Spaced repetition explained wit...
___________________________________________________________________
Anki SRS Algorithm : Spaced repetition explained with code
Author : PixelForg
Score : 97 points
Date : 2022-12-27 18:43 UTC (4 hours ago)
(HTM) web link (www.juliensobczak.com)
(TXT) w3m dump (www.juliensobczak.com)
| codetrotter wrote:
| This is very nice and thorough.
| daniels11 wrote:
| Nice article! This would've been very helpful a few years back
| when I was attempting to make an Anki clone while teaching myself
| to code.
|
| I'd still love to create a better open-source SRS algorithm at
| some point in the future. Mostly to use for language learning.
|
| When I was learning Chinese characters, I did a bit of a deep
| dive into the Anki algorithm and found that the biggest flaw
| (imho) was the Ease Factor knocking a card back to 0.
|
| If you mostly know a card, but miss it on one day for some reason
| - maybe you were tired or distracted, that card's learning
| progress should not be totally reset (as if you were learning it
| from scratch). That leads you to have too many cards to review on
| a daily basis. Instead, you should use some modifier to increase
| the interval to a reasonable level.
|
| More explanation here: https://readbroca.com/anki/ease-hell/
|
| I think it would be awesome to pair SRS with high quality images
| and audio, which I find most helpful for language learning. I've
| used Rosetta Stone and Duolingo in the past; Rosetta Stone has
| great audio and images but lacks a powerful SRS (it also has a
| number of other flaws in my mind, but I'll save that for another
| time). Duolingo is great for grammar and explanations, but I
| can't take the pronunciations and tediousness of it all.
| imron wrote:
| If you're interested in SRS algorithms you should visit the
| site of the guy who basically invented them [0].
|
| Most SRS algorithms seen in apps today are based on one of the
| super memo algorithms in one shape or form.
|
| Anki was based on SM2, which was already outdated in terms of
| supermemo algorithms when released, but was chosen due to its
| simplicity.
|
| 0: https://supermemo.guru/wiki/SuperMemo_Guru#Memory
| alwayslikethis wrote:
| * * *
| Anon1096 wrote:
| You're describing a separate problem than ease hell, what you
| are looking for is fixed just by changing the lapses->new
| interval setting to be from 0% to something higher like 50%. I
| do agree that the default 0 is pretty bad though.
| jakobov wrote:
| Shameless plug: I developed a space repetition algorithm which
| has a different goal. Mine is focused on learning a large number
| of new things fast, rather than adding new things over time and
| retaining memory of them.
|
| See: https://github.com/Jakobovski/SaneMemo
| googlryas wrote:
| I don't quite understand how that is different? Is the idea
| that you have nebulous knowledge of more things instead of
| locked in knowledge of fewer things?
|
| What's your use case for this? Would it be good for learning
| say the top 5000 words in a new language or something like
| that?
| ATMLOTTOBEER wrote:
| I think the idea is that this would be more optimized for
| school settings where what you actually want is an algorithm
| for cramming.
| _Algernon_ wrote:
| It could make sense to cram for an exam for instance
| cmehdy wrote:
| The article is great - a visual that explains SRS in one single
| graph near the top of the page is a great proof that the article
| is solid.
|
| But I also want to say: what a clean and enjoyable website!!
|
| Thank you for having shared this.
| fallat wrote:
| For anyone interested: https://len.falken.directory/code/sm2.git/
| if you want to run something locally pretty easily.
|
| This post hits exactly what I found when researching months ago
| :) Absolutely captures everything well.
|
| This post has convinced me to even use Anki's algorithm.
| Kelamir wrote:
| I've noticed this https://github.com/open-spaced-
| repetition/fsrs4anki set of modifications for the Anki algorithm
| is popular in the Anki Discord. Might be of interest to those
| exploring the code.
| nmca wrote:
| I'm surprised that no spaced repetition systems seem to exploit
| relationships between facts/cards. One might imagine that
| capturing a graph of similar ideas would allow for better
| algorithms.
| kashunstva wrote:
| > I'm surprised that no spaced repetition systems seem to
| exploit relationships between facts/cards.
|
| In fact, the Anki community largely endorses as an article of
| faith the complete insular atomicity of cards. I've always
| treated that orthodoxy with skepticism because I'm pretty sure
| that's not how my brain works. Nor anyone else's, probably.
| trane_project wrote:
| Shameless plug to my own project: https://GitHub.com/trane-
| project/trane
|
| Dependency relationships between exercises are core to its
| design.
|
| I tried to use Anki but this limitation is pretty baked into
| Anki and made it unusable for cases where there's a clear order
| in which things must be learned (music in my case).
| nmca wrote:
| This looks like a very cool project! The docs include this
| description of the core algorithm:
|
| > The space repetition algorithm in Trane is fairly simple
| and relies on computing a score for a given exercise based on
| previous trials rather than computing the optimal time at
| which the exercise needs to be presented again. This will
| most likely result in exercises being presented more often
| than they would in other spaced repetition software. Trane is
| not focused on memorization but on the repetition of
| individual skills until they are mastered, so I do not
| believe this to be a problem.
|
| Could you say more about it?
| trane_project wrote:
| Sure, here's the gist of how it works.
|
| - When Trane is asked for a batch of exercises, it performs
| a depth-first search of every branch in the dependency
| graph. It picks up exercises as it goes along, and stops
| exploring a branch when the dependencies of the last node
| are not met.
|
| - Each exercise is given a 1-5 score by the student and
| those scores are used to produce a combined score, rather
| than compute a date on which the exercise should be
| reviewed. Two reasons:
|
| - Trane is also meant to be used to learn skills, which do
| not fit the assumptions made by the SuperMemo algorithm.
|
| - I believe most of the gains come from regularly being
| reminded to perform an exercise, so even a simple scoring
| algorithm should get most of the gains. Eventually, I want
| to replace the existing scoring with something more
| complicated that also makes a distinction between
| memorization and skill exercises to get better results. But
| based on my testing, this works for the most part as is and
| the gains to be made from more complicated algorithms are
| marginal.
|
| - Once you get exercises from performing the search, you
| reduce that number into the final batch by grouping
| exercises based on difficulty and selecting a percentage
| from each bucket. This is done to make sure not too many
| very difficult or easy exercises are included in each
| batch.
|
| There's also some elements of randomness added to prevent
| the same exercises from appearing all the time. For
| example, the branches are explored in random order and the
| elements from each difficulty bucket are selected at
| random. But that is the gist of it.
| Scaevolus wrote:
| Anki supports linking cards when they're different facets of
| the same information (e.g., each language source/target for a
| given vocab word): https://faqs.ankiweb.net/linking-cards-
| together.html
|
| More precision isn't _that_ important-- at worst, you slightly
| over-review some information, but that 's no different from
| using a flash card's information in normal life outside of a
| review period.
| majikaja wrote:
| In my personal system, I tag facts by sub-topic and the daily
| reps are ordered so that facts of the same sub-topic are shown
| consecutively to avoid context switching (eg. AAABBCCCCD not
| CABCCDBACA). In addition, there is logic so that after the
| mandatory daily reps are done, it will show cards that are
| close to the due-date (eg. within 0.25*LastInterval), again
| batched by sub-topic. This lets me efficiently get reviews out
| of the way on days when I have time to do so, without deviating
| too far from the basic SRS philosophy. As a result, for mature
| cards, I end up focusing on a small number of sub-topics on any
| particular day. I find that works a lot better.
|
| All of this is trivial to implement - I think it's best to
| create from scratch and specialize it for one's own domain
| instead of using a pre-existing solution. In my case the
| relationships are straightforward (sub-topics) but in others it
| might be more fuzzy. Programmatic fact-generation/deletion is
| also something I put a lot of effort into (again, domain-
| specific).
| aliceryhl wrote:
| An interesting example is chessable, which is a spaced
| repetition system for chess. It keeps track of each move in a
| sequence of moves separately, but every time it asks you to re-
| learn a specific move, you are asked to play out the whole
| sequence, instead of just being shown the position and asking
| you to play the next move.
| _Algernon_ wrote:
| The Anki FAQ goes into the reasoning a bit:
| https://faqs.ankiweb.net/linking-cards-together.html
|
| >Some people want to extend this link between arbitrary cards.
| They want to be able to tell Anki "after showing me this card,
| show me that card", or "don't show me that card until I know
| this card well enough". This might sound like a nice idea in
| theory, but in practice it is not practical.
|
| >For one, unlike the sibling card case above, you would have to
| define all the relations yourself. Entering new notes into Anki
| would become a complicated process, as you'd have to search
| through the rest of the deck and assign relationships between
| the old and new material.
|
| >Secondly, remember that Anki is using an algorithm to
| determine when the optimum time to show you material again is.
| Adding constraints to card display that cause cards to display
| earlier or later than they were supposed to will make the
| spaced repetition system less effective, leading to more work
| than necessary, or forgotten cards.
|
| >The most effective way to use Anki is to make each note you
| see independent from other notes. Instead of trying to join
| similar words together, you'll be better off if you can
| determine the differences between them. Synonyms are rarely
| completely interchangeable - they tend to have nuances
| attached, and it's not unusual for a sentence to become strange
| if one synonym is replaced with another.
| nmca wrote:
| This seems to boil down to "it would be more complicated",
| mixed with "the trivial thing is a bad idea", both of which
| seem true.
|
| But I can imagine a more complicated scheme could be much
| better; done right.
| _Algernon_ wrote:
| I agree. It makes sense for cards which link previously
| learned facts to only show up once the base facts have been
| learned, for example.
|
| But I'm not sure how to design a good UI for it. I can
| easily see it as resulting in too much friction so nobody
| uses it.
| type-r wrote:
| I have found this to be a huge missing piece of any SRSs I've
| come across. Personally, I have a pipeline that looks at what
| knowledge I've already learned and only then reveals successive
| information.
|
| For example, I'm learning numbers in Spanish. First, I ask
| myself to go from doce => 12; then, 12 => doce. Then, once I
| have a few numbers in my memory, I can start to combine those
| into diez + dos = ? (answer doce). The idea is that this last
| example requires recalling three previously-learned Spanish
| numbers to arrive at the answer. This is a trivial example but
| this underlying structure of successive layers of knowledge
| shows up everywhere and is horribly under-exploited by our
| current learning systems.
|
| Programmatically this can be done. Manually, this stuff takes
| way too much work.
| trane_project wrote:
| Check out my project: https://GitHub.com/trane-project/trane
|
| It does exactly this because I also thought this was a
| critical limitation of current systems.
|
| I have a feature coming out soon (already merged to master
| but I need to make a proper release) which should make it way
| easier to use with Markdown files acting as the front and
| back of flashcards that are automatically picked up when
| starting up the program.
| DennisP wrote:
| How would you do it programmatically? Seems like you'd always
| need someone to give it the relationships.
| trane_project wrote:
| Yes. That's required. I think they meant that the program
| uses those relationships to automatically advance the
| student to the next topic when it makes sense.
| optionalsquid wrote:
| Wanikani [1], an SRS for learning Japanese kanji and
| vocabulary, kinda works like that, though on a simpler level
| than what I think you envision.
|
| Basically their system involves first learning to recognize
| common parts of kanji (radicals), then learning to recognize
| kanji made up of those radicals, and finally learning
| vocabulary that uses those kanji. Later items in the graph are
| unlocked by getting the previous items to a certain SRS stage.
| Additionally, the entire syllabus is divided into 60 discreet
| levels of (very) roughly the same number of items, the next
| level being unlocked by getting 90% of the current kanji to a
| certain SRS stage, which helps keep it all manageable.
|
| I tried using Anki before, but found the Wanikani "method" to
| work much better for me since I could constantly build on
| things I had learned previously.
|
| [1] https://wanikani.com/
| Syzygies wrote:
| [flagged]
| tsumnia wrote:
| The primary hurdle is the number of "activity types" practice
| can come from. Simply reviewing material isn't sufficient for
| learning and so we need more engaging practice as well. Also,
| similar to how "all students are different", topics encompass a
| large number of subskills that need to be trained and not all
| activity types target those skills. Also, "learning" something
| is quite complex - is simple repetition of the fact sufficient,
| or do we need the ability to apply the skill?
|
| Its actually research we're doing at NC State. At present, we
| understand spaced, deliberate practice is beneficial [1] but
| students will largely reject AI recommendations to complete
| lower-level practice [2]. So our current methods are to develop
| a system that maintains an "instructor in the loop" to create
| tailored training regimens (my terminology for it), similar to
| how a coach builds regimens for athletes. The hope is that if
| we can analyze how instructors made their recommendations, we
| could then design AI to mirror the decisions.
|
| [1] https://dl.acm.org/doi/pdf/10.1145/3373165.3373177
|
| [2]
| https://repository.lib.ncsu.edu/bitstream/handle/1840.20/398...
| GenericDev wrote:
| I'm so thrilled to have come across this. Thank you for sharing
| this OP!
___________________________________________________________________
(page generated 2022-12-27 23:00 UTC)