When perusing language learning articles, I see experiments where a small number of students learn words that don't exist (pseudowords) which they are subsequently tested on. Researchers then generalize such evidence to assert that students should learn languages in a certain way, arguing their proposed method is backed by this science.
However, there's a large non sequitur between Miss. Elliot's French class in Finland memorizing made-up words using one particular method and, say, me self-learning actual Chinese words using a plurality of methods. (For me in particular, I recognize ultra-rare words and skip them, which is what I would do with pseudowords.)
I personally find anecdotal evidence, perhaps through teachers or students, and perhaps via Reddit, YouTube, etc., far more applicable in the real world than these scientifically conducted experiments. In fact, it's sometimes these language-learning researchers (or people with advanced degrees in applied linguistics, etc.) appearing in e.g. YouTube videos giving anecdotal evidence. Hence...
Question: How important is anecdotal evidence when comparing L2-learning-method effectiveness in language-learning research?
I'm particularly interested in how much weight anecdotal evidence is given vs. these abstracted experiments. In certain fields anecdotal evidence means nothing (like physics), and in other fields anecdotal evidence is a lot more important (like psychology), where it's a form of qualitative data.
