When I first started working at ChinesePod this year, one of my first tasks was to create a difficulty level classification scheme. I created it as a chart, and that chart is now online under “Academic Questions ” in the Help section.
Newbie lessons are generally understood to be lessons appropriate to an absolute beginner. However, we sometimes make exceptions to what is written regarding Function and Application in the chart. For example, the Baby Talk series is classified as Newbie, even though its application doesn’t really match the standard. The same goes for some of the recent lessons with regards to function; asking “How do you say…?” for example, requires some substitution and thus very basic grammar, but we still classify it as Newbie for an obvious reason: it equips a Newbie to acquire new vocabulary on his own.
Challenges
I think some learners have mistakenly gotten the impression that ChinesePod does not have standards for setting the difficulty of the podcasts. Let me assure you that this is not the case; we have been using the chart for months. This is not to say that I believe that the difficulty level of our podcasts is always perfect, however.
The main challenges with setting difficulty are:
- Difficulty levels, by their very nature, must contain a degree of arbitrariness. As a result, judgment calls for each lesson are necessary. Some people may disagree with certain judgment calls, and the judgment calls will not always be perfect.
- Lessons have many parts (dialogue, banter, expansion, exercises, etc.) and are created by an entire team. Coordinating all the different parts to come together into a cohesive whole consistent with the difficulty level can be difficult.
- The ChinesePod user base is not evenly distributed throughout the entire spectrum of ability levels. Still, in creating podcast lessons, we are creating content for our current user base as well as our future base, which could have a different ability level distribution.
OK, but enough of the “this difficulty level thing is hard” whining. I’d like to address a few specific issues.
Difficulty Level Gaps
The difficulty level gaps arise from a number of factors. The main factors are:
- Non-sequential lessons
- “Drift”
- Judgment calls
First let me address the “non-sequential lessons” issue. This is a factor contributing to the level gaps, but it is not a problem. It is one of the foundations of the ChinesePod method, allowing users to find lessons on specific topics appropriate to their level without making assumptions about what the learner already knows. (Textbooks, for example, will expect you to know the material in chapters 1-9 when you start chaper 10.) The non-sequential nature of the podcasts opens up new possibilities, such as custom-built courses using My Course in the Learning Center.
Now here’s a problem: if lessons are not sequential, and each lesson is perfectly targeted to its difficulty level, how do students bridge the gaps between the levels?
The solution is to create a variety of lessons which cover a range of difficulties within a difficulty level.
This brings me to the second factor: “drift.” Lessons lately have not been as evenly distributed as the ones in the figure above. Recently, most elementary lessons have been quite easy, and too many intermediate lessons have been too difficult.
How did it happen? It wasn’t a conscious decision. “Drift” happens when the academic team isn’t constantly reevaluating and comparing lesson difficulty. It may also have involved the third factor: “judgment calls.” Occasionally, we’ll think a lesson’s difficulty level is OK, but the majority of our learners think it is too difficult.
I take responsiblity for the drift and judgment call issues, and I am already taking multiple steps to rectify the situation. I apologize for any difficulty level-related frustration you may have experienced, but I am definitely listening, and changes are already in motion.
Things are going to keep getting better. That’s what we do.
I think that sounds great. In a previous incarnation I have worked in quality management in various industries and understand the difficulties. The problem here is the difference between attributes and variables.
A variable quality measure is easy to apply for example the lesson must be less than 20mins (pass or fail, black and white), the lesson must weigh in at less than 25MB (again is passes or it doesn’t).
An attribute usually involves some sort of subjective measurement, is this lesson level to hard (shades of grey).
It seems that Chinesepod are doing everything I would expect to control these issues, everyone should remember that usually the 80:20 rule applies in these type of circumatances (get 80% there for 20% of total effort) the remaining 20% of effort taking 4 times as long. In fact you could make an arguement for saying that to achieve 100.00000% perfection would take an almost infinite amount of effort. Personally I am happy for the lessons to be mostly correct whilst still getting them at the same rate of output. Remember that this 20% is time based in this case It does not mean that people are only doing 20% of their job you can increase efficiency to a certain extent but you can’t keep finding ways to work (cheaper, faster, better).
My only additional comment would be what about a new category of bridging lessons two or three lessons that sit between each level and attempt to address the issues around the transition. To some extent, even the placebo effect may be beneficial here (some people may be more comfortable with a sort of ‘rite of passage’ through to the next level.
John
Excellent presentation. I have no doubt your opening here is going to generate a great deal of thought and output from the community. I especially like your plot showing how a gap has been created between Elementary and Intermediate Levels. As well as how CPod plans to address this gap. It has always been obvious to me that CPod does include many educational experts and it is just a matter of figuring out how to apply your skills to a new teaching method. Ken or Steve also mentioned self assessment testing I think this will be useful as well.
I have a few points to start off with.
1. Unless it might cause problems is there an understanding or belief within CPod HQ of how long a Newbie will be a Newbie or especially an Elementary will be at the Elementary Level before the expectation is one should be able to make the leap to Intermediate? Obviously CPoddies are different than normal students in a University. (we have lives, responsibilities, differing priorities as well as no outside motivator-like getting a C+ on an exam–) And we are all very different with different expectations of what we hope to learn from CPod. But it would be interesting to see what CPod might view as a path in time through the levels.
2. You plot showing lessons within each level with and without drift I think is good. However, can you explain a bit more. You suggest Newbie is flat. Ok this is so anyone last year, today , next year should be able to enter without going back to the beginning. Yes?
At the same time, do you think within the other levels there is or should be a drift upward with time in difficulty/complexity? In other words can you get much better over time without getting out of a comfort level within a level just because it is tending to be more difficult/complex with time. I think many of us have raised this point in different ways for a long time. How do you graduate upward? In a traditional class setting there is drift upward within Chinese 101 during the semester and when you graduate to Chinese 202 you start with lesson 1 in a new class room. And if there is a drift in complexity in time on one hand I would suggest the steepest slope(improvement should be in Elementary. However that will mean in time a gap for those trying to move from Newbie to Elementary. This is a very comples issue in my mind how to have no opening enrollment date to class.
3. I think your definitions of Low (newbie, elementary) Medium (intermediate, upper intermediate) and High with the abilities within are interesting and enlightening if I understand your assessment of skills acquired at each level. In terms of Function, Application and Fluency for Newbie/Elementary you use basic, limited, short, know little but at Intermediate the words are wide, wide and can speak. That’s a big jump Yes?
4. One last point on your drift plots. Again this presentation is excellent. But I would also suggest that it would imply that the most significant number of lessons in order to bridge the acknowledged gap should be the Upper Elementary and Lower Intermediate ones. Yes? I don’t think it is just because a few in that group are blog vocal. Your plot shows the obivous need for many lessons around that ceiling of entering the Intermediate Level.
And if that is true it would really be great to layout from the responses Ken asked for what is an Upper Elementary Lesson or Lower Intermediate Lesson for sure some of the recent lessons do seem to hit this area. As do I would say the first dozen or so Intermediate lessons broadcast.
I really look forward to other CPoddies comments and I hope you will stay active during the dialog as well.
Mike in Jubei
“Difficulty” is definately a highly subjective concept - especially when you have learners with highly heterogenous backgrounds and a nonsequential approach. But besides the qualitative dimensions like “Function” and “Grammar” you mention in the table I think there are indeed several “hard” criteria relevant for difficulty, e.g.:
- Length of a lesson in number of characters
- Length of a lesson in number of words
- Number of *different* words used in dialogue and/or conversation
- Percentage of words from the dialogue translated and/or explained
- Percentage of English in the conversation
- Average length of words in characters
(important especially for pronounciation)
- Usage frequency of characters used in Chinese
- Usage frequency of words used in Chinese
(some of the words in the “Photography” or “Calligraphy” lessons I had never heard before your shows - in English)
- As a classifier for the above two measures: Media source for frequency count
(daily family conversation, kid’s book, wuxia TV show, boulevard magazin, love novel, TV news, SciFi novel, newspaper, business conversation, business report, court talk, research journal, philosophy book, classical Chinese literature etc…)
There is definately some correlation between those criteria - but I do not think they are measuring the same dimension (time for a factor analysis?). Anyway: Given those criteria you can objectively *measure* that Elementry got easier over time while Intermediate got harder.
Besides that I personally try to get the most out of *all* levels handed out here (except “Newbie”). Unfortunately for the advanced shows that approach got extremely hard since the day the translations where cut away from the PDFs. Before that I used to learn a lot from advanced shows, too, even if I am by far not at that level yet (vocabulary, structures, cultural insight). So without translations for the advanced shows perceived difficulty jumped through the roof. By the way: For my English I still frequently use German/English translations (dict.leo.org) - if necessary backed with English definitions (http://www.m-w.com/) and of course Google for context…
John gets my vote for the best closing to a blog post in living memory:
“Things are going to keep getting better. That’s what we do.”
Beat that. I dare you.
Ken
KEN: “Beat that. I dare you.”
How about: “CPOD … 100% Da-Shan free!”
FDW,
Not bad.
Btw, I’m stil trying to figure out Walter Benjamin.
Ken
Thanks for all the work, and for sharing your thinking with us.
John, in the chart is difficulty level the x-axis (going across the page) or the y-axis (up the page) ?
The levels differ not only in content, but also in style of presentation, e.g. teachers speak English or Chinese.
One way to move across those boundaries is to have lesson difficulties overlap, so that a student can move up to a difficult presentation style with content that feels a little easier. But you’d have to know which were the easier lessons in the new level or you might miss the plot.
Another way to move across is to have a little bridging course.
Hi John
Another point I thought about biking home. Well since it was raining I didn’t have my i-Pod in my ears so not as dangerous as it sounds. Your plots on levels vs time are great. I hope/assume somehow along with other things there must be a publication/presentation out of this for yo and Chinesepod. Would you like to do the same for the banter? I think your plots may actually only refer to the pdf dialogue. Yes? The structured dialogues I think to some extent follow your plots both in terms of what they are and what you would hope they can be. Especially what they can be if you consider Henning’s point on word count. Before I actually went back and counted words and it was dramatic the jump from Elementary to Intermediate. Usually 5 times higher whereas Newbie to Elementary was at most 2 times more.
But more importantly the Elementary Banter today is 95% or higher in English and I would guess the last two months of Intermediates the lowest of the Intermediates lessons is maybe 20% English. So this is a significant gap. Even more I think from what I can grasp now compared to the early intermediate lessons , Jenny’s chinese banter was really as simple as it could possibly be back then. Now I am sure in low Intermediate lessons you and she attempt to keep it too difficult but it has crept up with time from the early lessons.
So my point is I think if you plot not just the ’structured” dialogue plots as you have done but if you look at the “banter” the gap is even greater.
So longer lessons at Elementary is one step. Adding degrees and amout of chinese banter within the Intemediate and Elementary levels is I believe an even more important step in bridging the gap between Elementary and Intermediate.
And Again John and All at Chinesepod for thinking and listening to us
Mike in Jubei
I agree. My issues with moving into the intermediate lessons wasn’t so much the lessons themselves, but the almost complete lack of English.
This may have already been suggested, but how about a ‘Lower Intermediate”? Becuase, even if there are easier level intermediate lessons available, I wouldn’t be sure how to pick them out.
I congratulate everyone on the excellent comments. In particular, Mike’s comments about “how long a Newbie will be a Newbie” struck home with me. I am a “Newbie” (what an awful degrading term) and after reading some of the recent comments I have determined that I spend too much time worrying about “graduating” to the next level and casting about for “magic bullets” or “miracle methods” to help me learn and not enough time just concentrating on the lessons as they have been presented. One of the things that C-Pod can’t control is self discipline. That must come from within. Let’s face it, Mandarin Chinese is a difficult language. With the amount of Newbie lessons curently available, if one listens several times to each podcast and also does the exercises at the rate of one lesson per day (or about an hour per day) it will take about four months before one needs to worry about crossing the line into Elementary territory. In the meantime there seems to be this pressure, real or just percieved, to hurry and catch up because the train is leaving the station. I really like the addition of the audio clips to the lessons. I wish there was a way that you could play them all together in a continuos string with only slight pauses in between. That would give Newbies practice with a kind of “rapid fire” effect even though to accomplished speakers the speed of the speach would be considered slow to normal. I am totally impressed by what ChinesePod has done for me so far and I have no doubt that it will continue to help me tremendously in the future. However, as far as learning to speak, read, and write Chinese in a fluent manner, the bulk of the effort is up to me. Oh, well, like my Mother always said to me, “Onward ever, backward NEVER!”
I think the Chinese language banter in the intermediate lessons is very valuable. Clearly it’s a challenge to have natural spontaneous conversation, but still be at a specific level. I think part of the reason Intermediate lessons are now harder is just that putting John and Jenny together results in more advanced banter that Ken and Jenny did, though I can see your trying to aim at the right level.
Perhaps it would be worthwhile to make a new sequence of lessons, like the Intro 1.. Intro 7 of newbie lessons but aimed at people just moving into intermediate level. They would focus on the kind of phrases that come up in the banter at this level, like
今天的课程关于什么呢？ What is todays lesson about
这个词是动词 this is a verb.
etc.
John,
It is great to see that there is so much time being spend on the ‘meta’-work. I’m sure that your presentation is the result of long hours of pondering and brainstorming about how to create the best possible podcasts and accompanying material. For me it is a clear indication that ChinesePod has crossed a barrier and is here to stay. I hope that the business sid of things works out in the end. There is no such a thing as a free lunch (sorry for the cliché, but it is true), so I hope that enough people out there realize what a tremendous job the CP team is doing and what great service they provide. Personally I have been using CP as complementary to regular evening classes. Gradually CP taken up more and more of my effort and my ‘regular’ course has sort of been sliding away. Since I have redirected most of my efforts to CP my chinese has improved immensely. I’m now at a stage where I can actually follow your advice (given a month or so ago) and I can form sentences in my head trying to express whatever I would be likely to say in a given situation. I’m also at a level now where I can understand bits and pieces of mandarin when I hear it, not enough to really understand what is being said, but enough to clearly distinguish the phrases, the general structure and enough words to know what the conversation is about. This breakthrough happend over the last few weeks and I feel that I owe entirely to ChinesePod. Keep up the good work!
Marc in Belgium
Lots of interesting ideas in this thread and the point that interests me the most keeps coming up over and over. The banter at the Intermediate Level is out of reach for intermediate learners. But this banter contains priceless learning opportunities as well. Instead of restructuring your whole format and changing everying around, why not just provide some sideband support for the banter, like you do for the core content. Make this available at the premium level (that should boost income a little :)). Lantian offered some great examples of banter material you might want to highlight in a post to “The Level Gaps on Chinese Pod”.
But, what ever you do, please include a transcript of the chinese banter. This is precious stuff for anyone just starting to comprehend chinese spoken at a normal pace. Also, by reviewing this you (ChinesePod Staff) can get a better idea about how to improve the banter for more effective leaning.
I spent a many hours and and a fair amount of patience of my chinese friends to translate one of the Intermediate lessons (#4 Catching up). I will post it here so you can more clearly see what all the fuss is about. (My sincere apologies for reducing Ken’s contributions to parenthetical remarks)
大家好，我是 Jenny. (Hi, I’m Ken)
今天我们要为大家准备了一个中级课程。(That’s right, …)
这也是很好的一种练习. (dui, dui …)
那今天我们有一段两个女孩子之间的对话是在谈论他们的长相. (Talking about the way they look …)
我们就一起来听听这短对话吧。
我们现在就开始吗？
那现在我们就一起来学习下不同的单词好吗？(Okay, so the new vocabulary ..)
第一个重要的单词是”最近”. (Recently)
在对话里面说的“李小姐最近怎么样？”(So, how you doing recently) 我们可以说”上海最近的天气怎么样” (So, hows the weather recently)
还有就是”最近我很忙” (I’ve been busy recently).
这都是不同的”最近”的用法。 那第二个我们看的就是“比”这个词非常非常重要的一个用来做”比较”的词。(Okay, so bi jiao means to compare something)
那在对话里面我们说“你好像比以前胖了一点了”(Not a sentence I would recomend ..)
那其他比较的时候你可以说“我比你有钱”(ah, richer than you)
逅者你可以说“她比我年轻”(She’s younger than I am ..)
大家都想要长大“我比你高”
还有一个词第三个就是“多”后面加动词。比如说“多运动”(Exercise more)
后者我们也可以说“多吃饭”(Eat more)
这个经常，爸爸妈妈会对自己的小孩子说“多吃”，还有“多工作” (Do more work)
当然我们也可以用“少”来说。比如说：“少吃”(eat less)”少运动”(exersize less)
在“多”和“少”的后面家动词 (the verb, right)
接下来，”前面的那个人” (the guy standing in the front)
最重要的这里就是“前面的”(the one in front)
当然我们也可以说“后面的”(the one at the back)
比如，“在你后面的那个人”(the guy standing behind you)
当然我们还有“鹏边的”(beside)
比如说“在我鹏边的那个人” (the person standing beside me)
好。接下来，我们还是看一下形容人的方法。“穿黑衣服的那个人” (The guy wearing the black clothes)
我们当然就也有很多，可以做很多的变化[bianhua]。比如说。“穿红衣服的那个人”， “长头发的那个人”
逅这用动做来形容 “打电话的那个人”， “在笑的那个人”(the guy who’s laughing)
好，那么我们已经学了很多很多的不同的生词，应该差不多了吧？
(Very important structures …)
特别是现在中级，我们需要到学习中心，到网上看。
对，很重要，对吗？
那么就这样吧。
好。那我们最后再听一遍那段对话
I would like to follow up on Ma Ding’s excellent point. I would be happy to pay more. $$$$$ Somebody maybe its Hank should pick up on that. If Chinesepod offered even more value. I am sure that is not everybody but I would not be surprised if there were others who would as well. And one of the nice features of Chinesepod is that there is no pay level police. So in other words when someone posts something that could only come from the basic or premium level it is ok. So I applaud Chinesepod for assuming that maybe if others see a bit of what they could have if they can afford to pay something is great. As it is I do pay the full amount and beleive it is money well spent not just on learning Chinese but as well as a way to enjoy myself in general.
I could make a list of things I would pay more for but I leave it to you guys to ask if you are interested.
Mike in Jubei
Ma Ding has asked a really pertinent question. From the business perspective we’re way behind on what we could be doing because we’re still ‘pre-money’ - we’ve not yet received funding. (If you must know.) Our start-up mode will come to and end quite soon, however - details when I can confirm them.
This may explain why our marketing is still rudimentary - we don’t yet have a marketing department! However, just this very week we hacked out the kind of segmentation that will help identify much more clealry which target/level is most likely to take the premium service. Then we’ll be able to do soem of the things you talked about. We are also taking up on a previous suggestion of yours - email survery. (Actually we’ll do the surveys on the site.)
Mike, by all means let us know what you’d be willing to pay more for. It’s a sign of what is important to you - and therefore other learners. I am more than willing to listen.
Ken Carroll
A lot of the ideas are for improved measurements, such as Mike’s suggestion to track the fractional content of each lesson in Chinese , Erid Grimm’s request to transcribe the banter, or Ma Ding’s surprise at the lack of user satisfaction data. (I’m only recently a premium member, so I figured it was coming). Granted, all of this takes money. I went back and looked at the chart after reading Mike’s suggestion. The “language” column of the chart jumps from “mostly English” for beginner level to “mostly Chinese” for intermediate. Tracking the data Mike suggests is perhaps the best single additional piece of information you can use and provide without much effort. Timing it may be easier than counting the words.
Wow, so many great comments… I guess I’ll just plunge into a huge reply.
chris,
Good suggestions. We will consider those as we work to close the gap.
Mike in Jubei,
1. We do have some estimates as to how long we think it will take users to get to certain levels using ChinesePod.
2. Your understanding of Newbie as “flat” is correct. How can it be anything but flat is a new learner should be able to start learning Chinese with any Newbie podcast? You have to remember that new people are starting all the time, so there can be no “gradually shift upward” that will benefit everyone. That is why the best approach is to cover the entire spectrum of difficulty within each level non-sequentially. This means that within one level, some lessons will be harder or easier for you, but (hopefully) all will be helpful.
3. Yes, it is a fairly big jump, but I think that’s true when learning any language. That’s one of the reasons we need so many Elementary lessons, and why it’s crucial to have enough more difficult Elementary lessons and easier Intermediate lessons.
4. Yes, and that is why we are currently targeting the high-elementary/low-intermediate range (but not totally exclusively).
海宁 / Henning,
Those are excellent suggestions for analyzing lesson content quantitatively, and we will certainly be trying some of them when we have the resources to do so. (Don’t forget that we’re not a huge company… yet!)
AuntySue,
In the charts, the y-axis corresponds to difficulty (the higher you go, the more difficult), and the x-axis roughly corresponds to time (although not really proprtionately).
Michael and AuntySue,
Your ideas on specific lessons to “bridge the gap” are good. We’ll see what we can do with that.
Marc,
That’s great to hear! And don’t worry, we plan to become an ever more useful resource.
Ma Ding,
We have some market research, but you’re right in that we do need to hear more from our customers, and we are working on that. Anyway, we know that we have way more Newbie and Elementary learners than Intermediate and Upper Intermediate, which is why each week the podcasts are 1/3 Newbie, 1/3 Elementary, 1/6 Intermediate, and 1/6 Upper Intermediate.
The advanced podcasts are not aimed as much at our current user base as at an intermational user base we are working hard to attract. We are filling a need of high intermediate/lower advanced learners that is not really met anywhere else on the web. Our new “Advanced Media” podcasts, for example, are pretty revolutionary and one-of-a-kind (and not just because it was my idea!).
I have read many posts on the multitude of Chinesepod weblogs and I find lots of people who echo my desire to be able to understand spoken chinese, instead of just having it bounce off. My level must be pretty close to intermediate because all the Elementary lessons are too easy - although I do pick up a gem of wisdom here and there. One of the contributors made the point that before you can understand a fluent speaker you need lots and lots of practice - listening and speaking, writing and reading. But how does one practice listening when people speak too fast and use unfamiliar vocabulary? My answer is change just one of these variables at a time. Currently the elementary lessons have a lot of slowly spoken conversations with some interesting vocabulary. But where are the rapidly spoken conversations with familiar vocabulary? I would love it if I could practice listening without learning new vocabulary at the same time. One idea would be to add bonus to the elementary lessons. After breaking down the example dialog, present a second longer dialog that uses roughly the same voculary in a different way.
Eric said:
“I would love it if I could practice listening without learning new vocabulary at the same time.”
Oh yes, it would be wonderful, even if there were one or two new words. I’d walk a mile over broken glass to listen to something, just listen and understand, without learning a complete new set of vocabulary every single time.
Yes, it’s a good idea. The tricky part is that each person’s vocabulary is a little different.
Hi John,
It’s a bit topic-askew, but where would you ‘level’ this sentence?
我这个星期三下午老时间去你那如何
A friend of mine texted this to me and it strikes me as quite formal, almost literary, but is it just normal polite? I recognize all the hanzi, but the positions and grammar are quite unfamiliar to me. If you have a moment, any expansion on it’s construction would be interesting. I doubt this kind of construction is in any of my texts and Chinese people seem very flustured trying to explain it to me.
我
这个星期三
下午老时间 (how is lao used here?)
去你 (wouldn’t the other person usually be put at the front to create the ‘topic’?)
那如何 (using ruhe at the end is a common construction?)
In contrast, this is how I might say/write this: 星期三你有没有时间？，如何星期三你有时间，我们见吧。It strikes me that my sentences are very ‘basic’, I totally get that my friend’s Chinese is like a complex English “Wednesday afternoon if you have time is good for me.” I’d like to transition my constructions to something like this but I don’t know how. Most other Chinese speakers ratchet down their Chinese to me soon after we’ve talked for any length of time, (and to be frank, if they didn’t I’d be lost) I dunno, maybe this is how fossilization happens. I don’t see this kind of construction/lexis in the upper-intermediate and I kinda assume it’s just natural in the advanced. That’s a gap, and I don’t think the drift will cover it. …
Lantian
This is exactly why I suggested I would pay extra to have an on-line ( with time delay)tutor for written essays or questions. Take any CPOD Lesson Plan and turn it into a written Lesson Plan. Then you write back and have it marked with red pen by a teacher who is WEB 2.0 understanding. By this I mean we don;t fear teachers and their grades, they exist to help us improve.
You would either be the role model student or the ultimate pain in the ass. But that is why I also suggested our on-line teachers should be able to publish for ALL to see some of our efforts. I volunteer mine but I had you in mind as the one who is out there we all could learn from.
Mike in Jubei
Lantian,
Intermediate, I suppose. It’s not especially hard, but a few unfamiliar terms/collocations combined with no punctuation cues caused you to parse the sentence wrong and get totally thrown. It should be:
我 (I)
这个星期三下午 (this Wednesday afternoon)
老时间 (at the usual time)
去你那 (= 去你那里, go to your place)
如何 (how about it?)
I’m guessing you were unfamiliar with “老时间” and “你那” right?
Heck … *I* know “你那” (or “你那儿” or “你那里”) and Latian outclasses me in by geometric magnitudes, so I’m guessing something else caught his eye. I’ll be curious to see what.
John, In response to my suggestion about including some additional dialog of familiar vocabulary at faster pace you responded: “Yes, it’s a good idea. The tricky part is that each person’s vocabulary is a little different.”
That doesn’t seem like the tricky part to me. How complicated is it to develop a baseline vocabulary list for elementary students? You can start by taking an elementary level text book (for example, “Integrated Chinese” or “Intensive Spoken Chinese”) and copy the vocabulary index out of the back of the book. Then adopt it as your standard. It doesn’t matter if the vocabulary list is matches the experience of each student perfectly. If you use this list to guide your course development in the future, students will converge on it.
I think the tricky part is developing engaging dialog that stays within a prescribed vocabulary. But, if you can do this and enhance each of the elementary lessons with additional Chinese dialog, you will have a more effective method of improving listening comprehension. And good listening comprehension is an absolute requirement at the Intermediate level.
I think that a more reasonable baseline vocabulary for the elementary level is words that were covered in the newbie level.
I think the point that the non-sequential nature of the lessons does make it unreasonable to assume that a listener has learned other material at the same lesson. But I think it is fair to assume knowledge of a prior level. If the listener is not comfortable with the material from a lower level, then they are at the wrong level.
It has been mentioned a few times that what makes the intermediate level difficult is not the vocabulary in the written dialog but the discussion between Jenny and either John or Ken. Even when the dialog is difficult, transcriptions, translations and pronunciations are available.
The right way to handle a transition intermediate would be to analyze the banter, and to build elementary lessons to include much of this vocabulary. Eric’s transcripion above should provide a good starting point.
Whatever anyone else does, I know that I will be pouring over that transcription as I try to adjust to the intermediate lessons. Thank you very much Eric for your effort. And please pass those thanks to those who helped you.
The same things could be said at newbie level, too. One of the characteristics of this smorgasbord approach is that the bulk of what you learn in one lesson is thrown away when you move on to the next, in contrast to a text book approach where a lesson’s content is reinforced over the next two lessons and pops up a bit in lessons after that.
There are a few words (ni, wo, shi) that tend to appear in most lessons, but there is little overlap. Is that good or bad? Yes, it depends. It’s different. It doesn’t allow long term memorisation to examinable level of old vocabulary, but if you’re willing, it lets you half learn (learn and put aside) a larger quantity of vocabulary.
I found a page of a Chinese newspaper, and marked in red all the words I know, not many. Then I marked in yellow all the words I recognise in some vague way. Lots of them! The yellow words represented opportunity for quick re-acquisition when those words are required. If I’d been doing a sequential text book style course, most of my marks would have been red, and the few yellows would seem bad, like failures of memory.
I like having a broad spread of familiar but not well remembered words, around the core of daily familiars. It’s empowering somehow, provided I haven’t invested a lot of effort in the yellow words. If I had, I’d be disappointed to have lost them by disuse.
So it depends on your attitude, and if that’s the only thing you can change, it makes sense to change it to suit. After all, there’s good and bad in every approach, and the good aspects of our approach aren’t available elsewhere.
Still, I sometimes tire of only hearing yellow and white words, when surely I have enough red words to understand some novel sentences that combine them. Next I think why not compose my own and swap them with other students? There I hit the fact that we have followed different pathways, and, maybe more so at newbie level, our red yellow and white word groups are quite different.
That makes me not want to invest much effort in learning the podcast vocabularies except for their immediate use, and that, again, can be a good thing or a bad thing, depending how you choose to regard it and what you do with it. Maybe it makes more sense to feel duty to seriously remember a word only after it’s occurred in four lessons, instead of one, because then you’re likely to have spotted a word that you will get reinforcement for later. (And by then you’ve probably memorised it for good without effort.) Then the return on investment is optimised, and each lesson contains a lot of throw-away freebies that can be picked up at less cost later if required.
On transcripts and payment…
I would LOVE to have transcripts of the banter. I don’t think translation would be necessary: give us transcripts and let us slog through it. (I’d just start a losing battle if I suggested traditional as well as simplified, so I won’t…) Often with the banter I’ll understand the meaning, but miss the idiomatic expression of that meaning, and that’s where a transcript would be great.
Main reason: I find the intermediate dialogs fairly easy–even more so when I’ve got the transcript in front of me. I can DO those conversations in Chinese, although I sometimes need the vocabulary filled in for me. What I CANNOT do is the casual banter.
(I used to enjoy the Advanced lessons but they are now alas too far beyond me… or rather, I’m not willing to spend the hours looking through dictionaries instead of just scrolling down the page.)
On payment: right now I’ve got the premium subscription because I’m on summer vacation and have more time. But at the end of the month I’ll go to Basic and probably stay there. However, if, hypothetically, the banter transcripts were available for the premium subscription… that could push me over.
Actually there’s tons of recycling throughout the newbie level. We use the same words again and again, but it may not seem like that because they’re used in different ways, different contexts. We don’t often draw attention to it, but it’s there.
Examine any language in the world and you’ll see that 80% of normal conversation comes from a surprisingly small number of words. I’ll try to write a longer post on this later in the week.
ken
Ken,
How’s funding looking? I manage a Private Equity fund in Shanghai. Let me know if you need assistance figuring out the maze that is pre-money start-up fundraising.
Nick
@John: Quantitative measures: It was not my intention to suggest that you should apply all of those meticulously…but maybe they can be used as an additional basis for rough estimation. And some of them you might indeed harvest automatically using some kind of simple handmade counting software.
@Mading/John: Regarding the target group: I agree that the biggest part of the audience is naturally at a Newbie or Elementry level. But you also have to consider there at that stage competition is also extremely fierce. I have found gazillions of free web ressources, text books, and learning software teaching you your first”你好”. But if you are looking for content on an Intermediate or Upper Intermediate level high-quality resources become extremely scarce. You either find highly advanced stuff or total-newbie content -only rarely Intermediate.
So I can follow John with his conclusion that Upper Intermediate is an USP of CP - I would add the “regular” Intermediate (and the earlier slightly “Upper Elementry” lessons) as well.
Nick,
We’ve secured funding but it won’t be official for a couple of weeks. I’ll let vyou know.
Ken
Hi Ma Ding,
Sorry again for that. It’s our spam filter. Anyways, I’ve tried recovered all the messages you wrote.
-Eileen
Hi John,
Wow, busy thread. Yah, you’re exactly right. I didn’t know those two combos. The ‘lao’ shijian I totally get…now that you explained it. Doh’. And ‘ni na li’ sure is a nice compression. I wonder how I’m suppossed to ‘get’ this though. Have I not been reading my interemediate texts enough? Are there even any texts that have phrases, examples like this? How is it that you know what they mean? I’m gonna start tossing out ‘ruhe’ at the end of my sentences instead of ‘ba’ 吧 and see where it gets me. I’m a bit disappointed with Ping’s ‘A Comprehensive Grammar’, he doesn’t cover 如何 at all.
As Jenny would say, ‘Thanks big man.’ Oh and to continue with the thread, pre-money, post-money, plain ol’money — I’m all for it, 如何.
John, 10 out of 10 for this article. I am now, for the first time, completely clear on the underlying logic of the whole thing (the diagrams helped a lot). Many thanks for taking so much trouble to explain it all and for doing such a good job of explaining it. Geoff.
Glad to read all the postings here…& thanks John for a very clear outline of where the levels are perceived to be. I agree that some brief translation of part of the banter would be helpful for lower Intermediates (me)…or a post lesson-posting of one or two phases/expressions from the banter instead of simply extra vocab. as Connie (thank you!) is currently doing -would also be great. Than a re-listening would be more beneficial rather than still puzzling. I realize it depends on individual effort/emphasis & time on what can be extracted. I’m really grateful to hear the language daily with no other access to the language currently …but I know with more focus I could be improving more quickly. What would help me more with picking up the spoken spontaneous language would be a slightly slower pace … even the speed of delivery in the dialogues is often too fast for me till I’ve worked at the transcript to grab hold of the words (often new),let alone get the meaning.Maybe that is just indicating where on the scale of listening to real language I’m at.
John, I looked at the chart and its great to see the big picture. But it seems to me, that the problem with level gap is not a matter the team drifting away from the chart, but rather the chart not providing enough guidance for the team. I hope you will forgive my bluntness on this point.
The chart is a great for communicating to Chinesepod users what to expect at each level, but it doesn’t seem to be sufficient for guiding the development of lessons. Chinesepod is like as a language course or text book series where the lessons are being written without curriculum guidelines. Maybe I am overstating this, but that is my impression. You guys do a great job of creating content that is meaningful and engaging, but curriculum guidelines can be useful for creating coherence. For example, if you are a teacher planning your 5th grade lessons it is helpful to know what the 4th graders have learned and what the 6th graders are expected to know.
The chart divides Elementary, Intermediate and Advanced levels into 2 sublevels each. That’s good, but from there the chart could use more granularity and specificity. For each sublevel, you could develop specific language objectives and methodology guidelines. The language objectives specify grammar, vocabulary and fluency objectives, while methodologies describe things like the core dialog, the English explanation, the Mandarin banter, the transcripts and exercises. I mean actually write down the vocabulary lists and the grammar rules expected at each level. And instead of specifying “mostly English” verses “mostly Chinese”, provide some percentage goals and content guidelines. The value of this would be to provide greater clarity to the content developers for targeting specific levels. The value to the user is that the system becomes a more effective learning tool.
And, getting back to my favorite topic, a well thought out set of curriculum guidelines would make it possible to progressively increase the difficulty of the spoken mandarin content as well as the rest of the material, to bridge the gap between elementary and intermediate.
After going through Eric’s transcript of lesson 4, I have more thoughts on the transition from Elementary to Intermediate.
I don’t claim to know all of the words used in the Elementary and Newbie lessons, but I have done all of the lessons. And in the transcript of the banter, I ran into some words that I didn’t remember hearing before. 比如 bǐrú, which means “for example” was used 5 times. Other words, such as 对话 duìhuà, “dialog”, or 用法 yòngfǎ, “usage” sounded familiar, but I couldn’t quite place them. Following Auntie Sue’s example, I highlighted those in yellow.
When I listened to the banter, I didn’t understand much of it. When I read the transcript, I highlighted some of it in red (for stuff I knew). When I listened to the lesson, pausing and rewinding to make sure I heard and matched each word, I was amazed at how much of the page I marked in red. The vast majority of the conversation used words that I had encountered, and so I just need to learn to hear what I already know. I love this aspect of the intermediate lessons.
Hearing brand new words used in conversation, without any translation, is the real hurdle in moving into the Intermediate lessons. Based on the frequency of 比如 in this one discussion, I think it probably meets the frequency requirements of the Elemenatary level. Probably a number of the other words that only appeared once also appear in a lot of other discussions, and so also occur frequently enough. Looking for opportunities to mix such words into the Elementary lessons would make the transition easier without having to build an “Upper Elementary” category.
Other words, such as 生词 shēngcí, “new word” may not be possible to work into a discussion at an Elementary level. I am not sure how you would use this except in a lesson about words. For some reason learning about learning seems more advanced to me. A brief vocabulary list might help bridge the gap.
Following the conversation with a transcript in front of me was a very thorough way to learn because I was so actively trying to identify what I knew and what I didn’t know. I think having a translation would have caused me to learn less. I would have glanced at the list, and said “ok, I know this and this, but not this”, and the ear training would have been lost on me. Even having pinyin might have had this effect.
I don’t know if I have the ability to listen to conversations and render them in characters, but I am going to try. If I am able, I will post what I come up with on the forum, and invite others to correct me. I think following along with that, looking for mistakes would have advantages that an official, authoritative transcript would not.
I think that posting a list of every word used in every conversation would make a reccomended vocabulary list too long to be of any use. In just this 1 lesson I found 25 items I didn’t think had been covered in lower levels, 5 of which I couldn’t find a meaning for. After doing a few lessons, it should be more clear which are the most common items.
Oh, and one note, in case someone compiles a list of amateur transcipts. There needs to be a facility for revision. In the transcript above, there were 3 times where different characters were used to represent sounds similar to huòzhě. Based on the context, I think the correct characters should have been 或者.
This is a fascinating discussion. As a graduate of a discipline that has tried to quantify learning with only very limited success ouside of the purely conceptual (psychology if anyone’s interested) it seems pretty clear to me that there is no exact science to learning. Having times when everything clicks together and you feel great, and times when it seems an impossible task, seems to me all part of the rollercoaster of learning such an intriguingly different language from our own.
I’ve started recently going back to the early Intermedidate lessons and rewinding the dialogue, trying to guess the pinyin and flipping through a dictionary until I get the right character. With some dialogues, like number 9 (Talking about Illness) and number 11 (Shopping) I’ve been quite successful leading to a couple of the click moments. I’ve just tried with number 13 (Discussing Work) though and got nowhere - impossible task moment.
Being forced to try and untangle something by listening does seem like a really good discipline and I feel it’s really helping me to make progress. But I just wish there was some way of preventing the helpless feeling you get when you don’t manage to get anywhere just by this method. I’ve found myself going to the relevant discussion page in the hope that some fellow Poddie (thanks for that neologism Jenny), has posted a transcript. But that only seemed to happen once.
Would there be any chance of someone from the production team being assigned to answering questions about new Intermediate lessons?(I realise it would be impractical for the ‘back catalogue’) . I imagine many of the questions we have would be the same ones, so they’d only have to be answered that might function for what one Poddie aptly terms the ‘get out of jail free’ card.
Hi John,
It seems you have a system charted out. That’s good but still begs the question as to what criteria you employ to section up/target your various levels. And how well that criteria allows for effective targeting. It would seem to me that the gap between what you designate as Elementary and Intermediate is larger than you anticipated. Given that, why is it more effective to create Higher and Lower Elementary within the one level labeled ‘Elementary’, along with Lower Intermediate and Middle Intermediate in the one level labeled ‘Intermediate’, and then mix and match them from one lesson to the next in an attempt to cover the bases. Wouldn’t it be easier to simply have more precisely targeted levels labeled Elementary, Upper Elementary, Intermediate, Upper Intermediate?
As I understand it your plan is that some days the lessons within any given level will be easier some days harder. Fine, but how extensive will the difference be. And how are we to know before trying a lesson out? Your approach seems a potential recipe for user confusion/frustration. I would still say, that, however you dice it, the gap between your current Elementary and your currrent Intermediate cannot be filled by tweaking. All that said, if you feel you have the answer to these concerns go for it. I look forward to improvement. Thats what you do
Difficulty is a hard thing to define and any course struggles with this. All traditional structured books jump around with some lessons seemingly much harder / easier than the next, particularly if you switch sources. I think the cpod chaps have a pretty decent system, although (as i have grumbled elsewhere about) i think there is an increased gulf between the new advanced and the upper intermediate lessons.
I have now realised that what i miss is the free flowing banter that aggie and jenny used to have on the advanced lessons that was great to just listen along to - the banter for me was always more interesting than the dialogue itself. The current advanced lessons are definitely an improvement as proper advanced lessons - most importantly in the use of more formal / “grown up” chinese in both the dialogue and the discussion, and in the use of more serious topics. But for me they are a real stretch and require a lot of time and effort to get anything out of.
I find the upper intermediate lessons great for explaining grammar points /vocab and for really using as a lesson that can be understood to near 100%, but i miss the fun and good practice of just listening to high tempo chinese banter that i can understand a large proportion of, which is what i used to get from the old advanced lessons.
Perhaps the answer would be to provide something similar to the “advanced media” lessons for intermediate users? The emphasis would be more on the discussion than on the language, but the nature of the language used and the topics would be simpler. I have found that many chinese people are quite adept at discussing quite complex topics in simple chinese, and i think Jenny and Aggie did this rather well in the earlier advanced lessons*. These podcasts could draw on vocab / topics already covered and would form an excellent method of review. As such they could potentially be offered without any supporting materials at all and would be pure listening practice.
Clearly offering purely free content without any link to the paid-for subscriber area is not great for business. But given that the cost of production would likely be much less than a proper lesson (less planning, support materials etc) this may be tolerable if it seen to be a popular and useful service. An alternative would be to make these only available to paying subsribers.
Either way, i think these podcasts would build on what are chinesepod’s main advantages compared to other learning media: providing lively, up-to-date, friendly and entertaining content that really can be listened to any time, any place (as opposed to being so hard that you really need to be sitting down with a dictionary to hand and stopping and starting the podcast to understand it!!). Such content is not easily obtained anywhere else at the moment, at least at the intermediate level - truly advanced learners already have a number of choices (e.g. BBC Chinese, VoA). It also fits with the philosophy of learning through 100% immersion that is promoted on the advanced site, and would provide a potential bridge into that site.
Well, that’s my tuppence worth.
Cheers
James
*As an interesting aside, i found this massively useful when i worked in shanghai earlier this year - amazingly I was able to be involved in discussions of quite complex concepts (credit risk management for one of the banks if you must know) using simple, straighforward chinese. For more formal occassions, or when i wasn’t involved i found the language shifted a gear into more complex structures and vocabulary that were much harder to follow. Admittedly, some people were less good at this “dumbed down” chinese - there were some members of staff i could never understand - but in general i found this simple, direct style a lifeline. I find this contrasts quite dramatically to english where i find most people really struggle to remove the idiomatic or the esoteric from their language when talking to non-native speakers. Perhaps this is the result of putonghua being a language that is still strictly taught in school (not the case in England for English) and that is used as a lingua franca across dialects? or maybe its just down to the intrinsically more clear, direct and simple structure of the chinese language? anyway, i digress…
Here’s what I’m looking for in a Premium Subscription:
(a) Sequential, progressive lessons that get me from elementary to intermediate, and eventually from intermediate to advanced. No gaps to magically jump across brute-force style. (The buck-shot, scatter-graph approach will help, but it’s still essentially a random roam, unless you connect the dots in a logically progressive way. So bite the bullet and go progressive.)
(b) For each newly produced podcast include review and drill sessions that are either embedded within the podcast or create it as a separate podcast. If separate, then you can locate it in the premium section and use it to attract new premium subscribers.
(c) Full transcripts.
That’s all; I’m easy.
If I could make a suggestion. You might be able to get some more insights from your graph if you labled both the X and Y axis. As it is, they both represent the same things thus creating graphic confusion.
According to your expanation above I think you only have information along a single Y axis (level of difficulty). By plotting all these dots on a Y axis you will see that all sorts of problems start to show up.
I’ll shut my trap now, which is no easy thing.:-)