标题: Can the Minimum Passing Score be curved up (above 70%)? [打印本页] 作者: richer1983 时间: 2011-7-11 20:03 标题: Can the Minimum Passing Score be curved up (above 70%)?
It was my understanding that the Minimum Passing Score was determined by taking the average of either the top ten scores or the top one percent and then multiply that by .7 (to obtain a number that is 70% of the top scores). I have heard many say that the top one percent are averaged but I also met a grader who told me that it was the top ten scores in years past.
So, by that rationale the minimum passing score can never be more than 70%, meaning you get a 70% you are guarenteed to pass. Is this correct? If not, can someone explain to me how the MPS can be curved up?
Thank you!作者: lc26mizzou 时间: 2011-7-11 20:03
I think it's the top 1% multiplied by the 70 pass mark to arrive at the cutoff. In December, the cutoff must have been pretty low cause I know I BOMBED the PM version like woah and ended up being a band 10. NO way I came close to a 70.作者: marsilni 时间: 2011-7-11 20:03
cfacowtown, Do you have more info on India being used to determine the overall MPS now? From simple statistics I would assume the larger sample size would just give us a better normal distribution ala the central limit theorem(you like how I used that?)
No offense to India but I highly doubt in the past they have like 50% passing rates and ours are at 35% so do you think it will have much effect?
Thanks
Ben作者: Benjiko 时间: 2011-7-11 20:03
Esco, if that is the case why is it that Europe was the only place to achieve the highest pass rate? Looking at the 2006 and 2007 overall pass rates(unfortunately not broken out by level) the US had a higher total pass percentage than Asia and Pacific Asia region in both years(just on a quick google search)
You may be correct but you also have a small sample size there ;-) I know of one indian friend who cleared level 1 on his first try but was already a CPA and MBA but had to take level 2 twice.
I am curious how this will matter and how it was done before, were we lumped in with the other regions too?
Ben作者: suyash1989 时间: 2011-7-11 20:03
According to my research, one of the two things will be proven wrong for sure this time around:
1) 70% score is passing
2) Only 30% of test takers pass
Let me explain why. The reason has to do with 3 choices instead of 4 this time around. First let's analyze the old system. Take an average student Joe Smith who just barely passed in Dec 2008. He scored 70% (per cfa institute - the passing score) and was the 30th percentile of test takers (per CFA pass rate of 30%). Joe knew x number of questions positively and y number of questions he had no clue about. So he randomly guessed on y questions with a 25% chance of getting it right. This leads to two equations:
x + y = 240 (total number of questions equal 240)
and
x + .25 y = 168 (total score must equal 168 to pass)
Solving these two equations leads to values of x = 144 and y = 96
Now Joe has a twin brother, Bruce Lee (unrelated to the kung foo star), with equal intelligence who takes the test in June 2009. Let's say Bruce also knows 144 questions with certainty and 96 questions he has no clue about. How much will Bruce score?
x + .33y (because now the probability of getting the unknowns right is .33 not .25)
Plugging x = 144 and y = 96 in this equation, Bruce will score:
175.68 out of 240 which translates to 73.2% !!!
So the twin brothers of equal intelligence and ability with same number of questions right would have scored 70% and 73.2% correctly, in their respective tests.
This leads me to believe that either the CFA passing score of 70% will lead to more candidates passing than 30% of the total candidates. Or, if they only pass 30% of candidates, then the passing score will likely be higher than 70% (more like 73.2%).
While you are technically correct I think all you are showing is that everyone this time around may have a higher score due to the perceived increase in guessing a right answer. It should still stay a level playing field with the level just simply being a bit higher due to the added probability of blindly guessing correctly.
The reason I doubt it has as much of an impact(besides for the people who really didn't know a lot on the test) is because on the old version you could usually eliminate 2 of the 4 questions giving you a 50/50 chance, and on this test you could usually eliminate 1 of the questions, giving you a 50/50 chance. So in reality for most people taking the test and not just having to totally guess the odds are not really that much improved.
Speaking of luck though it would be my luck to take the test the year a 70% does not equal a pass, but I doubt that will happen either.
Ben作者: ishfaque 时间: 2011-7-11 20:03
I don't think three answers rather than four will help anyone significantly, other than the person guessing everything (who will fail regardless). The typical person can usually reduce it down to two answers, and in this case it's two answers that would be in both the three and four question format. CFAI has said they just got rid of the obvious wrong answer. Therefore, 70% may still be an adequate threshold, but we'll see next month.作者: pacmandefense 时间: 2011-7-11 20:03
I didn't read the thread but props for possibly the best name on AF. well done.作者: pogo 时间: 2011-7-11 20:03
You should read my post on the "theory of minimum passing score"
Essentially all that matters still is how the top 1% of people do and what the CFA deems to be the MPS, that's it. If the new format helps people(which I don't think it does that much) that will just help others have a better chance of passing, not failing.
Assume the top 1% score 95%, they are likely not guessing on much of anything, so they should stay consistent. 70% of 95 is 67%. That is the MPS, if the new format somehow makes it so that those who would have gotten 66% now get 67%, then they pass.
The other factor is the MPS set by the CFA, I don't know how they combine these two if they are different but maybe they determine based on this test that the MPS should be 68% and not 67%. In either scenario, the level at which you would pass or fail is determined by the top scorers or the CFA institute which should not be affected by the slight increase in chance guessing.
Just my opinion but seems logical to me.
Ben作者: bluejazzy 时间: 2011-7-11 20:03
@legacy:
Yes, part of the idea is to show the higher score overall, but the greater idea that I am trying to drive home is that one of the two truths i.e. 70% passing score and 30th percentile passing rate will no longer hold true after the results. In my simplistic model, the test taker had no clue in the ones he guessed so the 50-50 theory doesn't apply there. I do agree that in the real test one must adjust for the educated guessing on some questions just as you describe - eliminating the 2 choices in old format and only 1 in new format - let's call it guessing convexity, ha ha). But on some questions, candidates would have no clue so that would be closer to the .33 probability rule.
Also, I did read your MPS theory and that makes a lot of sense. But here is one issue. For that theory to be true in the old version, and the two facts of 70% passing rate and 30th percentile passing rate to also be concurrently true, the top 1% must have scored 100%. 70% of 100% = 70%. Which is almost impossible. Now if the 70% passing score is not true then MPS can be true with a lower passing score of 67% or whatever the case may be. But then the claims of 70% being a pass score are not true for sure.
Summary: Either MPS is true or the 70% of passing score is true. But both cannot be true at the same time.作者: shootingstar 时间: 2011-7-11 20:03
@legacy
I definitely know that last time in December the passing rate was close to 30% rather than 35%. And I know the gold standard of 70% passing score, so that is what I entered in the linear equations above.
Off the analysis now:
Here is a philosophical argument against the MPS, if cfa does use this method. The MPS method seems to not account for easiness or the toughness of the test. Let us say a very easy test came (not too dissimilar from what happened in June 2009). Now, top 1% would be very close to 100, lets say 98. The MPS would say any candidate who scored 70% of 98 passes. This translates to 68.6. But what MPS does not account is HOW MANY and WHAT % of people are falling in this zone of 68.6>= to <= 100. So potentially you can have 50% of the candidates in this zone! Percentile based scoring would eliminate this bias and would ensure only 30% candidates pass. Of course the criticism there is that in case a smart batch of candidates appears, then the average candidate is at a disadvantage. So neither scoring approach is totally fair. Nor is an ad-hoc score because that would also not account for the easiness or the toughness of the test. A 70% score in Dec 08 test definitely meant more than 70% score in June 2009.
I do think cfa should have an either/or scoring approach under which if a candidate scores more than 70% then he passes for sure or if the candidate scores in top 25th percentile then the candidate passes for sure. But if the candidate is not in top 35th percentile or has not scored above 65% then he or she must not pass. So there is a small wedge between which can act as a buffer for the toughness or the easiness of the test and can be discretionally manipulated.
Edited 1 time(s). Last edit at Wednesday, June 10, 2009 at 03:57PM by mfin27.作者: burnsy562000 时间: 2011-7-11 20:03
mfin27,
I get what you are saying, the problem I have with percentiles(even though the probability is slim) is what you had mentioned about others being disadvantaged given a certain environment.
Obviously smarter people than myself have thought about this and likely come up with the best solution using the Angoff method, it does suck that we don't really have at least a general reference if we can't use the top 1% method as a guide. The Angoff method should theoretically take into account if a test is harder or easier because the experts would either say that the minimum qualified candidate would know say 168 questions for an easy test and 156 for a hard one(just as an example).
My guess is in reality they use a combination of the two methods to come up with a reasonable outcome and check against one another for a conflict. If the top 1% was scoring 90% and so 63% is the pass according to that standard, if the panel decides that the minimum qualified candidate should get 65% right, then they could say "wait a minute" and come up with a compromise, assuming a 2% deviation is out of ordinary and cause for concern of course.
I hear people saying that this test was easier and I agree but I agree because it was my second time taking it so I am also better prepared so I am biased and it should seem easier. I assume they know how to properly write the test and make it fairly consistent.
Prob just semantics at this point. ;-)
Ben作者: kmf229 时间: 2011-7-11 20:03
keelim, undoubtedly that would have an impact. That said, I'd say that the vast majority (upwards of 90%) of people who are going to pass the test will do so in a fashion where 10 minutes will make little difference. I guess my point is that even with both those factors considered, I doubt if it moved the average score more than about 1%.作者: orang3eph 时间: 2011-7-11 20:03
when i say indians, i mean candidates apperaing from india..That counts indians and chinese appearing from US as americans and so on..
