Actuarial Outpost
 
Go Back   Actuarial Outpost > Actuarial Discussion Forum > General Actuarial
FlashChat Actuarial Discussion Preliminary Exams CAS/SOA Exams Cyberchat Around the World Suggestions


General Actuarial Non-Specific Actuarial Topics - Before posting a thread, please browse over our other sections to see if there is a better fit, such as Careers - Employment, Actuarial Science Universities Forum or any of our other 100+ forums.

Reply
 
Thread Tools Search this Thread Display Modes
  #31  
Old 12-27-2018, 03:05 PM
Marcie's Avatar
Marcie Marcie is offline
Member
CAS
 
Join Date: Feb 2015
Posts: 9,774
Default

Quote:
Originally Posted by therealsylvos View Post
Just curious, what happens if the graders can't agree with each other to reconcile? What's the procedure? Is there some higher authority?

I ask because I see plenty of questions in examiners reports where they say they took off points for some inane reason, (like not explicitly saying a retro plan is balanced even though the question itself stated it) and can't see myself ever giving in if I was a grader and agreeing to dock points for that.
Rarely (like maybe just once), I have witnessed some heated discussions at "central grading" over grading rubric disagreements. I imagine there are others that happen during the weeks leading up to central grading that I haven't witnessed. Usually before Vegas each grading pair has their differences ironed out.

Quote:
Originally Posted by PeppermintPatty View Post
In theory, I suppose they could ask the exam chair (the chair and vice chair of their exam, and of the spring/fall exams, and the VP admissions are all physically present at the grading meetings) but in practice, the one who cares more or is more persuasive wins.

Yep, usually one presents a stronger argument or just cares more, and the score gets nudged in that direction.

I would think the part-chair or part-vice-chair would be the authority they would appeal to if a pair gets completely stuck in disagreement, which I've only seen happen once. (In that case I'm not sure if they needed a higher authority to help resolve or if they did it on their own -- they were still arguing when I was done for the day, but the next day they seemed to have worked it out.) Since CAS central grading happens in Vegas, I'm sure an impromptu kickboxing or mudwrestling match could be worked out.

Quote:
Originally Posted by PeppermintPatty View Post
They do start with a rubric created by the item-writer, and are supposed to discuss their final grading rubric with the authorities for the exam. But I personally think this is a weak point of the process. I believe that the grades are completely "fair" from candidate to candidate to the greatest extent humanly possible. But I'm not so convinced the grading is always "right". That IS why they publish the examiners' report and allow candidates (and others) to contest the grading rubric, which is usually pretty evident (at least in its broad strokes) in the report. I know of many cases where the CAS gave additional credit as a result of someone contesting the grading. (I even know an individual who passed as a result of someone else contesting the grading.)
Reply With Quote
  #32  
Old 12-27-2018, 03:31 PM
bsanders33 bsanders33 is offline
Member
CAS
 
Join Date: Jun 2018
Posts: 530
Default

Quote:
Originally Posted by almost_there View Post
That's nonsense. Actuary exams are very expensive compared to comparable professions e.g. accountancy, statistician and so on and require far more studying over many months. The standard of marking should be commensurate with that.

I think bsanders33 is keen to maintain the false meme that anyone who complains must be some rogue student, some bitter exam failure. This kind of backward thinking does not advance the profession in any way at all, quite the opposite.
I think there's probably always going to be room for improvement in the way exams are written and graded. And after you have finished your exams and attained fellowship, I would encourage you to join an exam committee and work to make the process better. I was on an exam committee for several years right after FCAS, working up to a chairperson of one of the parts. And I will say this -- it makes a huge difference in the process if you have folks on your committee who are very conscientious and committed and fair-minded. (Which, for the most part, I felt was the case in my committees.)

But I'm telling you the truth -- any time you spend fretting about this as a student, or complaining about it on message boards or social media, or getting frustrated about it .... is totally counter productive.

If you do have a specific complaint about the way one of your exams was graded, make the complaint through the proper channels, then forget it and move on. Think positive, work hard, work even harder if you have to, and just get it done.

Last edited by bsanders33; 12-27-2018 at 03:36 PM..
Reply With Quote
  #33  
Old 12-27-2018, 03:38 PM
Breadmaker's Avatar
Breadmaker Breadmaker is offline
Member
SOA
 
Join Date: May 2009
Studying for CPD - and nuttin' else!
College: Swigmore U
Favorite beer: Guinness
Posts: 5,368
Default

Quote:
Originally Posted by bsanders33 View Post
I think there's probably always going to be room for improvement in the way exams are written and graded. And after you have finished your exams and attained fellowship, I would encourage you to join an exam committee and work to make the process better. I was on an exam committee for several years right after FCAS, working up to a chairperson of one of the parts. And I will say this -- it makes a huge difference in the process if you have folks on your committee who are very conscientious and committed and fair-minded. (Which, for the most part, I felt was the case in my committees.)

But I'm telling you the truth -- any time you spend fretting about this as a student, or complaining about on message boards or social media, or getting frustrated about it .... is totally counter productive.

If you do have a specific complaint about the way one of your exams was graded, make the complaint through the proper channels, then forget it and move on. Think positive, work hard, work even harder if you have to, and just get it done.


I too have used my extensive four-letter vocab to describe the exam process during my writing days.
__________________
"I'm tryin' to think, but nuthin' happens!"
Reply With Quote
  #34  
Old 12-27-2018, 04:27 PM
PeppermintPatty's Avatar
PeppermintPatty PeppermintPatty is offline
Member
CAS
 
Join Date: Sep 2001
Posts: 42,121
Default

Quote:
Originally Posted by almost_there View Post
Yes, independently.
While it's obviously right for the graders to do the first pass independently, it feels suboptimal to me to not have them reconcile. When I reconcillled with my grading partner, some of the differences were actual differences (I think this is a major error of understanding, you think this is a minor error) but some were errors on the part of one or the other of us. "I was able to follow through his work using that wrong answer he got in step 2, and actually, the rest of his work was right, even though none of the other numbers look good." "Oh, I completely missed that he wrote more work over THERE".

There aren't a ton of such issues, but there are enough that I think it's really valuable to compare. And of course, it's also good to agree on how serious that common error is.

Quote:
Originally Posted by Marcie View Post
How many times was s/he "randomly" tested for PEDs?

How many total scripts is that 10% out of? Is it 10% every sitting? Randomly distributed?
I'm pretty sure he means the scores he got from the two graders differed by more than 10 of the 100 points in the exam.
Reply With Quote
  #35  
Old 12-27-2018, 04:37 PM
Marcie's Avatar
Marcie Marcie is offline
Member
CAS
 
Join Date: Feb 2015
Posts: 9,774
Default

Quote:
Originally Posted by PeppermintPatty View Post
While it's obviously right for the graders to do the first pass independently, it feels suboptimal to me to not have them reconcile. When I reconcillled with my grading partner, some of the differences were actual differences (I think this is a major error of understanding, you think this is a minor error) but some were errors on the part of one or the other of us. "I was able to follow through his work using that wrong answer he got in step 2, and actually, the rest of his work was right, even though none of the other numbers look good." "Oh, I completely missed that he wrote more work over THERE".



There aren't a ton of such issues, but there are enough that I think it's really valuable to compare. And of course, it's also good to agree on how serious that common error is.





Quote:
Originally Posted by PeppermintPatty View Post
I'm pretty sure he means the scores he got from the two graders differed by more than 10 of the 100 points in the exam. RN
IFYP, Sir.

OP said that IFoA admits that 10% of papers have grading discrepancies of 10 or more marks and that s/he/it knows of an example where it has happened to an individual 4 of the last 5 times.

Maybe that candidate kneels for 'God Save the Queen'?
Reply With Quote
  #36  
Old 12-27-2018, 05:01 PM
almost_there almost_there is offline
Member
 
Join Date: Jan 2017
Posts: 328
Blog Entries: 1
Default

Quote:
Originally Posted by bsanders33 View Post
I think there's probably always going to be room for improvement in the way exams are written and graded. And after you have finished your exams and attained fellowship, I would encourage you to join an exam committee and work to make the process better. I was on an exam committee for several years right after FCAS, working up to a chairperson of one of the parts. And I will say this -- it makes a huge difference in the process if you have folks on your committee who are very conscientious and committed and fair-minded. (Which, for the most part, I felt was the case in my committees.)

But I'm telling you the truth -- any time you spend fretting about this as a student, or complaining about it on message boards or social media, or getting frustrated about it .... is totally counter productive.

If you do have a specific complaint about the way one of your exams was graded, make the complaint through the proper channels, then forget it and move on. Think positive, work hard, work even harder if you have to, and just get it done.

I'm afraid students have no other choice except to vent frustrations online and make their complaint known publicly because IFoA fob off their complaints. I think the situation is so serious that IFoA must be subject to strict oversight where such complaints would be taken seriously. UK actuaries are suffering from a monopoly situation.
Reply With Quote
  #37  
Old 12-28-2018, 07:49 AM
bsanders33 bsanders33 is offline
Member
CAS
 
Join Date: Jun 2018
Posts: 530
Default

Quote:
Originally Posted by almost_there View Post
They're not minor grading issues, see the link I posted. One guy has suffered from it in 5 consecutive sittings. There's no use telling people oh just study harder to overcome a flawed marking process. This just makes the system to be a lottery and makes the profession look a joke.
Really, why are you even worrying about this guy who claims he got screwed five consecutive times.

You are wasting mental and emotional energy on this. And you are wasting time on it.

A wise man on this message board once said the exams aren't fair ... because the fair is something that comes once a year, in October. You missed the fair.

I'm giving you honest advice, although I'm sure you don't want to hear it and likely won't listen to it. The exam process is not perfect by any means, and it is not always "fair". And when you finish your exams, I hope that you will join an exam committee and do your part to improve the process. Because these are largely volunteer-based exams, and they are only ever going to be as good as the actuarial members who volunteer.

In the meantime, quit obsessing over the "marking" (here in the States we call it grading), quit worrying about some guy who claims he got screwed five consecutive times, quit this silly campaign you're on, and quit making excuses. Do something positive instead. Do what I did when I passed all the exams on the first try: Load up on caffeine (we didn't have adderall and modafenil in my day....), hit the books like a mother effer, pile up huge hours of high quality study time, and slay the exams like a warrior.

If you do that, you won't have to worry about the subjective decisions of the markers. Because these decisions will only impact whether you get an 8 a 9 or a ten. And at that point, who cares, you've passed no matter what they give you.
Reply With Quote
  #38  
Old 12-28-2018, 08:11 AM
Kalium Kalium is offline
Member
 
Join Date: Dec 2014
Posts: 151
Default

Quote:
Originally Posted by therealsylvos View Post
I had the same issue, so I signed up for an account.
Within any exam marking system resources are limited - unless exam fees are to be increased significantly further - so it is unreasonable to expect a perfect system.

If both initial marks are significantly above or below the pass mark, then even if they are different, does it really matter?

Any case where there is a significant difference that straddles or is close to the pass mark is probably be looked at at least a third time, and possibly more. That will result in any marking errors being identified and addressed before the pass list is published, so the initial marks are not necessarily representative.

Such an approach lets examiners concentrate on the more marginal cases, which has got to be the best approach.
Reply With Quote
  #39  
Old 12-28-2018, 08:29 AM
therealsylvos's Avatar
therealsylvos therealsylvos is offline
Member
CAS
 
Join Date: Mar 2014
Posts: 17,653
Default

Quote:
Originally Posted by Kalium View Post
Within any exam marking system resources are limited - unless exam fees are to be increased significantly further - so it is unreasonable to expect a perfect system.

If both initial marks are significantly above or below the pass mark, then even if they are different, does it really matter?

Any case where there is a significant difference that straddles or is close to the pass mark is probably be looked at at least a third time, and possibly more. That will result in any marking errors being identified and addressed before the pass list is published, so the initial marks are not necessarily representative.

Such an approach lets examiners concentrate on the more marginal cases, which has got to be the best approach.
Look at September 16 ST-4. The pass mark was a 64. One grader gave him a 74, passing easily, one gave him a 50, failing horribly. In what world is that an acceptable discrepancy? The IFOA makes plenty from exam fees and dues, but clearly it's being wasted on other ventures rather than ensuring bare minimum quality control on exam grading.

The review process is something I obviously know nothing about, but at least according to the poster of the image, in each case the third marker was simply the average of the two. How true that is I can't say.
__________________
Quote:
Originally Posted by ao fan View Post
i like laying in the tube and pretending that i'm dead.

Last edited by therealsylvos; 12-28-2018 at 09:06 AM..
Reply With Quote
  #40  
Old 12-28-2018, 08:34 AM
therealsylvos's Avatar
therealsylvos therealsylvos is offline
Member
CAS
 
Join Date: Mar 2014
Posts: 17,653
Default

Quote:
Originally Posted by bsanders33 View Post
Really, why are you even worrying about this guy who claims he got screwed five consecutive times.

You are wasting mental and emotional energy on this. And you are wasting time on it.

A wise man on this message board once said the exams aren't fair ... because the fair is something that comes once a year, in October. You missed the fair.

I'm giving you honest advice, although I'm sure you don't want to hear it and likely won't listen to it. The exam process is not perfect by any means, and it is not always "fair". And when you finish your exams, I hope that you will join an exam committee and do your part to improve the process. Because these are largely volunteer-based exams, and they are only ever going to be as good as the actuarial members who volunteer.

In the meantime, quit obsessing over the "marking" (here in the States we call it grading), quit worrying about some guy who claims he got screwed five consecutive times, quit this silly campaign you're on, and quit making excuses. Do something positive instead. Do what I did when I passed all the exams on the first try: Load up on caffeine (we didn't have adderall and modafenil in my day....), hit the books like a mother effer, pile up huge hours of high quality study time, and slay the exams like a warrior.

If you do that, you won't have to worry about the subjective decisions of the markers. Because these decisions will only impact whether you get an 8 a 9 or a ten. And at that point, who cares, you've passed no matter what they give you.
Talking about 9s and 10s just shows how out of touch you are. I have literally never heard of someone scoring above an 8 on a CAS upper level exam. Most years a 7 is the 95th percentile. I don't know the distribution for IFOA exams though.

Look at the image provided. This person had one grader give him a 7, and one grader give him a 3. Quite aside from anything else that speaks to their grading being horribly arbitrary.
__________________
Quote:
Originally Posted by ao fan View Post
i like laying in the tube and pretending that i'm dead.
Reply With Quote
Reply

Tags
complaints, ifoa, issues, marking

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off


All times are GMT -4. The time now is 07:40 PM.


Powered by vBulletin®
Copyright ©2000 - 2019, Jelsoft Enterprises Ltd.
*PLEASE NOTE: Posts are not checked for accuracy, and do not
represent the views of the Actuarial Outpost or its sponsors.
Page generated in 0.28134 seconds with 10 queries