Importance of the First 10 Questions - GMAT Prep What If Scenarios

Why Run GMAT Prep Scenarios?

The main reason for this post and analysis are the myths and debates related to the importance of the first 10 questions on the GMAT. At some point someone said that the first 10 questions are critical and absolutely pivotal to scoring high on the GMAT. The reasoning offered was that the adaptive algorithm is more aggressively varying the difficulty on the first 10 questions of the test and if one makes quite a few mistakes, the author of the theory argued, a test taker would not be able to recover since the subsequent questions were only providing incremental and much smaller contribution to the score. After this theory came out lots of people started spending huge amounts of time on the first 10 questions and trying to get them right at all cost, thinking it would result into a higher score, which prompted GMAC to come out with their "Myth" explanations that the first 10 questions actually do not matter. This debate has lived on for at least 10 years until 2 things happened:

1) GMAT Prep moved online (it used be a software you downloaded and the algorithm in the software was supposedly simplified and different from the algorithm used on the GMAT to prevent people from messing with it and figuring it out)

2) GMAC announced that the scoring algorithm of the online GMAT Prep is exactly the same as used for the GMAT. I don't have a link to the statement nor proof but I believe this was made in one of their webinars for test prep industry and is generally a widely accepted fact. Moreover since the GMAT Prep is a really good predictor of the final GMAT score, it should produce consistent results while running various scenarios. On the other hand, we have also run a number of these scenarios on the old GMAT Prep platform and the results are very consistent between the 2

Therefore, we are able to get a pretty clear picture about what happens when you miss certain questions and once and for all, and understand how important or not important are those first 10 questions. Note however, that these are lab tests in perfect condition (more on that in methodology).


Methodology:

1. We used GMAT Prep Tests 1 and 2 with a large question database.
2. When we evaluated a scenario, such as what I get the first 10 questions wrong on Quant or Verbal, we actually had to solve every question. We did not guess. We purposefully answered the first 10 questions incorrectly (after solving them ) and then proceeded to solve the other 21 questions to answer them correctly. In some cases, we used google
3. We ran scenarios multiple times to make sure the score consistent (oh what a fun day it was)
4. We evaluated Quant and Verbal sections of the GMAT Prep separately and recorded the results for both just in case.

What is the importance of the first 10 Quant Questions?

Experiment: First 10 Incorrect; 21 Correct
We attempted the worst case scenario and answered the first 10 questions incorrectly (not guess but incorrectly); then we answered the remaining 21 questions correctly. To make sure it was not a glitch, we have reset GMAT Prep and attempted the same process of getting first 10 questions wrong. We ran the experiment 3x times.


• Result 1: Q29, 11th percentile
  
• Result 2: Q30, 13th percentile
• Result 3: Q31, 14th percentile


Conclusion:
The first 10 questions are very important for quant. Q31 is the max quant score if you miss the first 10 questions

Obviously something is not right with the myth about the first 10 questions. If you miss the first 10 (probably not a very realistic scenario for anyone but it illustrates the point) you have no way of climbing out of the hole - almost all questions that were offered to me were generally easy/medium difficulty. The question mix was 18 PS / 13 DS questions in two attempts and 17 PS / 14 DS in another. It is interesting to note that when we simulated this same scenario in the previous GMAT Prep version with 37 Quant questions, this scenario resulted in a Score of Q38, 48th percentile - quite a bit higher! So, the importance of the first 10 questions seems to have increased on the current shorter version of the test (no surprise since now 10 questions correspond to 1/3 of the quant questions).

What is the importance of the last 10 questions?

Experiment: Answer 21 Correct | 10 Incorrect.
We attempt the worst-case scenario and answered the first 21 Quant questions correctly while answering the remaining 10 questions incorrectly (no guessing).


Result: Q49, 73rd percentile


Conclusion:
The last 10 question don't count as much as the first 10 on quant

The questions were much harder this time with some new topics/types appearing on the test such as coordinate geometry/probability etc. As long as you can get the first 21 right, you get a very high score and just guess the last 10 but it is not easy to answer 21 questions correctly in a row. At the same time, I have not guessed the last 10 but rather answered them incorrectly (which means I still had to solve those questions). If you are guessing, you will most likely get a higher score. Question mix was 18PS / 13 DS. When simulated in the previous GMAT Prep version, this scenario resulted in a score of Q50.

What is the importance of the middle 10 questions?

Experiment: Answer 11 Correct | 10 Incorrect | 10 Correct
We attempted the worst-case scenario and answered the first 11 Quant questions correctly; then answered the following 10 questions incorrectly, and finally, answered the remaining 10 questions correctly.


Result: Q49, 73rd percentile


Conclusion:
The middle 10 quant questions seem to have as high of value as the last 10 and by far much less important than the first 10 questions

The questions were not much easier than in the second scenario above even though I have made 10 consecutive mistakes. When simulated in the previous GMAT Prep version, this scenario resulted in the same score - Q49.

What if I guess or miss every third question?

Experiment: Guest every third questions | 10 incorrect | 21 correct
I will try to test a scenario in which a test takers answers every third question incorrectly (e.g. 3, 6, 9, etc. Thus giving themselves an extra 2 mins on quant). This scenario will simulate someone who takes 3 mins instead of 2 per question and then guesses every third question. Please note that this is the worst case scenario (we do not get any guesses right)


Result: Q48, 65th percentile


Conclusion:
Almost the same score as in the case of missing the last 10 questions.

Question mix was 18PS and 13 DS. Interesting that even missing every third question, you can still get a Q48, just 1 point lower than getting the first 21 questions correct in a row. It is much harder to get 21 questions correct in a row than to miss every third one as question difficulty will adjust down after 2 correct answers. When this scenario was simulated in the old Desktop GMAT Prep version, this scenario resulted in Score - Q49.

What is the Importance of not Leaving Questions Unanswered?

Experiment: Leave the Last 10 Questions Unanswered
We answered the first 21 questions correctly and then waited for the time to run out, leaving the last 10 questions unanswered. Then we will compare the results to getting the last 10 wrong.


Result: Q39, 28th percentile



Conclusion:
Leaving the questions unanswered carries a significant penalty. My score in this scenario was Q39 compare that with Q49 when I simply answered the last 10 questions wrong. Therefore, even a wrong answer is better than no answer at all! Answering all questions and finishing a test on time is very important. So, it's better to finish the test and answer all the final questions, even if some incorrectly than not finish the test and leave several final questions unanswered. Only scenario 1 (missing first 10 questions) gave a lower score than this case.

What is the Importance of DS questions?

Experiment: Miss all DS after the first 10 Questions
We attempted the first 10 questions correctly and then missed all subsequent DS questions, while answering PS correctly.


Result: Q50. 87th percentile


Conclusion:
We ended up answering the first 10 questions CORRECT and then missed 10 DS while answering 11 PS questions correctly - it was a hybrid case of missing 10 middle questions and the result was very close. In this case the question mix was 18 PS / 13 DS. We ran across 3 DS in the first 10 questions of the test.

What is the Importance of PS questions

Experiment: Miss all PS after the first 10 Questions
We are also going to analyze importance of PS vs. DS. We will attempt answering the first 10 questions correctly and then miss all PS questions, while answering all DS questions correctly.


Result: Q49, 73rd percentile



Conclusion:
There does not appear to be any difference that we can spot based on this scenario between the weights of the PS and DS questions. When answered first 10 questions CORRECT and then missed 11 PS and answering 10 DS questions correctly, the score was exactly the same as missing 10 DS questions or missing 10 middle questions. Questions mix was 18PS / 13DS and 7 of the first 10 questions were PS, consistently with the previous DS exercise.

Takeaways

One of the logical takeaways one can jump to prematurely is to drop everything and focus on the first 10 questions or first 21 questions and forget about the rest. However, that's not the main takeaway or the one that will give you the highest score. We will explain below.

Takeaway #1 do not leave any questions ananswered

Takeaway #2: don't be picky about questions - PS or DS, make sure to answer them as they are treated equally. Don't agonize, just answer one question at a time

Takeaway #3: The first 10 questions are indeed important. There is irrefutable evidence in the shape of Q30 that if you get the first 10 wrong, there is no way back. GMAT is unforgiving if you miss the first 10. However, even if you miss every third questions (meaning you make 3 mistakes in the first 10), you can recover so the first 10 questions are NOT FATAL as long as you have a realistic scenario and don't make a bunch of mistakes.

Takeaway #4: Pay close attention to the first 10 questions but do not sacrifice the test for their sake. You should treat them with higher level of attention and be a bit more careful but don't try to get every question right - it is not worth it.

Takeaway #5: Even if you did not listen to us and decided to get a perfect score on the first 21 questions, giving yourself 3 minutes per question, it is not an easy fit to answer 21 fairly challenging questions correctly even with 3 minutes of time. Chances are if you can answer 21 questions correctly in 60 mins, you can get a Q50 anyway without trying to game the system.

Takeaway #6: Don't worry about gaming the system or question order. You can afford to make mistakes and still get a Q50 or Q49. Some even reported Q51 with a few mistakes so focus on finishing the test on time and well, though do prioritize attention to the first 10 but if you are stuck, just guess and move on.

_________________

Importance of the first and last 9 Questions on the GMAT & Importance of RC and SC



Last year when Vercules ran the what-if scenarios in this thread we found some very interesting results. Not that the GMAT has changed (or just reduced in number and minutes), I decided to make some changes. I thought of doing things in quarters (9 out of 36 questions form a quarter).



1. Importance of the first 9 questions

In the first part of the test, I have tried to see what happens if we miss the first quarter entirely (i.e. the first 9 out of the 36 questions answered wrongly while answering correctly the others). To check for experimental questions and other question level based biases I ran the scenario thrice.

Experiment - GMAC says the first 10 questions are as important as the other questions. We test that theory. We have changed it to 9 because we want to take into account the reduced number of questions in the test.

Methodology - Run 3 instances of A) missing the first 9 questions . B) answering the next 27 questions correctly.

Result - V28, V29, V30

Analysis - To make this analysis more effective, let's first check the scenarios:

First Scenario - Missed the first 9 questions - 4 CRs, 4 SCs and 1 RC - V29 - 57th Percentile
Second Scenario - Missed the first 9 questions - 5 RCs, 3 SCs and 1 CR - V28 - 52nd Percentile
Third Scenario - Missed the first 9 questions - 2 RC, 3 SC and 4 CRs - V30 -59th Percentile

It appears that there's no pattern to when the first RC will appear. In the second test, the RC was as early as the third question and as a result it made me miss the entire RC, which, I think, contributed to the score decline more than the other two tests. Also, while the average difficulty of the test was pretty easy/medium, I did see a few questions with 35%-55% difficulty on GMAT Club which will easily include them in the 700-800 level category.

Last year missing the first 11 questions resulted in the V22 (27th percentile)

2. Importance of the last 9 questions

Experiment In the second part, I decided to see what happens when you mark every question in the test, save the last 9 questions, correctly. Not to my surprise, I got a V40 or above every single time.

Methodology Run 3 instances of missing the last 9 questions on purpose but everything else marked correctly.

Result V40, V42, V42

Analysis In the first scenario the 9 questions I missed were - 1 entire RC (3 questions), 4 SC and 2 CR. In the second scenario, I missed 6 RC questions each. Clearly, even though RC questions are more on the test, missing SC and CR questions wrongly makes a bigger dent in the score.

Side note Last year, missing the 11 questions at the end resulted in a score of V38. So it is definitely a good news for test takers

3. Importance of SC: Guess SC, all else correct

This time we did a sectional test to see what happens if I guess all SC questions but mark everything else correctly.

Methodology: Mark CR and RC questions correctly and mark C for all SC questions.

Result: V37 with 11 questions answered wrong.

4. Importance of RC: Guess RC, everything else correct

In similar fashion, I wanted to check what happens when we guess RC but keep the SC and CR answers correct. Surprising results follow!

Methodology: Mark CR and SC correctly but mark all RC questions C.

Result - SHOCKING V34

Analysis: It is pretty clear that RC matters a whole lot more now than it used to. I would advise with caution against using this strategy. Practice your RCs and go well prepared.

_________________

We can test outcome when one performs exceptionally in some sections but fails abysmally in other section.
The cases such as :
1. SC, CR > 80+, RC<30
2. RC, CR > 80+, SC<30
3. SC, RC> 80+, CR<30

It will be useful for the test takers to know GMAT better.
_________________

Hi,please share the distribution of question types in verbal.Like in quant,I see there are 17-18 PS and 13-14 DS,similarly would love to know number of ques from RC,SC AND CR
Thanks

Posted from my mobile device

Great stuff, Bunuel! Cheers to all your valuable contributions to the Quant forums.


I ran a question-type count on the new (online) Verbal section of Exam 1, and here is what I found: 15 SC, 13 RC, 8 CR.

Make of that what you will! Also note that this balance/frequency of question Verbal types on the new GMATPrep tests does NOT appear to reflect their true proportions on the actual GMAT, based on the debriefs we've read so far, which suggest that RC is now the most common question type:



My best guess for the actual number of counted questions on the GMAT is this: 15 PS, 13 DS, 12 RC, 10 SC, 8 CR.

Keep in mind that I only ran one simulation. It is entirely possible that every GMATPrep test now has a slightly different balance of question types, much like the real GMAT itself, due to the random distribution of experimental questions by type.

These analyses also demonstrate that although the GMAT Official Practice Exams are the obviously most realistic in terms of raw material, the scoring on the real GMAT is clearly different. For example, 2 of your demonstrations earned scores of Q50 despite answering 10 questions wrong, which would never happen on the real GMAT: according to 5 years of ESR evidence, for example, we have never seen more than 6 (counted) questions wrong correlate to a Q50. Similarly, you can answer up to 9 questions wrong (in a row!) on GMATPrep and yet still earn a V42. This scenario would never occur on the real GMAT, either: a V42 score correlates to only 2-5 counted questions incorrect.



For more info: https://gmatclub.com/forum/psa-the-scor ... 67046.html

For this reason, to be safe, I would subtract 20-30 points from the average of your GMAT Official Practice Tests in order to estimate your likely score on the real GMAT.

-Brian
_________________

Any attempts on the myth of skipping questions on verbal? I read that skipping the last questions (up to 4) in verbal insead of guessing might be a better idea (what if you get 3 very easy questions wrong because of guessing vs just skipping them alltogether?)

I think thats a what if worth doing as there are many people strrugling to finish verbal (i have difficulties to finish the section and usually rush or guess the last 4)

I also heard that if you answer the last X questions under Y minutes, the software detects that you are guessing and penalizes you for not managing your time. Is that true?

Thanks for the great content!

Posted from my mobile device

Interesting interpretation. I always thought skipping questions was equivalent to answering them incorrectly. Can someone verify?

You can't "skip" a question on the GMAT. You either get it right or wrong (you have to answer a question in order to move on to the next one), or you run out of time and are forced to omit questions at the end. Despite your suspicions, it appears from previous GMATPrep studies that the opposite is actually true: leaving questions unanswered is even worse than getting them wrong!

The GMAC is very clear that there is a "significant" penalty for not finishing all questions in a section, so if we take GMAC at its word, then it's actually better to answer (some/most/all?) questions at the end of the GMAT incorrectly, than it is to leave them unanswered.


Source: OG 2019

That being said, I don't necessarily always trust the GMAC at its word, so I welcome seeing this hypothesis tested experimentally using the new online tests.

-Brian
_________________

PS vs. DS Question Importance - First Post Updated

Scenario 6: Weight & Importance of DS questions

Experiment: We will try to analyze the weight and importance of DS vs PS in terms of final Quant Score
Methodology: Answer the first 10 questions correctly and then miss all DS questions, while answering PS correctly.

Result: Score - Q50. Percentile - 86th. First 10 questions CORRECT and then missed 10 DS while answering 11 PS questions correctly.
Total Score: every question in the verbal section was answered as "B" this time and it resulted in the final score of V6, 0 percentile. Total score: 460 (19th percentile).
Questions: 18PS and 13DS.

Scenario 7: Weight & Importance of PS questions

Experiment: We will try to analyze the weight and importance of DS vs PS.
Methodology: Answer the first 10 questions correctly and then miss all PS questions, while answering DS correctly.

Result: Score - Q49. Percentile - 75th. First 10 questions CORRECT and then missed 11 PS while answering 10 DS questions correctly.
Total Score: every question in the verbal section was answered as "D" this time and it resulted in the final score of V8, 1 percentile. Total score: 460 (19th percentile).
Questions: 18PS and 13DS.

Conclusion: It seems there is no or little difference in terms of weight that GMAC places on the DS vs. PS questions. Both Scenarios resulted into a very close score: Q50 with 10 DS wrong and Q49 with 11 PS wrong.

_________________

bb OmerPelman

In the official GMAT website under the FAQ section of "What are GMAT® Official Practice Exams and Practice Questions?" it is stated that -
"Practice with real GMAT questions from past exams and full-length practice exams that use the same scoring algorithm as the actual exam."

The link - https://www.mba.com/frequently-asked-qu ... -questions

Thank you for the link. Yes. That is correct - that’s the update that GMAC has made after upgrading to the online platform.

Posted from my mobile device
_________________

Hi ,
can somebody analyse the below mock test result for me because i am clueless how i got V28 , that is too low. I though miss one question in verbal.
Please provide your feedback as how can i improve my verbal score.

Brutal. GMAT, in my opinion, doesn't give you enough chances to redeem yourself after you miss an "easy" question, which is likely what happened to you.

It would be great to look more closely at some of these 3 question CCW strings. You no doubt know the test is adaptive. What may be happening to you, because you get two questions correct in a row, is that you may be tempting the test to deliver the next question at a higher level of difficulty.

Unfortunately, you regularly got that third question wrong, and so have sent the algorithm a clear signal.

You probably know that their algorithm is top-secret, so that any attempt to assign a difficulty level to these questions is purely subjective. However, this would be at least a decent place to start your investigation.

You may also like to read up about what to do next in Verbal, in terms of Return on Time Invested. I go into it at more length: https://www.gmat.amsterdam/gmat_secrets_of_success/gmat-english

Kind Regards,
Omer
_________________

Thank you for this awesome effort.
Is it possible to test something similar to that second scenario?
I'm looking for the result of something like this:
The difference between the effect of having around 10 incorrect answers distributed evenly over questions after number 10 and the effect of having 10 final questions incorrect.

Thanks

Hey, here I am to confirm.
Got 3 RC passages in the first 10 questions. I'm weak at RC so I got 6/10 wrong. Then I got 2 CRs wrong, then 17 Qs correct in a row, one mistake in SC and boom, there it is. 27/36 correct and a result of V33.
This is ridiculous. I am glad I'm reading this post now, thank you OP.

mcelroytutoring,

I hope you're wrong. I literally destroy SC and my accuracy for RC questions is ~30%. That's so sad!!!!

This analysis is GOLD!!!
Was continuously scoring V35-V37 in GMAT Prep, Manhattan and Veritas Tests.
Took the GMAT late December and scored a V40.
And I think this was mainly because of my time-management. Did not leave any question, Left a couple of questions in the middle to balance the time.

Can't thank souvik101990 and Bunuel enough.

So interesting...i feel more confident right now

What about a scenario for verbal in that you miss every 3rd question (Get 2 right, miss 1, get 2 right, miss one etc)? (Like Bunnel's Quant Test)

So I would say that the conclusions are the following:

Quant:

10 first questions are really important
10 last questions are not so important
10 mid questions are not so important
PS is a little more important than DS

Verbal:
RC is the most important section
9 last questions are not so important

Overall Conclusions:

ANSWER ALL QUESTIONS

If I am wrong, please tell me.
_________________