BAD TESTCASE gives random output due to order by NULL values

  • 0

    When i use either of the given two solution approaches to submit, the result is quite weird: it get random acceptance. if you try to submit same solution for multiple times, you will find it is either accepted or reject b/c of wrong answer, in a random fashion. I looked into the wrong answer test case it is always the one below:

    '''{"headers": {"survey_log": ["uid", "action", "question_id", "answer_id", "q_num", "timestamp"]},"rows": {"survey_log": [[5, "answer", 285, 123, 1, 1], [5, "answer", 285, 123, 1, 2], [5, "answer", 369, 123, 2, 3], [5, "skip", 369, null, 2, 4]]}}'''

    This test cases is problematic b/c neither question_id 285 nor 369 has action 'show', so that the ratio defined by (num_answer/num_show) will both give null values for 285 and 369, which means if you order them by this ratio, it will give random order.

    You can check this out using the "custom testcase" option and "run code" of solution on this test case. you will find both "your answer" and "expected answer" varies from run to run, which leads to random acceptance of your submission. (here i removed 'limit 1' in the test code to show the full order)
    3_1508202669378_output1.png 2_1508202669378_output2.png 1_1508202669378_output3.png 0_1508202669378_output4.png

    This problem needs to remove this testcase or add new rules for case where no 'show' action in any question_id

    btw, the provided "COUNT(IF...)" approach needs to be corrected: COUNT(IF(action = 'show', 1, 0) ) --->COUNT(IF(action = 'show', 1, null)

Log in to reply

Looks like your connection to LeetCode Discuss was lost, please wait while we try to reconnect.