R software
Budget: ₹1,500 – ₹2,500 INR
Suppose we have two documents, D1 and D2, each with 4 sentences. An algorithm is designed
to automatically select two sentences from the two documents to answer a question.
This algorithm works in the following steps.
• First, it randomly selects one document from either D1 or D2. Each document is equally likely to
be selected. Let d
(1) be the document that is selected.
• Next, it randomly selects a sentence among the first three sentences from d
(1), with probabilities
0.5 for the first sentence, 0.4 for the second sentence, and 0.1 for the third sentence. Let s
(1) be
this sentence.
• Third, it randomly selects a document from either D1 or D2, exactly as in the first step. Let d
(2)
be the selected document.
• Lastly, it selects a sentence s
(2) from d
(2)
.
– If d
(2) is the same as where the first sentence is from (i.e., d
(1) = d
(2)), the algorithm always
selects the sentence immediately following s
(1) in its original document.
– If the documents are not the same (i.e., d
(1) ≠ d
(2)), the algorithm randomly selects a sentence
with equal probabilities from d
(2)
.
If this algorithm is run twice independently, what is the probability that the two runs generate the
same answer (exact same two sentences s
(1), s(2) in the same order)? Estimate this probability by
simulating 10000 samples of two runs: two independent runs of the algorithm is considered as one
simulated sample. In other words, the algorithm should be run 20000 times in total in your simulation
2. Read section 8.3.5 Fisher’s exact test in the textbook. Using the same data as given in Figure
8.15, estimate the one-sided p-value from simulation. Specifically, run 100000 simulations of the fecal
infusion study under the null hypothesis. Estimate the probabilities of all the tables that are as or more
extreme than the actual study result (given in Figure 8.15). Note: please simulate the randomized trial,
not sampling from a known distribution. Hint: because the null hypothesis follows a known distribution,
you can check that your estimates are consistent with it.
to automatically select two sentences from the two documents to answer a question.
This algorithm works in the following steps.
• First, it randomly selects one document from either D1 or D2. Each document is equally likely to
be selected. Let d
(1) be the document that is selected.
• Next, it randomly selects a sentence among the first three sentences from d
(1), with probabilities
0.5 for the first sentence, 0.4 for the second sentence, and 0.1 for the third sentence. Let s
(1) be
this sentence.
• Third, it randomly selects a document from either D1 or D2, exactly as in the first step. Let d
(2)
be the selected document.
• Lastly, it selects a sentence s
(2) from d
(2)
.
– If d
(2) is the same as where the first sentence is from (i.e., d
(1) = d
(2)), the algorithm always
selects the sentence immediately following s
(1) in its original document.
– If the documents are not the same (i.e., d
(1) ≠ d
(2)), the algorithm randomly selects a sentence
with equal probabilities from d
(2)
.
If this algorithm is run twice independently, what is the probability that the two runs generate the
same answer (exact same two sentences s
(1), s(2) in the same order)? Estimate this probability by
simulating 10000 samples of two runs: two independent runs of the algorithm is considered as one
simulated sample. In other words, the algorithm should be run 20000 times in total in your simulation
2. Read section 8.3.5 Fisher’s exact test in the textbook. Using the same data as given in Figure
8.15, estimate the one-sided p-value from simulation. Specifically, run 100000 simulations of the fecal
infusion study under the null hypothesis. Estimate the probabilities of all the tables that are as or more
extreme than the actual study result (given in Figure 8.15). Note: please simulate the randomized trial,
not sampling from a known distribution. Hint: because the null hypothesis follows a known distribution,
you can check that your estimates are consistent with it.
Related categories:
R Programming Language