Image Reconstruction
Budget: ₹12,500 – ₹37,500 INR
Introduction:(Youtube Videos
Deep image reconstruction from human brain activity (Paper Explained)
Self-Supervision in Natural-Image Reconstruction from fMRI
Papers:
[1] Deep image reconstruction from human brain activity
[2] From voxels to pixels and back: Self-supervision in natural-image reconstruction from fMRI
Problem 1: what is the problem the two papers aim to solve, and why is this problem important or interesting? (5 points)
Problem 2: 1) summarize the two methods, including high-level ideas as well as technical details: the relevant details that are important to focus on (e.g., if there’s a model, define it; if there is a theorem, state it and explain why it’s important, etc) 2) what are the major differences of the two methods? (15 points)
Problem 3: reproduce experimental results in [1]. You need to show both the stimulus and the reconstructed images (Figure 2 and Figure 6 of [1]). (30 points)
Reference code: https://github.com/KamitaniLab/End2EndDeepImageReconstruction
Problem 4: reproduce experimental results in [2]. You need to show both the ground truth and the reconstructed images (Figure 5 of [2]). (30 points)
Reference code: http://www.wisdom.weizmann.ac.il/~vision/ssfmri2im/
Problem 5: run both methods on “Generic Object Decoding” dataset and test them on Deep Image Reconstruction dataset to see if the models have strong generalization ability. (30 points)
Problem 6: which method is better based on your results of Problem 3, Problem 4, and Problem 5? What is the main reason why it performs better (theoretically and practically)? (20 points)
Problem 7: Based on your answer to Problem 2 and Problem 6, what are the major obstacles of the task of image reconstruction from fMRI? Can you think of ways to address them? (20 points)
Deep image reconstruction from human brain activity (Paper Explained)
Self-Supervision in Natural-Image Reconstruction from fMRI
Papers:
[1] Deep image reconstruction from human brain activity
[2] From voxels to pixels and back: Self-supervision in natural-image reconstruction from fMRI
Problem 1: what is the problem the two papers aim to solve, and why is this problem important or interesting? (5 points)
Problem 2: 1) summarize the two methods, including high-level ideas as well as technical details: the relevant details that are important to focus on (e.g., if there’s a model, define it; if there is a theorem, state it and explain why it’s important, etc) 2) what are the major differences of the two methods? (15 points)
Problem 3: reproduce experimental results in [1]. You need to show both the stimulus and the reconstructed images (Figure 2 and Figure 6 of [1]). (30 points)
Reference code: https://github.com/KamitaniLab/End2EndDeepImageReconstruction
Problem 4: reproduce experimental results in [2]. You need to show both the ground truth and the reconstructed images (Figure 5 of [2]). (30 points)
Reference code: http://www.wisdom.weizmann.ac.il/~vision/ssfmri2im/
Problem 5: run both methods on “Generic Object Decoding” dataset and test them on Deep Image Reconstruction dataset to see if the models have strong generalization ability. (30 points)
Problem 6: which method is better based on your results of Problem 3, Problem 4, and Problem 5? What is the main reason why it performs better (theoretically and practically)? (20 points)
Problem 7: Based on your answer to Problem 2 and Problem 6, what are the major obstacles of the task of image reconstruction from fMRI? Can you think of ways to address them? (20 points)