Mine Texts From Wikipedia Dumps Python Script
Budget: $30 – $115 USD
I am looking for a freelancer that would create for me 2 python scripts that will take some texts from Wikipedia dumps and save them in the format of Parallel Corpus as 2 separate files in which line numbers would correspond to each other. For example:
FILE1 FILE2
Sentence01 Translated_Senentence1
Sentence02 Translated_Senentence2
Sentence03 Translated_Senentence3
.
.
.
Script 1 should use interwiki links to find articles in 2 languages that are linked and download their titles and save them as a corpus. 1 title one line, UTF-8 encoding.
Script 2 should use interwiki links to find images, tables or figures which are the same in wikipedias in both languages and save their captions as a text corpus. 1 caption one line, UTF-8 encoding.
FILE1 FILE2
Sentence01 Translated_Senentence1
Sentence02 Translated_Senentence2
Sentence03 Translated_Senentence3
.
.
.
Script 1 should use interwiki links to find articles in 2 languages that are linked and download their titles and save them as a corpus. 1 title one line, UTF-8 encoding.
Script 2 should use interwiki links to find images, tables or figures which are the same in wikipedias in both languages and save their captions as a text corpus. 1 caption one line, UTF-8 encoding.