WordSkew Text Analysis Software (Mac/Windows)
Budget: $750 – $1,500 USD
Basic operation. The user loads a large corpus (10s millions of words at least) and indicates how sentences, paragraphs, section are marked in the corpus. An index is formed efficiently.
The user then searches for a word or phrase or part-of-speech tag(s) and the program shows the frequency of occurrence in, for example, different positions in the sentence – 1st word, 2nd word, etc.. Or first tenth, 2nd tenth, etc. Similarly for paragraphs, sections and complete file. The results are given as a histogram/graph and a table. Above the table are some basic statistics – total number of hits, size of the corpus. Clicking on a bar of the graph or line of the table brings up the concordance lines for that position. The concordance lines can be saved to a file. In the concordance window it is also possible to see the collocates (2l, 1L, 1R, 2R).
The program has two options – counting words and counting characters (mainly for use for Chinese).
Searches can be for words, regex or Part-of-speech. (Regex version should allow metacharacters \w etc.)
A batch search option sends the results to a file named by the user.
Code needs to be commented. Keeping track of all the loops is complex.
Memory management and speed of processing a large set of files is important.
Output will be single stand-alone exec files for Mac or Windows.
A working version of the program (Delphi) and other info can be downloaded from wordskew.com. The interface can be improved on the current version. Note that I am aiming to start from the beginning and create a better version.
Videos
Demo -- https://www.youtube.com/watch?v=OPKWbj_H4Qg
loading files https://www.youtube.com/watch?v=4FwIK_U3Q5U
Searching https://www.youtube.com/watch?v=fbEnP4plnLU
The user then searches for a word or phrase or part-of-speech tag(s) and the program shows the frequency of occurrence in, for example, different positions in the sentence – 1st word, 2nd word, etc.. Or first tenth, 2nd tenth, etc. Similarly for paragraphs, sections and complete file. The results are given as a histogram/graph and a table. Above the table are some basic statistics – total number of hits, size of the corpus. Clicking on a bar of the graph or line of the table brings up the concordance lines for that position. The concordance lines can be saved to a file. In the concordance window it is also possible to see the collocates (2l, 1L, 1R, 2R).
The program has two options – counting words and counting characters (mainly for use for Chinese).
Searches can be for words, regex or Part-of-speech. (Regex version should allow metacharacters \w etc.)
A batch search option sends the results to a file named by the user.
Code needs to be commented. Keeping track of all the loops is complex.
Memory management and speed of processing a large set of files is important.
Output will be single stand-alone exec files for Mac or Windows.
A working version of the program (Delphi) and other info can be downloaded from wordskew.com. The interface can be improved on the current version. Note that I am aiming to start from the beginning and create a better version.
Videos
Demo -- https://www.youtube.com/watch?v=OPKWbj_H4Qg
loading files https://www.youtube.com/watch?v=4FwIK_U3Q5U
Searching https://www.youtube.com/watch?v=fbEnP4plnLU