Advance Java Application
Budget: $10 – $30 USD
In this assignment you will do some basic natural language processing (NLP). You will write a program
which will construct a language model (for several different languages) and, when asked to classify a text,
should select the language whose model is closer to the text in question. The language model (described
below) will be constructed from text files. You will be required to achieve this task without using any loops
(use streams and lambdas instead). After reading all your content into your representation structure, you
also need to process said content via multithreading and update the model concurrently.
Text Classification
We can finally wrap the whole problem. Whenever presented with the necessary data (local folder, all
the language folder, all the files in each folder, the text file to be classified and the n-gram dimension
value), your program should construct a model for every language and the mystery text to be classified.
Compute the similarity between the mystery text and each language and select the language with the
highest similarity (equivalently smallest angle distance).
Non-functional requirement
Your program must ideally not contain any loops (for, while, do-while) at all. For iteration needs use
streams. Also keep in mind that streams use lambdas and lambdas must be kept short and easy to read.
All the language folders must be processed concurrently. All the text files in the folder must be processed
concurrently. This means that multiple thread reading different documents of the same language, will try
to access the language model data structures concurrently. You need to apply synchronization techniques
to ensure thread-safety and ultimately correct behavior of the program.
which will construct a language model (for several different languages) and, when asked to classify a text,
should select the language whose model is closer to the text in question. The language model (described
below) will be constructed from text files. You will be required to achieve this task without using any loops
(use streams and lambdas instead). After reading all your content into your representation structure, you
also need to process said content via multithreading and update the model concurrently.
Text Classification
We can finally wrap the whole problem. Whenever presented with the necessary data (local folder, all
the language folder, all the files in each folder, the text file to be classified and the n-gram dimension
value), your program should construct a model for every language and the mystery text to be classified.
Compute the similarity between the mystery text and each language and select the language with the
highest similarity (equivalently smallest angle distance).
Non-functional requirement
Your program must ideally not contain any loops (for, while, do-while) at all. For iteration needs use
streams. Also keep in mind that streams use lambdas and lambdas must be kept short and easy to read.
All the language folders must be processed concurrently. All the text files in the folder must be processed
concurrently. This means that multiple thread reading different documents of the same language, will try
to access the language model data structures concurrently. You need to apply synchronization techniques
to ensure thread-safety and ultimately correct behavior of the program.