Python script for multi-text classification using transformer(s) on custom trained dataset
Budget: $100 – $300 AUD
Hi,
I'm after a python script that I can use to:
1. Custom train a transformer such as Bert, AlBert, GPT etc on a custom data set. These datasets are customer service transcripts of either calls, emails or chat. This data will be supplied in CSV format as a series of documents where each row is a document. The basic header of the file would be: [ID, Text].The key here is I don't want to use any existing pre-trained models. I want to train each model for a specific use case with a large subset of sample text data.
2. With this trained model I then need to be able to generate document embeddings for a classified dataset. Basically, I need to be able to feed another CSV file into the model. This file would have a unique identifier for the document that enables it to map to a classification or other meta data, or it could also contain the classification as either a categorical attribute or binary. Against this data set I need to be able to output a file that maps document embeddings to each of those unique ID's for the classified dataset. This output can be saved as CSV. The sample headers for this doc might look something like: [ID, Text, Classification, Dimension 1, ..., Dimension 512] (or as configured by the transformer output).
3. Similarly I need to be able to generate embeddings for unseen data. This could effectively be the same step as above if the Classification field is omitted. Again, being able to parse through a CSV file with [ID, Text] and output [ID, Text, Dimension 1, ...., Dimension 512]. If it simplifies the script these steps could be the same as the mapping to the classification can occur separately
The specific choice of the architecture is up to you. The script just needs to comment key steps in the workflow and where any configuration needs to occur. In summary I want to generate document embeddings using a transformer trained on a custom dataset.
It will be difficult for me to supply exact samples of data so some substitution will need to occur for your testing as long as the data structure is correct. I can supply one sample set of data to help with the process.
I'm after a python script that I can use to:
1. Custom train a transformer such as Bert, AlBert, GPT etc on a custom data set. These datasets are customer service transcripts of either calls, emails or chat. This data will be supplied in CSV format as a series of documents where each row is a document. The basic header of the file would be: [ID, Text].The key here is I don't want to use any existing pre-trained models. I want to train each model for a specific use case with a large subset of sample text data.
2. With this trained model I then need to be able to generate document embeddings for a classified dataset. Basically, I need to be able to feed another CSV file into the model. This file would have a unique identifier for the document that enables it to map to a classification or other meta data, or it could also contain the classification as either a categorical attribute or binary. Against this data set I need to be able to output a file that maps document embeddings to each of those unique ID's for the classified dataset. This output can be saved as CSV. The sample headers for this doc might look something like: [ID, Text, Classification, Dimension 1, ..., Dimension 512] (or as configured by the transformer output).
3. Similarly I need to be able to generate embeddings for unseen data. This could effectively be the same step as above if the Classification field is omitted. Again, being able to parse through a CSV file with [ID, Text] and output [ID, Text, Dimension 1, ...., Dimension 512]. If it simplifies the script these steps could be the same as the mapping to the classification can occur separately
The specific choice of the architecture is up to you. The script just needs to comment key steps in the workflow and where any configuration needs to occur. In summary I want to generate document embeddings using a transformer trained on a custom dataset.
It will be difficult for me to supply exact samples of data so some substitution will need to occur for your testing as long as the data structure is correct. I can supply one sample set of data to help with the process.