Python Regular Expression and Data Wrangling
Budget: $10 – $30 USD
I'm trying to extract data from a semi-structured text file about twitter data. The text file contains information about the tweets, i.e., “user name”, “user code”, “user description”, “number of followers”, “whether or not the user account is verified”, “date of the tweet”, and the “tweet text”. I need to extract the data from the text file and transform the data into a XML format with the following elements:
1. users: this tag wraps all the users
2. user: this tag wraps all the tweets from a particular user and keeps the meta data for
each user such as number of followers, verified or not, user description etc. If a user has
multiple tweets, the meta data of the latest tweet (i.e., the tweet with the most
recent date) must be used.
3. Tweets: wraps all the tweets of a specific user
4. tweet: for each user, this tag represents the text of the user tweet
So far I've managed to design the Regex required to extract all the necessary information and read it into separate lists (user_names, verified, description, no_followers, tweets, dates)
I'm stuck now with combining those lists without repeating the values and with using the metadata of the recent tweet. Here is an example of two tweets from the same user:
Tweet 1:
$user_name.: Kosmos
$user_code.: 100010075
$user_desc.: https://t.co/diXbRhdZdS
$No. followers.: 104.0 $verified?.: False $tweet_date.: 2021-02-13 17:51:00
$tweet_text.: #Bitcoin Adoption: #Apple #Pay Can Be Used To #Spend #BTC - https://t.co/6ZBVnZ4xE1
The mainstream #adoption of dig… https://t.co/71kuPx4km7
Tweet 2:
$user_name.: Kosmos
$user_code.: 100010075
$userdescription.: https://t.co/diXbRhdZdS
$followerNo.: 155.0 $verified?.: False $tweet_date.: 2021-04-10 12:02:44
$tweet.: $447 #Million In #BTC #Shorts #Liquidated in Past Four #Hours as #Bitcoin #Blasts Past $61,000 -… https://t.co/7J8lHoWdmA
The metadata from the second tweet should be in the final list because it has the most recent date, in addition to the tweet text from both tweets. (THIS HAS TO BE DONE WITHOUT datetime MODULE)
I will include a snippet of what I've come up so far, If you need any additional files or the text file please let me know.
Thank you!
1. users: this tag wraps all the users
2. user: this tag wraps all the tweets from a particular user and keeps the meta data for
each user such as number of followers, verified or not, user description etc. If a user has
multiple tweets, the meta data of the latest tweet (i.e., the tweet with the most
recent date) must be used.
3. Tweets: wraps all the tweets of a specific user
4. tweet: for each user, this tag represents the text of the user tweet
So far I've managed to design the Regex required to extract all the necessary information and read it into separate lists (user_names, verified, description, no_followers, tweets, dates)
I'm stuck now with combining those lists without repeating the values and with using the metadata of the recent tweet. Here is an example of two tweets from the same user:
Tweet 1:
$user_name.: Kosmos
$user_code.: 100010075
$user_desc.: https://t.co/diXbRhdZdS
$No. followers.: 104.0 $verified?.: False $tweet_date.: 2021-02-13 17:51:00
$tweet_text.: #Bitcoin Adoption: #Apple #Pay Can Be Used To #Spend #BTC - https://t.co/6ZBVnZ4xE1
The mainstream #adoption of dig… https://t.co/71kuPx4km7
Tweet 2:
$user_name.: Kosmos
$user_code.: 100010075
$userdescription.: https://t.co/diXbRhdZdS
$followerNo.: 155.0 $verified?.: False $tweet_date.: 2021-04-10 12:02:44
$tweet.: $447 #Million In #BTC #Shorts #Liquidated in Past Four #Hours as #Bitcoin #Blasts Past $61,000 -… https://t.co/7J8lHoWdmA
The metadata from the second tweet should be in the final list because it has the most recent date, in addition to the tweet text from both tweets. (THIS HAS TO BE DONE WITHOUT datetime MODULE)
I will include a snippet of what I've come up so far, If you need any additional files or the text file please let me know.
Thank you!