TrueRatr Website Developed By Cornell Researchers Filters Sarcastic Reviews of Apps
Cornell Tech and Bloomberg have collaborated to create a website called TrueRatr that can filter sarcastic reviews of apps and give interested users authentic reviews from competent users. TrueRatr was born out of Cornell University's Tech Challenge where a team of students from all domains, engineering, MBA and design put their minds together to figure out a way of removing useless product reviews most of which were sarcastic in nature. The team of students comprising of Mengjue Wang, Ming Chen, Hesed Kim, Brendan Ritter, Shreyas Kulkarni, and Karan Bir were mentored by Christopher Hong of Bloomberg who had devoted quite a lot of time studying sarcasm for his master’s thesis. He highlights the fact that detecting sarcasm has never been easy because sarcasm cannot be defined by rules. While previous attempts at detecting sarcasm have focused on use of certain words and punctuation, his algorithm looks for "sentiment shift" which is use of positive and negative words in the same phrase.
When the team began working on TrueRatr they took 50 random sarcastic and 50 random non-sarcastic Amazon reviews as their test set and trained a system to conduct sentiment analysis. They were able to achieve 71 percent accuracy. Encouraged by this result the team created Open Sarcasm Project where they asked public to post sarcastic product reviews they found on various portals. After collecting a sample of 1188 reviews they fine-tuned the TrueRatr system and obtained a 75 percent precision score.
Once they had perfected the tool, they wanted it to benefit the public. So they decided to create the TrueRatr website that analyses reviews of Mac OS X and iOS apps on the Apple Store, eliminates sarcastic ones and produces the true review of an app. We tried out the website and found that TrueRatr considers badly composed reviews as sarcastic ones too. For example, we tried finding out true reviews of Tinder (comedic choice, of course) and website picked up “I found my feature bae†as the most sarcastic out of 161 sarcastic reviews. It’s pretty obvious that the reviewer misspelled ‘future’ as ‘feature’.
We could have gone ahead and tested this system for tons of other apps but the website is takes around 10 minutes or more for analysing reviews. If you have time to spare check the website and post any interesting reviews if you find any. If you are a techie, then you can check out the entire coding on <a href="https://github.com/kshreyas91/trueratr" target="_blank" rel="nofollow noopener noreferrer">GitHub - kshreyas91/trueratr</a> and make additions to it too.
Source: <a href="https://arstechnica.com/information-technology/2016/01/snark-attack-cornell-students-teach-software-to-detect-sarcasm/" target="_blank" rel="nofollow noopener noreferrer">Snark attack: Cornell students teach software to detect sarcasm! | Ars Technica</a>
When the team began working on TrueRatr they took 50 random sarcastic and 50 random non-sarcastic Amazon reviews as their test set and trained a system to conduct sentiment analysis. They were able to achieve 71 percent accuracy. Encouraged by this result the team created Open Sarcasm Project where they asked public to post sarcastic product reviews they found on various portals. After collecting a sample of 1188 reviews they fine-tuned the TrueRatr system and obtained a 75 percent precision score.
Once they had perfected the tool, they wanted it to benefit the public. So they decided to create the TrueRatr website that analyses reviews of Mac OS X and iOS apps on the Apple Store, eliminates sarcastic ones and produces the true review of an app. We tried out the website and found that TrueRatr considers badly composed reviews as sarcastic ones too. For example, we tried finding out true reviews of Tinder (comedic choice, of course) and website picked up “I found my feature bae†as the most sarcastic out of 161 sarcastic reviews. It’s pretty obvious that the reviewer misspelled ‘future’ as ‘feature’.
We could have gone ahead and tested this system for tons of other apps but the website is takes around 10 minutes or more for analysing reviews. If you have time to spare check the website and post any interesting reviews if you find any. If you are a techie, then you can check out the entire coding on <a href="https://github.com/kshreyas91/trueratr" target="_blank" rel="nofollow noopener noreferrer">GitHub - kshreyas91/trueratr</a> and make additions to it too.
Source: <a href="https://arstechnica.com/information-technology/2016/01/snark-attack-cornell-students-teach-software-to-detect-sarcasm/" target="_blank" rel="nofollow noopener noreferrer">Snark attack: Cornell students teach software to detect sarcasm! | Ars Technica</a>
0
