Trying to find ways to scrape news...
Hello, hope all is well! I'm currently working on a sentiment classifer system for a greater utilisty of attenuation for market prediction.
Currently, for such a sentiment classifier system, I require a lot of news, for a given topic. Particularly, if I'm trying to predict the market for say Gold, I would require a lot of news on Gold to train the sentiment classifier.
I've tried some ways but it has been quite difficult. GDELT has proven to be quite unfortunate, though I still support it for its amazing work.
Can anyone help me find ways whre I can obtain either the URLs of news for a large span of time for a given topic, or even better the data itself!
I've been also looking into web-scraping, and if someone have perfected a recipe for doing so, given an URL, I would be happy if you could guide me on that!
Thanks!