Is Web Scraping possible using Outsystems ?

Is Web Scraping possible using Outsystems ?

Is Web Scraping possible using Outsystems ? 

Thanks & Regards

It is, Ravi. Have you taken a look at HTTPRequestHandler and Xml extensions?

Those 2 should get you started:
  1. HTTPRequestHandler allows you to fetch a page with GET or POST method (if parameters are necessary just pass them through accordingly or with this same extension's other actions);
  2. Then work your way through the HTML page with the other extension. Xml extension even has an XPath feature that eases your work.
Let me know if this worked for you.
Hi Ravi and Pedro,

The ardoHTTP component is way nicer (and powerful) than HTTPRequestHandler.
Kudos to Ricardo Silva for the great work!

How could I forget that precious component?!!? It has saved me tons of work. Thanks Davide! I second my recommendation to yours.
Thank you Pedro and Davide for your valuable suggestions. I will take a look at them and keep you posted.
Those suggestions by Pedro and Davide are the way to go for small / medium complexity scenarios.

However, if web scraping is the central feature of your project, with lots of complex source sites, then you'll be better with an existing dedicated product. I can't recommend one myself, since I've briefly analyzed this a long time ago, but if you search in google you'll see a bunch of dedicated products for web scraping.
...if you want to play around it (pet project) you can also try to grab some code out there. I once wrapped this sample into an Extension: and had lot's of fun with it :)
Paulo -

That specialized tool is called "Perl". ;)

My first full time job out of college was writing Web scraping filters in Perl for a meta search engine...