You are browsing as a guest. Sign up (or log in) to start making projects!

59m 47s logged

SO MANY BUGS!

So, my scraper is working perfectly on my laptop but sometimes when I run it on the nest server it does not work for specific websites. For example, for some food network recipes it seems to be redirecting me to an alternative link. My hypothesis for this is that it thinks i’m coming from the uk (perhaps server location) and is trying to redirect me to the uk website. Hence, my logs show for a lot of the failed attempts the page title being “Recipe and Global Dishes | Food Network UK”. Next steps for me may include retrieving the url to see if a redirect occured, and perhaps checking if I can configure the headless chrome i’m using to show as being a static country such as always being in the us… We will see…

0
3

Comments 0

No comments yet. Be the first!