Add Crawlera proxy to pyton script | Using crawlera proxy

Crawlera is one of proxy used to rotate our IP's. Proxies are beneficial when we are doing web extraction kind of stuff.

Below is small example in how to use crawlera proxy with our python script which also works for AWS lambda functions :

import requests
from requests.packages.urllib3.exceptions import InsecureRequestWarning
requests.packages.urllib3.disable_warnings(InsecureRequestWarning)

class JobCreator():
      def getProxies():
          url = "http://httpbin.org/ip"
          proxy_host = "proxy.crawlera.com"
          proxy_port = "8010"
          proxy_auth = "yourauthkey"
          proxies = {"https": "https://{}@{}:{}/".format(proxy_auth, proxy_host, proxy_port),
                "http": "http://{}@{}:{}/".format(proxy_auth, proxy_host, proxy_port)}
          return proxies

      def call(self):
         proxies = getProxies()
         html = requests.get('https://api.myip.com', verify=False, proxies=proxies)
         print(html.content)

# Lambda Handler
def handler(event, context):
    print("Job Creation Initiated.")
    obj = JobCreator()
    obj.call(event)
    print("Jobs Created Successfully.")

#handler('test','test')
#call()

To know more on how to use AWS lambda functions yo can leave messages in below comment box.