In today’s video we are going to be working on a real-life Python script for Devops.

Many websites in the world today are protected against DDoS attacks by Cloudflare.
Cloudflare provides this protection by serving as an intermediary to all traffic from the web. So instead of a website visitor connecting to the server hosting the website, directly, the visitor connects to one of the servers from Cloudflare. In this way Cloudflare can filter all the requests which look suspicious.

But this only works if an attacker doesn’t bypass the servers from Cloudflare!

If an attacker discovers the IP address of the server hosting the website, then it could attack the website directly!

For that reason, we need to prevent this by whitelisting all the IP addresses from Cloudflare, so that even if an attacker knows what the real IP address of our website, they will not be able to attack it so easily, because their IP address will not be included in the whitelisted list of IP addresses.

Cloudflare provides a url https://www.cloudflare.com/ips-v4 and https://www.cloudflare.com/ips-v6 to download the range of IP addresses(CIDR) used by Cloudflare servers.

We are going to be using Python to read the content of those urls and return as a result a list of strings containing all the CIDR for IPV4 and IPV6.

We will also create a list of unit tests to ensure that our code always returns a non empty list of valid CIDR IP addresses.

def getIPAddresses(url):
      headers = {
        'User-Agent': 'My User Agent 1.0',
        'From': '[email protected]'  # This is another valid field
      }
      result = []
      try:
        req = urllib.request.Request(url, headers=headers)
        with urllib.request.urlopen(req) as f:
            ips = f.read().decode('utf-8').split("n")
            result = ips
      except urllib.error.URLError as e:
        print(e.reason)

      return result      

import urllib
import urllib.request

def whitelistCloudflareIps():
    ips_v4 = getIPAddresses("https://www.cloudflare.com/ips-v4")
    ips_v6 = getIPAddresses("https://www.cloudflare.com/ips-v6")
    return ips_v4 + ips_v6

whitelistCloudflareIps()

['173.245.48.0/20',
 '103.21.244.0/22',
 '103.22.200.0/22',
 '103.31.4.0/22',
 '141.101.64.0/18',
 '108.162.192.0/18',
 '190.93.240.0/20',
 '188.114.96.0/20',
 '197.234.240.0/22',
 '198.41.128.0/17',
 '162.158.0.0/15',
 '104.16.0.0/13',
 '104.24.0.0/14',
 '172.64.0.0/13',
 '131.0.72.0/22',
 '2400:cb00::/32',
 '2606:4700::/32',
 '2803:f800::/32',
 '2405:b500::/32',
 '2405:8100::/32',
 '2a06:98c0::/29',
 '2c0f:f248::/32']

import unittest
import re

class TestGetIpAddressesFromCloudflare(unittest.TestCase):

  def test_whitelistCloudflareIpsForIpv4(self):
    ips = getIPAddresses("https://www.cloudflare.com/ips-v4")
    self.assertTrue(len(ips)> 0)

  def test_whitelistCloudflareIpsForIpv6(self):
    ips = getIPAddresses("https://www.cloudflare.com/ips-v6")
    self.assertTrue(len(ips)> 0)

  def test_whitelistCloudflareIpsForIpv4ShouldReturnValidIpRanges(self):
    ips = getIPAddresses("https://www.cloudflare.com/ips-v4")
    self.assertTrue(len(ips)> 0)
    for ip in ips:
      pattern_ipv4_cidr = "d+.d+.d+.d+/d+"
      result = re.fullmatch(pattern_ipv4_cidr, ip)
      self.assertNotEqual(result, None)

  def test_whitelistCloudflareIpsForIpv6ShouldReturnValidIpRanges(self):
    ips = getIPAddresses("https://www.cloudflare.com/ips-v6")
    self.assertTrue(len(ips)> 0)
    for ip in ips:
      pattern_ipv6_cidr = "(d|[a-f])+:(d|[a-f])+::/d+"
      result = re.fullmatch(pattern_ipv6_cidr, ip)
      self.assertNotEqual(result, None)

if __name__ == '__main__':
   unittest.main(argv=['first-arg-is-ignored'], exit=False)

....
----------------------------------------------------------------------
Ran 4 tests in 0.161s

OK

Resources

Recommended Courses for Data Science

• Learn Python for Beginners 👉🏼 Python for Everybody
• Learn Deep Learning with Andrew Ng 👉🏼 Neural Networks and Deep Learning by Andrew Ng
• Learn Data Science with Coursera Plus 👉🏼 Coursera Plus For Data Science

Videos

Source Code

https://github.com/armindocachada/python-for-beginners-exercises/blob/main/Cloudflare.ipynb