Python 3.4 set returning duplicates : I'm pulling a list of URL's off a census website than putting them in a set to make sure I don't end up with duplicates, then exporting that list of non-duplicate URL's into a .csv file. However, my set continues to return duplicate values, which shouldn't be possible. Here's my code:

import bs4

from bs4 import BeautifulSoup

import requests

import csv

source_link = "https://www.census.gov/data/tables/2016/demo/popest/state-total.html"

s = requests.get(source_link)

usable_html = s.text


setupsoup = BeautifulSoup(usable_html, 'lxml')

silver = csv.writer(open("WGUCSV.csv", "r+"))

silver.writerow(["URL"])

for set(gold) in setupsoup.findAll('a', href=True):

	gold.add['href']

	print (gold)

	silver.writerow(gold)



As a bonus question, I also need a way to convert my resulting relative URL's into absolute URL's, preferably BEFORE sorting them into a non-duplicated list. I really thought adding them all to a set would filter out duplicates on it's own.