Question:

How to write specific dictionary list for each cycle of loop?

Problem:

1st, I am VERY new to Python so please don't take my head off :) I've been reading and trying all day to get this to work, but I just can't figure it out.

I have a list:allLeagues = [] that is created from scraping tabs of the webpage on a sports site (list changes throughout the day) which contains ex.

['NFL', 'NFL1Q', 'MLB', 'SOCCER', 'MLBLIVE', 'NFL1H', 'LoL', 'NBASZN', 'CSGO', 'KBO', 'CS2', 'TENNIS', 'MMA', 'CRICKET']

I then cycle through each league/tab and grab everything I need:

AllPlayers = []

for league in leagues:

driver.find_element(By.XPATH, f"//div[@class='name'][normalize-space()='{league}']").click()

time.sleep(2)

# Wait for the stat-container element to be present and visible

stat_container = WebDriverWait(driver, 10.until(EC.visibility_of_element_located((By.CLASS_NAME, "stat-container")))

# Find all stat elements within the stat-container

# i.e. categories is the list ['Points','Rebounds',...,'Turnovers']

categories = driver.find_element(By.CSS_SELECTOR, ".stat-container").text.split('\n')

# Iterate over each stat element

for category in categories:

# Click the stat element

line = '-'*len(category)

print(line + '\n' + category + '\n' + line)

driver.find_element(By.XPATH, f"//div[text()='{category}']").click()

projections = WebDriverWait(driver, .until(EC.presence_of_all_elements_located((By.CSS_SELECTOR, ".projection")))

for projection in projections:

names = projection.find_element(By.XPATH, './/div[@class="name"]').text

points= projection.find_element(By.XPATH, './/div[@class="presale-score"]').get_attribute('innerHTML')

text = projection.find_element(By.XPATH, './/div[@class="text"]').text.replace('\n','')

print(names, points, text)

players = {'Name': names, 'Prop':points, 'Line':text}

AllPlayers.append(players)

which cycles through each league/tab and grabs everything I need.

Where I am getting confused now is that I would like to append each cycle of the loop to separate lists. I can get it all into one list, but I would like each different league's data to be in it's own list. For example, when scraping the data from the NFL tab, to append to NFL[] in the dictionary.

Everything I have been reading has said to create a dictionary, so I tried:

list_dict = {}

for league in allLeagues:

list_dict[league] = []

print(list_dict)

which returns this: Dictionary Print (tried posting code, but SO said my post looked like spam, and also can't upload images yet here since I'm so new) Not exactly sure what I'm doing wrong there, but I'm assuming all I need is the last line of that dictionary:

{'NFL': [], 'NFL1Q': [], 'MLB': [], 'SOCCER': [], 'MLBLIVE': [], 'NFL1H': [], 'LoL': [], 'NBASZN': [], 'CSGO': [], 'KBO': [], 'CS2': [], 'TENNIS': [], 'MMA': [], 'CRICKET': []}

SO that's where I'm at. Trying to understand my error in creating the dictionary from the list. And also appending the scraped data from the loop into each separate dictionary entry/list.

Any help is greatly appreciated! Thank you!

Solution:

You can do it something like this:

leagues = { league: [] for league in allLeagues}

for league, players in leagues.items():

...

for category in categories:

....

players = {'Name': names, 'Prop':points, 'Line':text}

players.append(players)

leagues.items() will take out the key and value of the dict. So it will return you the name of the league and the (in the beginning) empty list belonging to that league which we call players in the for loop.

Then in the end you just append the players you've scraped to that list.

A concept to be aware of here is immutability vs mutability. Basically python lists are mutable, so you can take out the reference to the list in the for loop, add elements to it with append and these changes will be reflected in the dictionary even if you don't write the full list back to the dictionary.

See here for example for more details on that concept, it's pretty important in Python and it can often lead to issues/misunderstandings.

Suggested blogs:

>How to Use RTK Queries in a React App?

>How you can create array with associative array in other array php?

>How you can send email from a shared inbox in Python

>How you can Show or hide elements in React?

>Instance deployment failed to install Composer dependencies

>Integrate SASS into the build and consume a UI toolkit

>Invoking Python script from scons and pass ARGLIST

>Migrate From Haruko To AWS App: 5 Simple Steps

>PHP cURL to upload video to azure blob storage

>PHP Error Solved: htaccess problem with an empty string in URL