Class function returns generator
Question:
I’m sure this is a stupid error on my part but I’m sick of debugging so hoping someone can spot the problem. I’ve included the larger script for reference sake but the part that I’m struggling with is the method _get_subcat within Categories. When I attempt to print the results I get [<generator object Category._get_subcat at 0x000002873CFE76F0>]. Can someone explain this to me?
class GamesDict:
def __init__(self):
self._dic = json.load(open('full.json'))['eventGroup']
self.categories = self._dic['offerCategories']
self.export_runline()
self.export_moneyline()
self.export_total()
self.export_propshomeruns()
self.export_propshits()
self.export_propstotalbases()
def export_runline(self):
self.runline = Runline().to_dict(self.categories)
def export_moneyline(self):
self.moneyline = Moneyline().to_dict(self.categories)
def export_total(self):
self.total = Total().to_dict(self.categories)
def export_propshomeruns(self):
self.propshomeruns = BatterPropsHomeruns().to_dict(self.categories)
print(self.propshomeruns)
def export_propshits(self):
self.propshomeruns = BatterPropsHits().to_dict(self.categories)
def export_propstotalbases(self):
self.propshomeruns = BatterPropsTotalBases().to_dict(self.categories)
class Parser:
CATEGORY = int
SUBCATEGORY = int
DISPLAY_NAME: str
def to_dict(self, markets: Iterable[dict[str, Any]]) -> list:
return [market for market in self._get_records(markets)]
@classmethod
def _get_records(cls, data: Iterable[dict[str, Any]]) -> Iterator[tuple]:
raise NotImplementedError()
class Category(Parser):
@classmethod
def _get_records(cls, data: Iterable[dict[str, Any]]) -> tuple[tuple, ...]:
for item in data:
if item['offerCategoryId'] == cls.CATEGORY:
yield from cls._get_subcatdes(item['offerSubcategoryDescriptors'])
@classmethod
def _get_subcatdes(cls, data: Iterable[dict[str, Any]]) -> tuple[tuple, ...]:
for item in data:
if item['subcategoryId'] == cls.SUBCATEGORY:
yield cls._get_subcat(item['offerSubcategory'].get('offers'))
@classmethod
def _get_subcat(cls, data: Iterable[dict[str, Any]]) -> tuple[tuple, ...]:
for item in data:
for x in item:
if x.get('isOpen',()):
yield from cls._get_data(x['outcomes']) # why does this not work?
class Runline(Category):
CATEGORY = 493
SUBCATEGORY = 4519
DISPLAY_NAME = 'Run Line'
@classmethod
def _get_data(cls, market: dict[str, Any]) -> tuple[tuple, ...]:
event_id = market['eventId']
away_team, home_team = market['outcomes'][:2]
away_abbr = TEAMS.get(away_team['label'])
home_abbr = TEAMS.get(home_team['label'])
away_odds = away_team['oddsDecimal']
home_odds = home_team['oddsDecimal']
away_line = away_team['line']
home_line = home_team['line']
return (
(away_abbr, event_id, away_odds, away_line, BOOK),
(home_abbr, event_id, home_odds, home_line, BOOK),
)
class Moneyline(Category):
CATEGORY = 493
SUBCATEGORY = 4519
DISPLAY_NAME = 'Moneyline'
@classmethod
def _get_data(cls, market: dict[str, Any]) -> tuple[tuple, ...]:
event_id = market['eventId']
away_team, home_team = market['outcomes'][:2]
away_abbr = TEAMS.get(away_team['label'])
home_abbr = TEAMS.get(home_team['label'])
away_odds = away_team['oddsDecimal']
home_odds = home_team['oddsDecimal']
return (
(away_abbr, event_id, away_odds, BOOK),
(home_abbr, event_id, home_odds, BOOK),
)
class Total(Category):
CATEGORY = 493
SUBCATEGORY = 4519
DISPLAY_NAME = 'Total'
@classmethod
def _get_data(cls, market: dict[str, Any]) -> tuple[tuple, ...]:
event_id = market['eventId']
away_team, home_team = market['outcomes'][:2]
line = away_team['line']
over = away_team['oddsDecimal']
under = home_team['oddsDecimal']
return (
(event_id, line, over, under, BOOK),
)
class Props(Category):
@classmethod
def _get_data(cls, market: dict[str, Any]) -> tuple[tuple, ...]:
event_id = market['eventId']
cont1, cont2 = market['outcomes'][:2]
player = cont1.get('participant','')
line = cont1.get('line','')
over = cont1.get('oddsAmerican','')
under = cont2.get('oddsAmerican','')
return (
(player, event_id, line, over, under),
)
class BatterPropsHomeruns(Props):
CATEGORY = 743
SUBCATEGORY = 6606
DISPLAY_NAME = 'Home Runs'
class BatterPropsHits(Props):
CATEGORY = 743
SUBCATEGORY = 6606
DISPLAY_NAME = 'Hits'
class BatterPropsTotalBases(Props):
CATEGORY = 743
SUBCATEGORY = 6606
DISPLAY_NAME = 'Total Bases'
GamesDict()
Answers:
I think you meant for _get_subcatdes to yield from cls._get_subcat(...) instead of yield cls._get_subcat(...).
cls._get_subcat is a generator, so cls._get_subcat(...) returns an iterator (over the values yielded by _get_subcat), and that iterator is what _get_subcatdes will yield. yield from cls._get_subcat(...) would cause _get_subcatdes to successively yield the values produced by that iterator, which I think is what you’re expecting.
A couple of notes:
-
One of my debugging heuristics is that if I see that a generator is yielding an iterator instead of the iterator’s values, I make sure that I didn’t somehow forget to use yield from when I meant yield.
-
But since OP went to the trouble of annotating the expected types, a more structured approach would be to actually hand this code over to a static type checker. I used mypy, and I had to make some adjustments to the annotations:
- in
class Parser, I replaced CATEGORY = int with CATEGORY: int (and the same with SUBCATEGORY).
- I changed the return type annotations for the generators from
-> tuple[tuple, ...] to -> Iterator[tuple] (although I suppose I could have been more precise).
- At the suggestion of mypy, I added
from typing import Iterable, Iterator, Any.
I probably could have done some more. That done, mypy reported various problems, two of which seem relevant:
main.py:58: error: Incompatible types in "yield"
(actual type "Iterator[Tuple[Any, ...]]", expected type "Tuple[Any, ...]")
That’s because yield was used instead of yield from.
main.py:64: error: "str" has no attribute "get"
main.py:65: error: Invalid index type "str" for "str";
expected type "Union[SupportsIndex, slice]"
Those (which might answer the question "why does this not work?") are the result of using for x in item where item is a dict, which iterates over the dictionary’s keys (which are of type str). I’m pretty sure the intent was to iterate over item‘s values.
There are lots more errors and warnings which would be useful to attend to, but for me it’s a clear example of the value of static type-checking. Particularly if you’ve already gone to the trouble to annotate.
I’m sure this is a stupid error on my part but I’m sick of debugging so hoping someone can spot the problem. I’ve included the larger script for reference sake but the part that I’m struggling with is the method _get_subcat within Categories. When I attempt to print the results I get [<generator object Category._get_subcat at 0x000002873CFE76F0>]. Can someone explain this to me?
class GamesDict:
def __init__(self):
self._dic = json.load(open('full.json'))['eventGroup']
self.categories = self._dic['offerCategories']
self.export_runline()
self.export_moneyline()
self.export_total()
self.export_propshomeruns()
self.export_propshits()
self.export_propstotalbases()
def export_runline(self):
self.runline = Runline().to_dict(self.categories)
def export_moneyline(self):
self.moneyline = Moneyline().to_dict(self.categories)
def export_total(self):
self.total = Total().to_dict(self.categories)
def export_propshomeruns(self):
self.propshomeruns = BatterPropsHomeruns().to_dict(self.categories)
print(self.propshomeruns)
def export_propshits(self):
self.propshomeruns = BatterPropsHits().to_dict(self.categories)
def export_propstotalbases(self):
self.propshomeruns = BatterPropsTotalBases().to_dict(self.categories)
class Parser:
CATEGORY = int
SUBCATEGORY = int
DISPLAY_NAME: str
def to_dict(self, markets: Iterable[dict[str, Any]]) -> list:
return [market for market in self._get_records(markets)]
@classmethod
def _get_records(cls, data: Iterable[dict[str, Any]]) -> Iterator[tuple]:
raise NotImplementedError()
class Category(Parser):
@classmethod
def _get_records(cls, data: Iterable[dict[str, Any]]) -> tuple[tuple, ...]:
for item in data:
if item['offerCategoryId'] == cls.CATEGORY:
yield from cls._get_subcatdes(item['offerSubcategoryDescriptors'])
@classmethod
def _get_subcatdes(cls, data: Iterable[dict[str, Any]]) -> tuple[tuple, ...]:
for item in data:
if item['subcategoryId'] == cls.SUBCATEGORY:
yield cls._get_subcat(item['offerSubcategory'].get('offers'))
@classmethod
def _get_subcat(cls, data: Iterable[dict[str, Any]]) -> tuple[tuple, ...]:
for item in data:
for x in item:
if x.get('isOpen',()):
yield from cls._get_data(x['outcomes']) # why does this not work?
class Runline(Category):
CATEGORY = 493
SUBCATEGORY = 4519
DISPLAY_NAME = 'Run Line'
@classmethod
def _get_data(cls, market: dict[str, Any]) -> tuple[tuple, ...]:
event_id = market['eventId']
away_team, home_team = market['outcomes'][:2]
away_abbr = TEAMS.get(away_team['label'])
home_abbr = TEAMS.get(home_team['label'])
away_odds = away_team['oddsDecimal']
home_odds = home_team['oddsDecimal']
away_line = away_team['line']
home_line = home_team['line']
return (
(away_abbr, event_id, away_odds, away_line, BOOK),
(home_abbr, event_id, home_odds, home_line, BOOK),
)
class Moneyline(Category):
CATEGORY = 493
SUBCATEGORY = 4519
DISPLAY_NAME = 'Moneyline'
@classmethod
def _get_data(cls, market: dict[str, Any]) -> tuple[tuple, ...]:
event_id = market['eventId']
away_team, home_team = market['outcomes'][:2]
away_abbr = TEAMS.get(away_team['label'])
home_abbr = TEAMS.get(home_team['label'])
away_odds = away_team['oddsDecimal']
home_odds = home_team['oddsDecimal']
return (
(away_abbr, event_id, away_odds, BOOK),
(home_abbr, event_id, home_odds, BOOK),
)
class Total(Category):
CATEGORY = 493
SUBCATEGORY = 4519
DISPLAY_NAME = 'Total'
@classmethod
def _get_data(cls, market: dict[str, Any]) -> tuple[tuple, ...]:
event_id = market['eventId']
away_team, home_team = market['outcomes'][:2]
line = away_team['line']
over = away_team['oddsDecimal']
under = home_team['oddsDecimal']
return (
(event_id, line, over, under, BOOK),
)
class Props(Category):
@classmethod
def _get_data(cls, market: dict[str, Any]) -> tuple[tuple, ...]:
event_id = market['eventId']
cont1, cont2 = market['outcomes'][:2]
player = cont1.get('participant','')
line = cont1.get('line','')
over = cont1.get('oddsAmerican','')
under = cont2.get('oddsAmerican','')
return (
(player, event_id, line, over, under),
)
class BatterPropsHomeruns(Props):
CATEGORY = 743
SUBCATEGORY = 6606
DISPLAY_NAME = 'Home Runs'
class BatterPropsHits(Props):
CATEGORY = 743
SUBCATEGORY = 6606
DISPLAY_NAME = 'Hits'
class BatterPropsTotalBases(Props):
CATEGORY = 743
SUBCATEGORY = 6606
DISPLAY_NAME = 'Total Bases'
GamesDict()
I think you meant for _get_subcatdes to yield from cls._get_subcat(...) instead of yield cls._get_subcat(...).
cls._get_subcat is a generator, so cls._get_subcat(...) returns an iterator (over the values yielded by _get_subcat), and that iterator is what _get_subcatdes will yield. yield from cls._get_subcat(...) would cause _get_subcatdes to successively yield the values produced by that iterator, which I think is what you’re expecting.
A couple of notes:
-
One of my debugging heuristics is that if I see that a generator is yielding an iterator instead of the iterator’s values, I make sure that I didn’t somehow forget to use
yield fromwhen I meantyield. -
But since OP went to the trouble of annotating the expected types, a more structured approach would be to actually hand this code over to a static type checker. I used mypy, and I had to make some adjustments to the annotations:
- in
class Parser, I replacedCATEGORY = intwithCATEGORY: int(and the same withSUBCATEGORY). - I changed the return type annotations for the generators from
-> tuple[tuple, ...]to-> Iterator[tuple](although I suppose I could have been more precise). - At the suggestion of mypy, I added
from typing import Iterable, Iterator, Any.
I probably could have done some more. That done, mypy reported various problems, two of which seem relevant:
main.py:58: error: Incompatible types in "yield" (actual type "Iterator[Tuple[Any, ...]]", expected type "Tuple[Any, ...]")That’s because
yieldwas used instead ofyield from.main.py:64: error: "str" has no attribute "get" main.py:65: error: Invalid index type "str" for "str"; expected type "Union[SupportsIndex, slice]"Those (which might answer the question "why does this not work?") are the result of using
for x in itemwhereitemis adict, which iterates over the dictionary’s keys (which are of typestr). I’m pretty sure the intent was to iterate overitem‘s values.There are lots more errors and warnings which would be useful to attend to, but for me it’s a clear example of the value of static type-checking. Particularly if you’ve already gone to the trouble to annotate.
- in