dask-delayed

How to access nested data in Dask Bag while using dask mongo Question: Below is the sample data – ({‘age’: 61, ‘name’: [‘Emiko’, ‘Oliver’], ‘occupation’: ‘Medical Student’, ‘telephone’: ‘166.814.5565’, ‘address’: {‘address’: ‘645 Drumm Line’, ‘city’: ‘Kennewick’}, ‘credit-card’: {‘number’: ‘3792 459318 98518’, ‘expiration-date’: ’12/23′}}, {‘age’: 54, ‘name’: [‘Wendolyn’, ‘Ortega’], ‘occupation’: ‘Tractor Driver’, ‘telephone’: ‘1-975-090-1672’, ‘address’: {‘address’: …

limit number of CPUs used by dask compute Question: Below code uses appx 1 sec to execute on an 8-CPU system. How to manually configure number of CPUs used by dask.compute eg to 4 CPUs so the below code will use appx 2 sec to execute even on an 8-CPU system? import dask from time …

Parallelizing list filtering Question: I have a list of items that I need to filter based on some conditions. I’m wondering whether Dask could do this filtering in parallel, as the list is very long (a few dozen million records). Basically, what I need to do is this: items = [ {‘type’: ‘dog’, ‘weight’: 10}, …

Dask: Continue with others task if one fails Question: I have a simple (but large) task Graph in Dask. This is a code example results = [] for params in SomeIterable: a = dask.delayed(my_function)(**params) b = dask.delayed(my_other_function)(a) results.append(b) dask.compute(**results) Here SomeIterable is a list of dict, where each are arguments to my_function. In each iteration …

Dask: How to return a tuple of futures in client.submit Question: I need to return a tuple from a task which has to be unpacked in the main process because each element of the tuple will go to different dask tasks. I would like to avoid unnecessary communication so I think that the tuple elements …